The HOME directory may not be available in containers. It's also the
wrong variable on some platforms. Use os.path.expanduser() instead of
HOME.
Fixes: https://pagure.io/freeipa/issue/7837
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Fixes a test ipatests/test_cmdline/test_cli.py:test_cli_fs_encoding()
which sets IPA_CONFDIR and attempts to interpret the resulting error
message. However, if the test is run on an enrolled machine (a
developer's laptop, for example), check_client_configuration() will
succeed because it ignores IPA_CONFDIR and, as result, api.finalize()
will fail later with a stacktrace.
Pass an environment object and test an overridden config file existence
in this case to fail with a proper and expected message.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Get the value of `PAGER` environment variable in case it's defined, check the executable, if it exists - use a pager, otherwise - print function.
Fixes: https://pagure.io/freeipa/issue/7746
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
pylint 2.2.0 has a new checker for unnecessary pass statements. There is
no need to have a pass statement in functions or classes with a doc
string.
Fixes: https://pagure.io/freeipa/issue/7772
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Thomas Woerner <twoerner@redhat.com>
If the server responds 401 and the response body is empty, the
client raises ResponseNotReady. This occurs because:
1. For a non-200 response, the response read only if the
Content-Length header occurs.
2. The response must be read before another request (e.g. the
follow-up request with WWW-Authenticate header set), and this
condition was not met. For details see
https://github.com/python/cpython/blob/v3.6.7/Lib/http/client.py#L1305-L1321.
This situation should not arise in regular use, because the client
either has a session cookie, or, knowing the details of the server
it is contacting, it establishes the GSS-API context and includes
the WWW-Authenticate header in the initial request.
Nevertheless, this problem has been observed in the wild. I do not
know its ordinary cause(s), but one can force the issue by removing
an authenticated user's session cache from /run/ipa/ccaches, then
performing a request.
Resolve the issue by always reading the response. It is safe to
call response.read() regardless of whether the Content-Length header
appears, or whether the body is empty.
Fixes: https://pagure.io/freeipa/issue/7752
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Some incorrect errors are possible if a client installation
fails and a configuration rollback is required.
These include:
1. Unconfigured automount client failed: CalledProcessError(Command
['/usr/sbin/ipa-client-automount', '--uninstall', '--debug']
returned non-zero exit status 1: '')
Caused by check_client_configuration() not returning the correct
return value (2).
2. WARNING: Unable to revert to the pre-installation state ('authconfig'
tool has been deprecated in favor of 'authselect'). The default sssd
profile will be used instead.
The authconfig arguments would have been: authconfig --disableldap
--disablekrb5 --disablesssdauth --disablemkhomedir
If installation fails before SSSD is configured there is no state
to roll back to. Detect this condition.
3. An error occurred while removing SSSD's cache.Please remove the
cache manually by executing sssctl cache-remove -o.
Again, if SSSD is not configured yet then there is no cache to
remove. Also correct the missing space after the period.
https://pagure.io/freeipa/issue/7729
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Replace six.moves and six.StringIO/BytesIO imports with cannonical
Python 3 packages.
Note: six.moves.input behaves differently than builtin input function.
Therefore I left six.moves.input for now.
See: https://pagure.io/freeipa/issue/7715
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
In Python 3, six.integer_types is (int,). In most places, the alias can
be simply replaced with int. In other places, it was possible to
simplify the code by unpacking the tuple.
See: https://pagure.io/freeipa/issue/7715
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
In Python 3, six.string_types is just an alias for str.
See: https://pagure.io/freeipa/issue/7715
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Python 2 had old style and new style classes. Python 3 has only new
style classes. There is no point to subclass from object any more.
See: https://pagure.io/freeipa/issue/7715
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
tox / pytest is complaining about lots and lots of invalid escape
sequences in our code base. Sprinkle raw strings or backslash escapes
across the code base to fix most occurences of:
DeprecationWarning: invalid escape sequence
There is still one warning that keeps repeating, though:
source:264: DeprecationWarning: invalid escape sequence \d
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
pyasn1 0.4 changed handling of ANY containers in a backwards
incompatible way. For 0.3.x, keep explicit wrap and unwrap in octet
strings for ANY container members. For >= 0.4, let pyasn1 do the job.
This patch also makes sorting of extended_key_usage_bytes() stable and
adds tests.
Tested with pyasn1 0.3.7 and 0.4.4.
Fixes: https://pagure.io/freeipa/issue/7685
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
The replica_file option is not needed anymore. Threfore the option can
be removed.
See: https://pagure.io/freeipa/issue/7689
Signed-off-by: Thomas Woerner <twoerner@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The replica_file option is only supported for DL0. The option will be
marked deprecated for now.
See: https://pagure.io/freeipa/issue/7669
Signed-off-by: Thomas Woerner <twoerner@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
With increasing the minimal domain level to 1 ipa-replica-install will
refuse to install if the domain has domain level 0.
See: https://pagure.io/freeipa/issue/7669
Signed-off-by: Thomas Woerner <twoerner@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Fix the following violations aiming to support Pylint 2.0
- `unneeded-not` (C0113):
Consider changing "not item in items" to "item not in items" used
when a boolean expression contains an unneeded negation.
- `useless-import-alias` (C0414):
Import alias does not rename original package Used when an import
alias is same as original package.e.g using import numpy as numpy
instead of import numpy as np
- `raising-format-tuple` (W0715):
Exception arguments suggest string formatting might be intended Used
when passing multiple arguments to an exception constructor, the
first of them a string literal containing what appears to be
placeholders intended for formatting
- `bad-continuation` (C0330):
This was already included on the disable list, although with current
version of pylint (2.0.0.dev2) violations at the end of the files
are not being ignored.
See: https://github.com/PyCQA/pylint/issues/2278
- `try-except-raise` (E0705):
The except handler raises immediately Used when an except handler
uses raise as its first or only operator. This is useless because it
raises back the exception immediately. Remove the raise operator or
the entire try-except-raise block!
- `consider-using-set-comprehension` (R1718):
Consider using a set comprehension Although there is nothing
syntactically wrong with this code, it is hard to read and can be
simplified to a set comprehension.Also it is faster since you don't
need to create another transient list
- `dict-keys-not-iterating` (W1655):
dict.keys referenced when not iterating Used when dict.keys is
referenced in a non-iterating context (returns an iterator in
Python 3)
- `comprehension-escape` (W1662):
Using a variable that was bound inside a comprehension Emitted when
using a variable, that was bound in a comprehension handler, outside
of the comprehension itself. On Python 3 these variables will be
deleted outside of the comprehension.
Issue: https://pagure.io/freeipa/issue/7614
Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
pylinti 2 is having a hard time to handle name mangled, magic attributes
correctly. Double under attributes like __d are internally renamed to
_Env__d. After multiple failed attempts, it was easier to just add more
pylint disable to the implementation.
pylint 2 also thinkgs that Env.server is defined much later or the env
doesn't have that member at all. Ignore the false warnings, too.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Armando Neto <abiagion@redhat.com>
THe ipa console command takes an optional filename argument. The
filename argument was broken, because the implementation passed a file
object to exec() instead of a string or compiled object.
ipa console now uses compile() to compile the code with print_function
__future__ feature.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
ipa console is a useful tool to use FreeIPA's API in an interactive
Python console. The patch adds readline tab completion and history
support.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
In order to support pylint 2.0 the following violations must be fixed:
- `chained-comparison` (R1716):
Simplify chained comparison between the operands This message is
emitted when pylint encounters boolean operation like
"a < b and b < c", suggesting instead to refactor it to "a < b < c".
- `consider-using-in` (R1714):
Consider merging these comparisons with "in" to %r To check if a
variable is equal to one of many values,combine the values into a
tuple and check if the variable is contained "in" it instead of
checking for equality against each of the values.This is faster
and less verbose.
Issue: https://pagure.io/freeipa/issue/7614
Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Aiming to support pylint 2.0 some functions and methods must have their
return statements updated in order to fix two new violations:
- `useless-return` (R1711):
Useless return at end of function or method Emitted when a single
"return" or "return None" statement is found at the end of function
or method definition. This statement can safely be removed because
Python will implicitly return None
- `inconsistent-return-statements` (R1710):
Either all return statements in a function should return an
expression, or none of them should. According to PEP8, if any return
statement returns an expression, any return statements where no value
is returned should explicitly state this as return None, and an
explicit return statement should be present at the end of the
function (if reachable)
Issue: https://pagure.io/freeipa/issue/7614
Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
During parallel replica installation, a request sometimes fails with
CA_REJECTED or CA_UNREACHABLE. The error occur when the master is
either busy or some information haven't been replicated yet. Even
a stuck request can be recovered, e.g. when permission and group
information have been replicated.
A new function request_and_retry_cert() automatically resubmits failing
requests until it times out.
Fixes: https://pagure.io/freeipa/issue/7623
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
- Add missing executable bits to all scripts
- Remove executable bits from all files that are not scripts,
e.g. js, html, and Python libraries.
- Remove Python shebang from all Python library files.
It's frown upon to have executable library files in site-packages.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Armando Neto <abiagion@redhat.com>
Python 3 has moved all collection abstract base classes to
collections.abc. Python 3.7 started to deprecate the old aliases.
The whole import block needs to be protected with import-error and
no-name-in-module, because Python 2 doesn't have collections.abc module and
collections.abc.Mapping, while Python 3 doesn't have collections.Mapping.
Fixes: https://pagure.io/freeipa/issue/7609
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Update regular expression validator to prevent user and group creation.
Fixes: https://pagure.io/freeipa/issue/7572
Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
In CIS hardened mode, the process umask is 027. This results in some
files not being world readable. Ensure that write_certificate_list()
calls in client installer, server installer, and upgrader create cert
bundles with permission bits 0644.
Fixes: https://pagure.io/freeipa/issue/7594
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Instead of multiple timeout values all over the code base, all
replication waits now use a common timeout value from api.env of 5
minutes. Waiting for HTTP/replica principal takes 90 to 120 seconds, so
5 minutes seem like a sufficient value for slow setups.
Fixes: https://pagure.io/freeipa/issue/7595
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
For now translation classes have old style % formatting way only.
But 'format' is convenience, preferred in Python3 string formatting method.
Fixes: https://pagure.io/freeipa/issue/7586
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
While enabling console output in the server installation the
"Allow trust agents to retrieve keytab keys for cross realm
principals" ACI was throwing an unparseable error because
it has a subkey which broke parsing (the extra semi-colon):
userattr="ipaAllowedToPerform;read_keys#GROUPDN";
The regular expression pattern needed to be updated to handle
this case.
Related: https://pagure.io/freeipa/issue/6760
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
The server installer had no console logger set so print
statements were used for communication. Now that a logger
is enabled the extra prints need to be dropped.
A number of logger.info statements have been upgraded
to debug since they do not need to appear on the console
by default.
https://pagure.io/freeipa/issue/6760
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
On multiple occasions, SRV query answers were not properly sorted by
priority. Records with same priority weren't randomized and shuffled.
This caused FreeIPA to contact the same remote peer instead of
distributing the load across all available servers.
Two new helper functions now take care of SRV queries. sort_prio_weight()
sorts SRV and URI records. query_srv() combines SRV lookup with
sort_prio_weight().
Fixes: https://pagure.io/freeipa/issue/7475
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Pylint3 falsely reports warning W1662: using a variable that was bound
inside a comprehension for the cases where the same name is reused for a
loop after the comprehension in question.
Rename the variable in a loop to avoid it.
If the code looks like the following:
arr = [f for f in filters if callable(f)]
for f in arr:
result = result + f()
pylint3 would consider 'f' used outside of comprehension. Clearly, this
is a false-positive warning as the second 'f' use is completely
independent of the comprehension's use of 'f'.
Reviewed-By: Aleksei Slaikovskii <aslaikov@redhat.com>
mod_ssl's limiting of client cert verification depth was causing
the replica installs to fail when master had been installed with
external CA since the SSLCACertificateFile was pointing to a file
with more than one certificate. This is caused by the default
SSLVerifyDepth value of 1. We set it to 5 as that should be
just about enough even for possible sub-CAs.
https://pagure.io/freeipa/issue/7530
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
In Python 3, cryptography requires certificate data to be binary. Even
PEM encoded files are treated as binary content.
certmap-match and cert-find were loading certificates as text files. A
new BinaryFile type loads files as binary content.
Fixes: https://pagure.io/freeipa/issue/7520
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Add absolute_import from __future__ so that pylint
does not fail and to achieve python3 behavior in
python2.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
With the recent encryption of the HTTPD keys, it's also necessary
to count with this scenario during upgrade and create the password
for the HTTPD private key along the cert/key pair.
This commit also moves the HTTPD_PASSWD_FILE_FMT from ipalib.constants
to ipaplatform.paths as it proved to be too hard to be used that way.
https://pagure.io/freeipa/issue/7421
Reviewed-By: Christian Heimes <cheimes@redhat.com>
This commit adds configuration for HTTPD to encrypt/decrypt its
key which we currently store in clear on the disc.
A password-reading script is added for mod_ssl. This script is
extensible for the future use of directory server with the
expectation that key encryption/decription will be handled
similarly by its configuration.
https://pagure.io/freeipa/issue/7421
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
When ipa host-add --random is unable to add a reverse record (for instance
because the server does not manage any reverse zone), the command
adds the host but exits (return code=1) with an error without actually
outputing the random password generated.
With this fix, the behavior is modified. The commands succeeds (return code=0)
but prints a warning.
This commit also adds a unit test.
https://pagure.io/freeipa/issue/7374
Reviewed-By: Christian Heimes <cheimes@redhat.com>
This will cause the command to continue with no password set
at all which is not what we want.
We want to loop forever until the passwords match or the
user gives up and types ^D or ^C.
https://pagure.io/freeipa/issue/7383
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Part of the mod_nss -> mod_ssl move. This patch allows loading
necessary certificates for Apache to function from PKCS#12 files.
This should fix CA-less and domain level 0 installations.
Related: https://pagure.io/freeipa/issue/3757
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
This was useful during initial development and as a simple
in-tree unit test but it isn't needed anymore.
Related: https://pagure.io/freeipa/issue/3757
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
FileStore.backup_file() docstring claimed not to store a
copy of the same file but the behavior of the method did not
match this description.
This commit makes the backed-up file filename derivation
deterministic by hashing its content by SHA-256, thus it
should not back up two files with the same filename and content.
Reviewed-By: Aleksei Slaikovskii <aslaikov@redhat.com>
The Python 3 refactoring effort is finishing, it should be safe
to turn all scripts to run in Python 3 by default.
https://pagure.io/freeipa/issue/4985
Reviewed-By: Christian Heimes <cheimes@redhat.com>
"ipa-server-install --uninstall" no longer fails with error message
"'Env' object has no attribute 'basedn'" when executed on a system that
has no freeIPA server installation.
Fixes: https://pagure.io/freeipa/issue/7063
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Felipe Volpone <fbarreto@redhat.com>
Use Python 3's reprlib with customizations to create same API.txt under
Python 2 and 3. Some plugins have been slightly altered to use stable
sorting for dynamically created parameter lists.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The 'CertificateInvalid' message is used for malformed certificates.
The user error messages says "Invalid certificate...", but in X.509
"validity" has a specific meaning that does not encompass
well-formedness. For clarify, change the user-visible message to
say "Malformed".
Part of: https://pagure.io/freeipa/issue/7390
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
ipa rpc server did set the LANG environment variable on each
request and it was not thread safe which led to unpredictable
mixed languages output. Also, there were mistakes regarding
setting the Accept-Language HTTP header.
Now on each request we're setting the "languages" property
in the context thread local variable and client is setting
the Accept-Language HTTP header correctly.
Also, as the server is caching the schema and the schema can
be generated for several languages it's good to store different
schema fingerprint for each language separately.
pagure: https://pagure.io/freeipa/issue/7238
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
