For now JSON service is not available without authentication
to IPA. But some of Web UI pages expect translations before
or without Login process. This endpoint serves i18n requests
only.
Fixes: https://pagure.io/freeipa/issue/7559
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Petr Vobornik <pvoborni@redhat.com>
Globally disabling the following violations:
- `assignment-from-no-return` (E1111):
Assigning to function call which doesn't return. Used when an
assignment is done on a function call but the inferred function
doesn't return anything.
- `keyword-arg-before-vararg` (W1113):
Keyword argument before variable positional arguments list in the
definition of %s function When defining a keyword argument before
variable positional arguments, one can end up in having multiple
values passed for the aforementioned parameter in case the method is
called with keyword arguments.
Locally disabling the following:
- `subprocess-popen-preexec-fn` (W1509):
Using preexec_fn keyword which may be unsafe in the presence of
threads The preexec_fn parameter is not safe to use in the presence
of threads in your application. The child process could deadlock
before exec is called. If you must use it, keep it trivial! Minimize
the number of libraries you call into.
https://docs.python.org/3/library/subprocess.html#popen-constructor
Fixed violations:
- `bad-mcs-classmethod-argument` (C0204):
Metaclass class method %s should have %s as first argument Used when
a metaclass class method has a first argument named differently than
the value specified in valid-metaclass-classmethod-first-arg option
(default to "mcs"), recommended to easily differentiate them from
regular instance methods.
- Note: Actually `cls` is the default first arg for `__new__`.
- `consider-using-get` (R1715):
Consider using dict.get for getting values from a dict if a key is
present or a default if not Using the builtin dict.get for getting a
value from a dictionary if a key is present or a default if not, is
simpler and considered more idiomatic, although sometimes a bit slower
Issue: https://pagure.io/freeipa/issue/7614
Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Fix the following violations aiming to support Pylint 2.0
- `unneeded-not` (C0113):
Consider changing "not item in items" to "item not in items" used
when a boolean expression contains an unneeded negation.
- `useless-import-alias` (C0414):
Import alias does not rename original package Used when an import
alias is same as original package.e.g using import numpy as numpy
instead of import numpy as np
- `raising-format-tuple` (W0715):
Exception arguments suggest string formatting might be intended Used
when passing multiple arguments to an exception constructor, the
first of them a string literal containing what appears to be
placeholders intended for formatting
- `bad-continuation` (C0330):
This was already included on the disable list, although with current
version of pylint (2.0.0.dev2) violations at the end of the files
are not being ignored.
See: https://github.com/PyCQA/pylint/issues/2278
- `try-except-raise` (E0705):
The except handler raises immediately Used when an except handler
uses raise as its first or only operator. This is useless because it
raises back the exception immediately. Remove the raise operator or
the entire try-except-raise block!
- `consider-using-set-comprehension` (R1718):
Consider using a set comprehension Although there is nothing
syntactically wrong with this code, it is hard to read and can be
simplified to a set comprehension.Also it is faster since you don't
need to create another transient list
- `dict-keys-not-iterating` (W1655):
dict.keys referenced when not iterating Used when dict.keys is
referenced in a non-iterating context (returns an iterator in
Python 3)
- `comprehension-escape` (W1662):
Using a variable that was bound inside a comprehension Emitted when
using a variable, that was bound in a comprehension handler, outside
of the comprehension itself. On Python 3 these variables will be
deleted outside of the comprehension.
Issue: https://pagure.io/freeipa/issue/7614
Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
When multiple replicas are installed in parallel, two replicas may try
to create the cn=replica entry at the same time. This leads to a
conflict on one of the replicas. replica_config() and
ensure_replication_managers() now handle conflicts.
ipaldap now maps TYPE_OR_VALUE_EXISTS to DuplicateEntry(). The type or
value exists exception is raised, when an attribute value or type is
already set.
Fixes: https://pagure.io/freeipa/issue/7566
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Thierry Bordaz <tbordaz@redhat.com>
In order to support pylint 2.0 the following violations must be fixed:
- `chained-comparison` (R1716):
Simplify chained comparison between the operands This message is
emitted when pylint encounters boolean operation like
"a < b and b < c", suggesting instead to refactor it to "a < b < c".
- `consider-using-in` (R1714):
Consider merging these comparisons with "in" to %r To check if a
variable is equal to one of many values,combine the values into a
tuple and check if the variable is contained "in" it instead of
checking for equality against each of the values.This is faster
and less verbose.
Issue: https://pagure.io/freeipa/issue/7614
Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Aiming to support pylint 2.0 some functions and methods must have their
return statements updated in order to fix two new violations:
- `useless-return` (R1711):
Useless return at end of function or method Emitted when a single
"return" or "return None" statement is found at the end of function
or method definition. This statement can safely be removed because
Python will implicitly return None
- `inconsistent-return-statements` (R1710):
Either all return statements in a function should return an
expression, or none of them should. According to PEP8, if any return
statement returns an expression, any return statements where no value
is returned should explicitly state this as return None, and an
explicit return statement should be present at the end of the
function (if reachable)
Issue: https://pagure.io/freeipa/issue/7614
Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
During parallel replica installation, a request sometimes fails with
CA_REJECTED or CA_UNREACHABLE. The error occur when the master is
either busy or some information haven't been replicated yet. Even
a stuck request can be recovered, e.g. when permission and group
information have been replicated.
A new function request_and_retry_cert() automatically resubmits failing
requests until it times out.
Fixes: https://pagure.io/freeipa/issue/7623
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
There is a mismatch between 'i18n' krb_auth_msg and 'LoginScreen'
widget kerberos_msg. The former links to "unauthorized.html", but the latter
to "ssbrowser.html". Both should link to "ssbrowser.html" page.
Fixes: https://pagure.io/freeipa/issue/7624
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Tune 389-DS replication settings to improve performance and avoid
timeouts. During installation of a replica, the value of
nsDS5ReplicaBindDnGroupCheckInterval is reduced to 2 seconds. At the end
of the installation, the value is increased sensible production
settings. This avoids long delays during replication.
See: https://pagure.io/freeipa/issue/7617
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
The method IPASystemRecords.get_locations_records() has a race condition.
The IPASystemRecords object creates a mapping of server names to server
data. get_locations_records() uses server_find() again to get a list of
servers, but then operates on the cached dict of server names.
In parallel replication case, the second server_find() call in
get_locations_records() can return additional servers. Since the rest of
the code operates on the cached data, the method then fails with a KeyError.
server_data is now an OrderedDict to keep same sorting as with
server_find().
Fixes: https://pagure.io/freeipa/issue/7566
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
Calls to `os.fsync(f.fileno())` need to be accompained by `f.flush()`.
Commit 8bbeedc93f introduces the helper
`ipapython.ipautil.flush_sync()`, which handles all calls in the right
order.
However, `flush_sync()` takes as parameter a file object with fileno
and name, where name must be a path to the file, this isn't possible
in some cases where file descriptors are used.
Issue: https://pagure.io/freeipa/issue/7251
Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Commit 7284097eed kept
find_providing_servers('CA') call before enable_services(). Therefore the
list of known CA servers did not contain the current replica.
ipa-replica-install on the first replica with --setup-ca still printed
the CA topology warning.
See: https://pagure.io/freeipa/issue/7566
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Service entries in cn=FQDN,cn=masters,cn=ipa,cn=etc are no longer
created as enabled. Instead they are flagged as configuredService. At
the very end of the installer, the service entries are switched from
configured to enabled service.
- SRV records are created at the very end of the installer.
- Dogtag installer only picks fully installed servers
- Certmonger ignores all configured but not yet enabled servers.
Fixes: https://pagure.io/freeipa/issue/7566
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
When installing multiple replicas in parallel, one replica may create
SRV entries for other replicas, although the replicas aren't fully
installed yet. This may cause some services to connect to a server, that
isn't ready to serve requests.
The DNS IPASystemRecords framework now skips all servers that aren't
ready IPA masters.
See: https://pagure.io/freeipa/issue/7566
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
server_find and server_role plugin were hiding IPA master role
information. It's now possible to fetch IPA master role information and
to filter by IPA master role, e.g. to ignore servers that have some
services configured but not (yet) enabled.
See: https://pagure.io/freeipa/issue/7566
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Fix `ERROR 'str' object has no attribute 'decode'` when --zonemgr is
passed to ipa-server-install.
Solution copied from commit 75d26e1f01,
function `ipaserver.install.bindinstance.zonemgr_callback` duplicates
the behavior of the method affected by this patch.
Issue: https://pagure.io/freeipa/issue/7612
Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
- Add missing executable bits to all scripts
- Remove executable bits from all files that are not scripts,
e.g. js, html, and Python libraries.
- Remove Python shebang from all Python library files.
It's frown upon to have executable library files in site-packages.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Armando Neto <abiagion@redhat.com>
Python 3 has moved all collection abstract base classes to
collections.abc. Python 3.7 started to deprecate the old aliases.
The whole import block needs to be protected with import-error and
no-name-in-module, because Python 2 doesn't have collections.abc module and
collections.abc.Mapping, while Python 3 doesn't have collections.Mapping.
Fixes: https://pagure.io/freeipa/issue/7609
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
ipaldap's LDAPClient client turns INVALID_CREDENTIAL error into
ACIError. Catch the ACIError and wait until the user has been
replicated.
Apparently no manual or automated test ran into the timeout during
testing.
Fixes: Fixes: https://pagure.io/freeipa/issue/7593
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
When AD forest A has a trust with a forest B that claims ownership
of a domain name (TLN) owned by an IPA forest, we need to build
exclusion record for that specific TLN, not our domain name.
Use realmdomains to find a correct exclusion entry to build.
Fixes: https://pagure.io/freeipa/issue/7370
Reviewed-By: Armando Neto <abiagion@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
The directory /etc/httpd/alias contains public key material. It must be
world readable and executable, so any client can read public certs.
Note: executable for a directory means, that a process is allowed to
traverse into the directory.
Fixes: https://pagure.io/freeipa/issue/7594
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
In CIS hardened mode, the process umask is 027. This results in some
files not being world readable. Ensure that write_certificate_list()
calls in client installer, server installer, and upgrader create cert
bundles with permission bits 0644.
Fixes: https://pagure.io/freeipa/issue/7594
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
DogtagInstance.setup_admin and related methods have multiple LDAP
replication race conditions. The bugs can cause parallel
ipa-replica-install to fail.
The code from __add_admin_to_group() has been changed to use MOD_ADD
ather than search + MOD_REPLACE. The MOD_REPLACE approach can lead to
data loss, when more than one writer changes a group.
setup_admin() now waits until both admin user and group membership have
been replicated to the master peer. The method also adds a new ACI to
allow querying group member in the replication check.
Fixes: https://pagure.io/freeipa/issue/7593
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Instead of multiple timeout values all over the code base, all
replication waits now use a common timeout value from api.env of 5
minutes. Waiting for HTTP/replica principal takes 90 to 120 seconds, so
5 minutes seem like a sufficient value for slow setups.
Fixes: https://pagure.io/freeipa/issue/7595
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
replication.wait_for_entry() now can wait for an attribute value to
appear on a replica.
Fixed timeout handling caused by bad rounding and comparison. For small
timeouts, the actual time was rounded down. For example for 60 seconds
timeout and fast replica, the query accumulated to about 0.45 seconds
plus 60 seconds sleep. 60.45 is large enough to terminate the loop
"while int(time.time()) < timeout", but not large enough to trigger the
exception in "if int(time.time()) > timeout", because int(60.65) == 60.
See: https://pagure.io/freeipa/issue/7593
Fixes: https://pagure.io/freeipa/issue/7595
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
For now formatting is applied for bare messages before translating.
This breaks python-brace-format and message becomes untranslatable
at all.
Fixes: https://pagure.io/freeipa/issue/7586
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
For now formatting is applied for bare messages before translating.
This breaks python-brace-format and message becomes untranslatable
at all.
Fixes: https://pagure.io/freeipa/issue/7586
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Translation objects have support for format(). This allows to
get rid of unicode() which has been removed in Python3.
Also some messages to be translated at request time should
not use format()
Fixes: https://pagure.io/freeipa/issue/7586
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
For now formatting is applied for bare messages before translating.
This breaks python-brace-format and message becomes untranslatable
at all.
Also some messages to be translated at request time should
not use format().
Fixes: https://pagure.io/freeipa/issue/7586
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
For now formatting is applied for bare messages before translating.
This breaks python-brace-format and message becomes untranslatable.
Fixes: https://pagure.io/freeipa/issue/7586
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
ipa-replica-install only set ca_host in its temporary
/etc/ipa/default.conf, when it wasn't installing a replica with CA. As a
consequence, the replica installer was picking a random CA server from
LDAP.
Always set the replication peer as ca_host. This will ensure that the
installer uses the same replication peer for CA. In case the replication
peer is not a CA master, the installer will automatically pick another
host later.
See: https://pagure.io/freeipa/issue/7566
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
The server installer had no console logger set so print
statements were used for communication. Now that a logger
is enabled the extra prints need to be dropped.
A number of logger.info statements have been upgraded
to debug since they do not need to appear on the console
by default.
https://pagure.io/freeipa/issue/6760
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
The server installation and uninstallation overlaps both the
server and client installers. The output could be confusing
with a server uninstall finishing with the message:
The ipa-client-install command was successful
This was in part due to the fact that the server was not
configured with a console format and verbose was False which
meant that no logger messages were displayed at all.
In order to suppress client installation errors and avoid
confusion add a list of errors to ignore. If a server install
was not successful and hadn't gotten far enough to do the
client install then we shouldn't complain loudly about it.
https://pagure.io/freeipa/issue/6760
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
On multiple occasions, SRV query answers were not properly sorted by
priority. Records with same priority weren't randomized and shuffled.
This caused FreeIPA to contact the same remote peer instead of
distributing the load across all available servers.
Two new helper functions now take care of SRV queries. sort_prio_weight()
sorts SRV and URI records. query_srv() combines SRV lookup with
sort_prio_weight().
Fixes: https://pagure.io/freeipa/issue/7475
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Increase the WSGI daemon worker process count from 2 processes to 5
processes. This allows IPA RPC to handle more parallel requests. The
additional processes increase memory consumption by approximante 250 MB
in total.
Since memory is scarce on 32bit platforms, only 64bit platforms are
bumped to 5 workers.
Fixes: https://pagure.io/freeipa/issue/7587
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The command description is taken from python docstring. Thus
commands should have them and should include the callings of
gettext to be translated.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
The schema compat plugin is disabled on upgrades but it is
possible that it is not configured at all and this will
produce a rather nasty looking error message.
Check to see if it is configured at all before trying to
disable it.
https://pagure.io/freeipa/issue/6610
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Sometimes named-pkcs11 is not being stopped or reloaded during
uninstall and it causes a lot of problems while testing, for example,
backup and restore tests are failing because of ipa-server-install
fails on checking DNS step.
Fixes backup/restore tests runs. Maybe something else.
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Stanislav Levin
Armando Neto
Christian Heimes
Rob Crittenden
Justin Stephenson
Alexander Bokovoy
Armando Neto
Aleksei Slaikovskii