In ipa-replica-install, many components use their own instance
of the FileStore to backup configuration files to the pre-install
state. This causes issues when the calls are mixed, like for
instance:
ds.do_task1_that_backups_file (using ds.filestore)
http.do_task2_that_backups_file (using http.filestore)
ds.do_task3_that_backups_file (using ds.filestore)
because the list of files managed by ds.filestore does not include
the files managed by http.filestore, and the 3rd call would remove
any file added on 2nd call.
The symptom of this bug is that ipa-replica-install does not save
/etc/httpd/conf.d/ssl.conf and subsequent uninstallation does not
restore the file, leading to a line referring to ipa-rewrite.conf
that prevents httpd startup.
The installer should consistently use the same filestore.
Fixes https://pagure.io/freeipa/issue/7684
Reviewed-By: Christian Heimes <cheimes@redhat.com>
ipa-server-install is checking if the forwarder(s) specified with
--forwarder argument support DNSSEC. When the --no-dnssec-validation
option is added, the installer should not perform the check.
Fixes: https://pagure.io/freeipa/issue/7666
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
Setting ca_host caused replication failures on DL0
because it was trying to connect to wrong CA host.
Trying to avoid corner-case in ipaserver/plugins/dogtag.py
when api.env.host nor api.env.ca_host had not CA configured
and there was ca_host set to api.env.ca_host variable.
See: https://pagure.io/freeipa/issue/7566
Resolves: https://pagure.io/freeipa/issue/7629
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Commit 811b0fdb46 introduced a regression
when configuring replication with a master < 3.3
Even if 389-ds schema is extended with nsds5ReplicaReleaseTimeout,
nsds5ReplicaBackoffMax and nsDS5ReplicaBindDnGroupCheckInterval
attributes, it will return UNWILLING_TO_PERFORM when a mod
operation is performed on the cn=replica entry.
This patch ignores the error and logs a debug msg.
See: https://pagure.io/freeipa/issue/7617
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Some commands print tracebacks or unclear error message when
they are called on a machine where ipa packages are installed but
IPA is not configured.
Consistently report 'IPA is not configured on this system' in this
case.
Related to https://pagure.io/freeipa/issue/6261
Reviewed-By: Christian Heimes <cheimes@redhat.com>
--no-host-dns is supposed to avoid all DNS lookups so pass
this as the force value when creating the host in a replica
installation.
https://pagure.io/freeipa/issue/7656
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
There was an issue with ipa-server-upgrade and it was
showing an error while upgrading:
DN... does not exists or haven't been updated, caused
by not moving pointer to file begining when re-reading.
Resolves: https://pagure.io/freeipa/issue/7644
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
The old domain level check to suggest to use ipa-replica-prepare has been
converted to make sure that domain fulfills minimal domain level
requirement (no DL0).
See: https://pagure.io/freeipa/issue/7669
Signed-off-by: Thomas Woerner <twoerner@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
As there is the minimal domain level setting MIN_DOMAIN_LEVEL, it should
be used instead of DOMAIN_LEVEL_0.
See: https://pagure.io/freeipa/issue/7669
Signed-off-by: Thomas Woerner <twoerner@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
This pull should not remove code, therefore it is needed to add addtional
conditionals to calm down pylint beacuse of unreachable code.
See: https://pagure.io/freeipa/issue/7669
Signed-off-by: Thomas Woerner <twoerner@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Raise "Domain level 0 is not supported anymore" error if there are
remainaing args after parsing. Remove all "DOMAIN LEVEL 0" and
"DOMAIN LEVEL 1" prefixes from the man page.
See: https://pagure.io/freeipa/issue/7669
Signed-off-by: Thomas Woerner <twoerner@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
In the case that the domain level is set to 0 or replica_file is set (not
None) an error will be raised.
See: https://pagure.io/freeipa/issue/7669
Signed-off-by: Thomas Woerner <twoerner@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
The template directory /var/log/dirsrv/slapd-X could be created with the
wrong SELinux context.
Related to: https://pagure.io/freeipa/issue/7662
Reviewed-By: Christian Heimes <cheimes@redhat.com>
The session directory /etc/httpd/alias/ could be created with the wrong
SELinux context. Therefore httpd was not able to write to this directory.
Fixes: https://pagure.io/freeipa/issue/7662
Related-to: 49b4a057f1 (Create missing
/etc/httpd/alias for ipasession.key)
Reviewed-By: Christian Heimes <cheimes@redhat.com>
ipa-replica-install --kra-install can fail when the topology already has
a KRA, but replica is installed from a master with just CA. In that
case, Custodia may pick a machine that doesn't have the KRA auditing and
signing certs in its NSSDB.
Example:
* master with CA
* replica1 with CA and KRA
* new replica gets installed from master
The replica installer now always picks a KRA peer.
The change fixes test scenario TestInstallWithCA1::()::test_replica2_ipa_dns_install
Fixes: https://pagure.io/freeipa/issue/7518
See: https://pagure.io/freeipa/issue/7008
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Globally disabling the following violations:
- `assignment-from-no-return` (E1111):
Assigning to function call which doesn't return. Used when an
assignment is done on a function call but the inferred function
doesn't return anything.
- `keyword-arg-before-vararg` (W1113):
Keyword argument before variable positional arguments list in the
definition of %s function When defining a keyword argument before
variable positional arguments, one can end up in having multiple
values passed for the aforementioned parameter in case the method is
called with keyword arguments.
Locally disabling the following:
- `subprocess-popen-preexec-fn` (W1509):
Using preexec_fn keyword which may be unsafe in the presence of
threads The preexec_fn parameter is not safe to use in the presence
of threads in your application. The child process could deadlock
before exec is called. If you must use it, keep it trivial! Minimize
the number of libraries you call into.
https://docs.python.org/3/library/subprocess.html#popen-constructor
Fixed violations:
- `bad-mcs-classmethod-argument` (C0204):
Metaclass class method %s should have %s as first argument Used when
a metaclass class method has a first argument named differently than
the value specified in valid-metaclass-classmethod-first-arg option
(default to "mcs"), recommended to easily differentiate them from
regular instance methods.
- Note: Actually `cls` is the default first arg for `__new__`.
- `consider-using-get` (R1715):
Consider using dict.get for getting values from a dict if a key is
present or a default if not Using the builtin dict.get for getting a
value from a dictionary if a key is present or a default if not, is
simpler and considered more idiomatic, although sometimes a bit slower
Issue: https://pagure.io/freeipa/issue/7614
Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
When multiple replicas are installed in parallel, two replicas may try
to create the cn=replica entry at the same time. This leads to a
conflict on one of the replicas. replica_config() and
ensure_replication_managers() now handle conflicts.
ipaldap now maps TYPE_OR_VALUE_EXISTS to DuplicateEntry(). The type or
value exists exception is raised, when an attribute value or type is
already set.
Fixes: https://pagure.io/freeipa/issue/7566
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Thierry Bordaz <tbordaz@redhat.com>
In order to support pylint 2.0 the following violations must be fixed:
- `chained-comparison` (R1716):
Simplify chained comparison between the operands This message is
emitted when pylint encounters boolean operation like
"a < b and b < c", suggesting instead to refactor it to "a < b < c".
- `consider-using-in` (R1714):
Consider merging these comparisons with "in" to %r To check if a
variable is equal to one of many values,combine the values into a
tuple and check if the variable is contained "in" it instead of
checking for equality against each of the values.This is faster
and less verbose.
Issue: https://pagure.io/freeipa/issue/7614
Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Aiming to support pylint 2.0 some functions and methods must have their
return statements updated in order to fix two new violations:
- `useless-return` (R1711):
Useless return at end of function or method Emitted when a single
"return" or "return None" statement is found at the end of function
or method definition. This statement can safely be removed because
Python will implicitly return None
- `inconsistent-return-statements` (R1710):
Either all return statements in a function should return an
expression, or none of them should. According to PEP8, if any return
statement returns an expression, any return statements where no value
is returned should explicitly state this as return None, and an
explicit return statement should be present at the end of the
function (if reachable)
Issue: https://pagure.io/freeipa/issue/7614
Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
7284097eed introduced a regression in
DNSSEC master installation. For standalone and replica installation,
services have to be enabled before checking bind config.
Fixes: https://pagure.io/freeipa/issue/7635
See: https://pagure.io/freeipa/issue/7566
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
During parallel replica installation, a request sometimes fails with
CA_REJECTED or CA_UNREACHABLE. The error occur when the master is
either busy or some information haven't been replicated yet. Even
a stuck request can be recovered, e.g. when permission and group
information have been replicated.
A new function request_and_retry_cert() automatically resubmits failing
requests until it times out.
Fixes: https://pagure.io/freeipa/issue/7623
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Tune 389-DS replication settings to improve performance and avoid
timeouts. During installation of a replica, the value of
nsDS5ReplicaBindDnGroupCheckInterval is reduced to 2 seconds. At the end
of the installation, the value is increased sensible production
settings. This avoids long delays during replication.
See: https://pagure.io/freeipa/issue/7617
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
Calls to `os.fsync(f.fileno())` need to be accompained by `f.flush()`.
Commit 8bbeedc93f introduces the helper
`ipapython.ipautil.flush_sync()`, which handles all calls in the right
order.
However, `flush_sync()` takes as parameter a file object with fileno
and name, where name must be a path to the file, this isn't possible
in some cases where file descriptors are used.
Issue: https://pagure.io/freeipa/issue/7251
Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Extend fix 0f31564b35 to also move
the DS SSL setup so that the xmlrpc_uri is configured to point
to the remote master we are configuring against.
https://pagure.io/freeipa/issue/7566
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Commit 7284097eed kept
find_providing_servers('CA') call before enable_services(). Therefore the
list of known CA servers did not contain the current replica.
ipa-replica-install on the first replica with --setup-ca still printed
the CA topology warning.
See: https://pagure.io/freeipa/issue/7566
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Service entries in cn=FQDN,cn=masters,cn=ipa,cn=etc are no longer
created as enabled. Instead they are flagged as configuredService. At
the very end of the installer, the service entries are switched from
configured to enabled service.
- SRV records are created at the very end of the installer.
- Dogtag installer only picks fully installed servers
- Certmonger ignores all configured but not yet enabled servers.
Fixes: https://pagure.io/freeipa/issue/7566
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Fix `ERROR 'str' object has no attribute 'decode'` when --zonemgr is
passed to ipa-server-install.
Solution copied from commit 75d26e1f01,
function `ipaserver.install.bindinstance.zonemgr_callback` duplicates
the behavior of the method affected by this patch.
Issue: https://pagure.io/freeipa/issue/7612
Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
- Add missing executable bits to all scripts
- Remove executable bits from all files that are not scripts,
e.g. js, html, and Python libraries.
- Remove Python shebang from all Python library files.
It's frown upon to have executable library files in site-packages.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Armando Neto <abiagion@redhat.com>
Skip the existing reverse zone overlap check during DNS installation
when both --auto-reverse and --allow-zone-overlap arguments are
provided.
https://pagure.io/freeipa/issue/7239
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
ipaldap's LDAPClient client turns INVALID_CREDENTIAL error into
ACIError. Catch the ACIError and wait until the user has been
replicated.
Apparently no manual or automated test ran into the timeout during
testing.
Fixes: Fixes: https://pagure.io/freeipa/issue/7593
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Make CA bundles, certs, and cert directories world-accessible in
upgrader.
Fixes: https://pagure.io/freeipa/issue/7594
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
The directory /etc/httpd/alias contains public key material. It must be
world readable and executable, so any client can read public certs.
Note: executable for a directory means, that a process is allowed to
traverse into the directory.
Fixes: https://pagure.io/freeipa/issue/7594
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
In CIS hardened mode, the process umask is 027. This results in some
files not being world readable. Ensure that write_certificate_list()
calls in client installer, server installer, and upgrader create cert
bundles with permission bits 0644.
Fixes: https://pagure.io/freeipa/issue/7594
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
DogtagInstance.setup_admin and related methods have multiple LDAP
replication race conditions. The bugs can cause parallel
ipa-replica-install to fail.
The code from __add_admin_to_group() has been changed to use MOD_ADD
ather than search + MOD_REPLACE. The MOD_REPLACE approach can lead to
data loss, when more than one writer changes a group.
setup_admin() now waits until both admin user and group membership have
been replicated to the master peer. The method also adds a new ACI to
allow querying group member in the replication check.
Fixes: https://pagure.io/freeipa/issue/7593
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Instead of multiple timeout values all over the code base, all
replication waits now use a common timeout value from api.env of 5
minutes. Waiting for HTTP/replica principal takes 90 to 120 seconds, so
5 minutes seem like a sufficient value for slow setups.
Fixes: https://pagure.io/freeipa/issue/7595
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
replication.wait_for_entry() now can wait for an attribute value to
appear on a replica.
Fixed timeout handling caused by bad rounding and comparison. For small
timeouts, the actual time was rounded down. For example for 60 seconds
timeout and fast replica, the query accumulated to about 0.45 seconds
plus 60 seconds sleep. 60.45 is large enough to terminate the loop
"while int(time.time()) < timeout", but not large enough to trigger the
exception in "if int(time.time()) > timeout", because int(60.65) == 60.
See: https://pagure.io/freeipa/issue/7593
Fixes: https://pagure.io/freeipa/issue/7595
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
ipa-replica-install only set ca_host in its temporary
/etc/ipa/default.conf, when it wasn't installing a replica with CA. As a
consequence, the replica installer was picking a random CA server from
LDAP.
Always set the replication peer as ca_host. This will ensure that the
installer uses the same replication peer for CA. In case the replication
peer is not a CA master, the installer will automatically pick another
host later.
See: https://pagure.io/freeipa/issue/7566
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
The server installer had no console logger set so print
statements were used for communication. Now that a logger
is enabled the extra prints need to be dropped.
A number of logger.info statements have been upgraded
to debug since they do not need to appear on the console
by default.
https://pagure.io/freeipa/issue/6760
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
The server installation and uninstallation overlaps both the
server and client installers. The output could be confusing
with a server uninstall finishing with the message:
The ipa-client-install command was successful
This was in part due to the fact that the server was not
configured with a console format and verbose was False which
meant that no logger messages were displayed at all.
In order to suppress client installation errors and avoid
confusion add a list of errors to ignore. If a server install
was not successful and hadn't gotten far enough to do the
client install then we shouldn't complain loudly about it.
https://pagure.io/freeipa/issue/6760
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Increase the WSGI daemon worker process count from 2 processes to 5
processes. This allows IPA RPC to handle more parallel requests. The
additional processes increase memory consumption by approximante 250 MB
in total.
Since memory is scarce on 32bit platforms, only 64bit platforms are
bumped to 5 workers.
Fixes: https://pagure.io/freeipa/issue/7587
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The schema compat plugin is disabled on upgrades but it is
possible that it is not configured at all and this will
produce a rather nasty looking error message.
Check to see if it is configured at all before trying to
disable it.
https://pagure.io/freeipa/issue/6610
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Sometimes named-pkcs11 is not being stopped or reloaded during
uninstall and it causes a lot of problems while testing, for example,
backup and restore tests are failing because of ipa-server-install
fails on checking DNS step.
Fixes backup/restore tests runs. Maybe something else.
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Fix 994f71ac8a was incomplete. Under some
circumstancs the DM hash and CA keys were still retrieved from two different
machines.
Custodia client now uses a single remote to upload keys and download all
secrets.
Fixes: https://pagure.io/freeipa/issue/7518
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Simo Sorce <ssorce@redhat.com>
If this is enabled it can cause a deadlock with SSSD trying
to look up entries and it trying to get data on AD users
from SSSD.
When reading the entry from LDIF try to get the camel-case
nsslapd-pluginEnabled and fall back to the all lower-case
nsslapd-pluginenabled if that is not found. It would be nice
if the fetch function was case sensitive but this is likely
overkill as it is, but better safe than blowing up.
Upon restoring it will always write the camel-case version.
https://pagure.io/freeipa/issue/6721
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
PR https://github.com/freeipa/freeipa/pull/1747 added the first template
for FreeIPA client package. The template file was added to server
templates, which broke client-only builds.
The template is now part of a new subdirectory for client package shared
data.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
The server, replica, and client installer now print the current version
number on the console, before the actual installer starts. It makes it
easier to debug problems with failed installations. Users typically post
the console output in a ticket.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
The password was only indirectly validated when trying to
disable replication agreements for the restoration.
Only validate the password if the IPA configuration is available
and dirsrv is running.
https://pagure.io/freeipa/issue/7136https://pagure.io/freeipa/issue/7535
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
The site and module configs are split on Debian, server setup needs
to match that.
Fixes: https://pagure.io/freeipa/issue/7554
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
