Pylint 2.9 introduced new check:
> New checker consider-using-dict-items. Emitted when iterating over
dictionary keys and then indexing the same dictionary with the key
within loop body.
Fixes: https://pagure.io/freeipa/issue/9117
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
The DogtagInstance.spawn_instance() and uninstall() have
been modified to specify the paths of PKI installation
logs using --log-file option on PKI 11.0.0 or later.
This allows IPA to have a full control over the log files
instead of relying on PKI's default log files.
Fixes: https://pagure.io/freeipa/issue/8966
Signed-off-by: Endi Sukma Dewata <edewata@redhat.com>
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: François Cami <fcami@redhat.com>
This is the experimental fix to workaround the issue with
PKI on Azure Pipelines. PKI is the most sensitive to slow
systems(at least, appropriate delays and timeouts should be
adjusted for such).
Somehow Azure's test envs became slower then they were earlier
(for example, CA subsystem start changed
~(20-30)sec -> ~(45-60)sec). This triggered various issues with
subsystems of PKI in CI.
PKI honors `PKISPAWN_STARTUP_TIMEOUT_SECONDS` env variable (if
not set the timeout is 60sec) for pkispawn tool. The default
timeout is inconsistent with IPA's one (startup_timeout=120sec), which in
particular, is used in ipa-pki-wait-running tool).
Related: https://pagure.io/freeipa/issue/8830
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Francois Cami <fcami@redhat.com>
This is needed so we can mock the DogtagInstance class
and have control over the version of tomcat.
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
In this patch, we ensure we upgrade all AJP adapters with the same
secret value if any are missing. This ensures that both IPv4 and IPv6
adapters have the same secret value, so whichever httpd connects to
will be in sync. This is consistent with what Dogtag does when
provisioning them.
Notably missing from this patch is handling of multiple unrelated AJP
adapters. In an IPA scenario (and default PKI scenario) this shouldn't
be necessary. However, with external load balancing, this might happen.
This patch benefits IPA in the scenario when:
1. pkispawn runs on an older PKI version (pre-AJP secret, so ~8.2?)
2. pki gets upgraded to 10.10.1 before IPA can provision a secret,
resulting in split IPv4/IPv6 adapters -- this would only happen
on a direct migration from 8.2 -> 8.4
3. ipa upgrade script then runs to provision an AJP secret value for
use with both Dogtag and IPA.
Without this patch, only the first (IPv4) adapter would have a secret
value provisioned in the above scenario.
Signed-off-by: Alexander Scheel <ascheel@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
The output from pki-spawn is already displayed to the user
as well as a short traceback so re-displaying the CalledProcess
error provides no value and only provokes confusion,
particularly because it is condensed and includes embedded
newlines.
Re-raise the exception from None so that the traceback is
removed and while there is still an immense traceback from
the admintool class it is significantly shorter than before
and removes:
"During handling of the above exception, another exception occurred"
The handling is in fact expected.
This changes the user-facing installer output from:
[1/28]: configuring certificate server instance
Failed to configure CA instance: CalledProcessError(Command ['/usr/sbin/pkispawn', '-s', 'CA', '-f', '/tmp/tmpr5x2l0gm', '--debug'] returned non-zero exit status 1: 'INFO: Connecting to LDAP server at ldap://ipa.example.test:389\nINFO: Connecting to LDAP server at ldap://ipa.example.test:389\nDEBUG: Installing Maven dependencies: False\nERROR: KeyError: \'CA\'\n File "/usr/lib/python3.9/site-packages/pki/server/pkispawn.py", line 575, in main\n raise KeyError(\'CA\')\n\n')
See the installation logs and the following files/directories for more information:
/var/log/pki/pki-tomcat
[error] RuntimeError: CA configuration failed.
CA configuration failed.
The ipa-server-install command failed. See /var/log/ipaserver-install.log for more information
to
[1/28]: configuring certificate server instance
Failed to configure CA instance
See the installation logs and the following files/directories for more information:
/var/log/pki/pki-tomcat
[error] RuntimeError: CA configuration failed.
CA configuration failed.
The ipa-server-install command failed. See /var/log/ipaserver-install.log for more information
The output is similarly reduced in the installer log. There is no
reason to acknowledge that a CalledProcessError was raised since
the output is already available and it's just an intermediary.
Hopefully this will encourage users to focus on the logs rather than
the malformed traceback.
https://pagure.io/freeipa/issue/8565
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
This option was inconsistent between invocations and there is
no need to stop certmonger after stopping tracking. It was also
apparently causing dbus timeout errors, probably due to the amount
of work that certmonger does at startup.
https://pagure.io/freeipa/issue/8506https://pagure.io/freeipa/issue/8533
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The dogtaginstance.is_installed() method currently relies on
the presence of the directory /var/lib/pki/pki-tomcat/{ca|kra},
even if it is empty.
An unwanted consequence is ipa-server-upgrade wrongly assuming the KRA
is installed and crashing when trying to upgrade a not-installed
component.
The fix relies on the command "pki-server subsystem-show {ca|kra}" to
detect if a subsystem is installed. The command does not require PKI
to be running (hence can be called anytime) and is delivered by
the pki-server package which is already required by ipa server pkg.
Fixes: https://pagure.io/freeipa/issue/8596
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
New classes for user and group names provide a convenient way to access
the uid and primary gid of a user / gid of a group. The classes also
provide chown() and chgrp() methods to simplify common operations.
The wrappers are subclasses of builtin str type and behave like ordinary
strings with additional features. The pwd and grp structs are retrieved
once and then cached.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Since commits:
0102d836f4de217557a6
pkispawn will not honor the pki_log_level configuration item.
All 10.9 Dogtag versions have these commits.
This affects FreeIPA in that it makes debugging Dogtag installation issues next
to impossible.
Adding --debug to the pkispawn CLI is required to revert to the previous
behavior.
Fixes: https://pagure.io/freeipa/issue/8503
Signed-off-by: François Cami <fcami@redhat.com>
Reviewed-By: Thomas Woerner <twoerner@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Container platforms for Fedora and RHEL simplify FreeIPA container
effort. Paths are based on patches from
https://github.com/freeipa/freeipa-container
Fixes: https://pagure.io/freeipa/issue/8401
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Francois Cami <fcami@redhat.com>
After upgrading a deployment from CA-less to CA-ful it is necessary
to install the RA Agent credential on non-CA servers. To facilitate
this, extract this behaviour from CAInstance so that it is callable
from other code.
Several other methods became @staticmethod as a result of this
change. This makes those methods callable without an instance of
CAInstance and also documents that those methods do not use 'self'.
Part of: https://pagure.io/freeipa/issue/7188
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
For each CA server, a Dogtag user account is created for the ACME
service to use to authenticate to the CA subsystem. This commit
cleans up the Dogtag account upon server uninstallation.
The user deletion behaviour is extracted to a common method used for
both ACME RA account deletion (on uninstall) and removal of the
temporary admin account (during replica install).
Part of: https://pagure.io/freeipa/issue/4751
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Add a method for creating a group (if it does not exist). This will
be used to create a group for ACME RA accounts.
Part of: https://pagure.io/freeipa/issue/4751
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Extract the user and group membership creation behaviour from
DogtagInstance.setup_admin to its own method, 'create_user'. The
ACME setup routine will use it to create ACME RA accounts.
The @staticmethod decorator documents that 'create_user' does not
use 'self' or 'cls'. I preferred not to lift to a top-level def
because it is very much a "DogtagInstance" behaviour.
Part of: https://pagure.io/freeipa/issue/4751
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
PKIConnection now defaults to specifying verify=True. We've introduced
a new parameter, cert_paths, to specify additional paths (directories or
files) to load as certificates. Specify the IPA CA certificate file so
we can guarantee connections succeed and validate the peer's certificate.
Point to IPA CA certificate during pkispawn
Bump pki_version to 10.9.0-0.4 (aka -b2)
Fixes: https://pagure.io/freeipa/issue/8379
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1849155
Related: https://github.com/dogtagpki/pki/pull/443
Related: https://bugzilla.redhat.com/show_bug.cgi?id=1426572
Signed-off-by: Alexander Scheel <ascheel@redhat.com>
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
We do two things:
1. Fix the xpath for AJP connector verification. An AJP connector is
one which has protocol="AJP/1.3", NOT one that has port="8009". An
AJP connector can exist on any port and port 8009 can have any
protocol. Secrets only make sense on AJP connectors, so make the
xpath match the existing comment.
2. Add some background in-line documentation about AJP secret
provisioning. This should help future developers understand why this
was added to IPA and what limitations there are in what PKI or IPA
can do. Most notably, explain why Dogtag can't upgrade the AJP
connector to have a secret in the general case.
Signed-off-by: Alexander Scheel <ascheel@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
During upgrade, if discrepancies are detected in Certmonger tracking
request configuration we remove and re-create tracking requests.
The default behaviour of the CAInstance and KRAInstance
stop_tracking_certificates() method is to stop certmonger after the
requests have been removed. This behaviour results in an
unnecessary restart of certmonger and has also been observed to
cause problems. For example, subsequent certmonger operations have
to start the certmonger process and can fail because certmonger is
not yet properly initialised (manifesting as D-Bus errors).
Suppress the unnecessary restart(s) of certmonger during tracking
request update.
Related: https://pagure.io/freeipa/issue/8186
Reviewed-By: Alexander Bokovoy <abbra@users.noreply.github.com>
This partially restores the pre-ec73de969f state of `http_proxy`,
which fails to restart the apache service during master
installation. The failure happens because of apache is not
configured yet on 'pki-tomcatd' installation phase. The mentioned
code and proposed one relies on the installer which bootstraps the
master.
Fixes: https://pagure.io/freeipa/issue/8233
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
AJP implementation in Tomcat is vulnerable to CVE-2020-1938 if used
without shared secret. Set up a shared secret between localhost
connector and Apache mod_proxy_ajp pass-through.
For existing secured AJP pass-through make sure the option used for
configuration on the tomcat side is up to date. Tomcat 9.0.31.0
deprecated 'requiredSecret' option name in favor of 'secret'. Details
can be found at https://tomcat.apache.org/migration-9.html#Upgrading_9.0.x
Fixes: https://pagure.io/freeipa/issue/8221
Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
As we need to store credentials for AJP protocol comminucation,
ensure only root can read the configuration file.
Related: https://pagure.io/freeipa/issue/8221
Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Commit 49cf5ec64b fixed a bug that
prevented migration from externally-signed to self-signed IPA CA.
But it introduced a subtle new issue: certmonger-initiated renewal
renews an externally-signed IPA CA as a self-signed CA.
To resolve this issue, introduce the `--force-self-signed' flag for
the dogtag-ipa-ca-renew-agent script. Add another certmonger CA
definition that calls this script with the `--force-self-signed'
flag. Update dogtag-ipa-ca-renew-agent to only issue a self-signed
CA certificate if the existing certificate is self-signed or if
`--force-self-signed' was given. Update `ipa-cacert-manage renew'
to supply `--force-self-signed' when appropriate.
As a result of these changes, certmonger-initiated renewal of an
externally-signed IPA CA certificate will not issue a self-signed
certificate.
Fixes: https://pagure.io/freeipa/issue/8176
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
There is a loop which keeps trying to bind as the admin user
which will fail until it is replicated.
In the case where there is a lot to replicate the default
5 minute timeout may be insufficient. Provide a hint for
tuning.
Fixes: https://pagure.io/freeipa/issue/7971
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
The HSM state is stored in fstore, so that CA and KRA installer use the
correct token names for internal certificates. The default token is
"internal", meaning the keys are stored in a NSSDB as usual.
Related: https://pagure.io/freeipa/issue/5608
Co-authored-by: Magnus K Karlsson <magnus-ka.karlsson@polisen.se>
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
The CA_BACKUP_KEYS_P12 file is not enabled when pki_backup_keys is
set to False. It's the case when FreeIPA is configured with HSM support.
Related: https://pagure.io/freeipa/issue/7677
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Replace renewal CA and profile name literals with corresponding
symbols from ipalib.constants.
Part of: https://pagure.io/freeipa/issue/7991
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
The profile for every Dogtag system cert tracking request is now
explicitly specified. So remove the code that handled unspecified
profiles.
Part of: https://pagure.io/freeipa/issue/7991
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
The Dogtag "Server-Cert cert-pki-ca" certificate is treated
specially, with its own track_servercert() method and other special
casing. But there is no real need for this - the only (potential)
difference is the token name. Account for the token name difference
with a lookup method and treat all Dogtag system certs equally
w.r.t. tracking request creation and removal.
Part of: https://pagure.io/freeipa/issue/7991
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Enabling "fresh" renewals (c.f. "renewal"-based renewals that
reference the expired certificate and its associated request object)
will improve renewal robustness.
To use fresh renewals the tracking request must record the profile
to be used. Make dogtaginstance record the profile when creating
tracking requests for both CA and KRA.
Note that 'Server-Cert cert-pki-ca' and the 'IPA RA' both use
profile 'caServerCert', which is the default (according to
dogtag-ipa-renew-agent which is part of Certmonger). So we do not
need any special handling for those certificates.
This commit does not handle upgrade. It will be handled in a
subsequent commit.
Part of: https://pagure.io/freeipa/issue/7991
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
File permissions from the rpm freeipa-server-common and
freeipa-client-common do not match the runtime permissions. This results
in mode failures on rpm -Va.
Fix the expected file permissions on rpm spec file for
/var/lib/ipa/pki-ca/publish
/var/named/dyndb-ldap/ipa
/etc/ipa/pwdfile.txt
/etc/pki/ca-trust/source/ipa.p11-kit
(new format SQLite)
/etc/ipa/nssdb/cert9.db
/etc/ipa/nssdb/key4.db
/etc/ipa/pkcs11.txt
(old format DBM)
/etc/ipa/cert8.db
/etc/ipa/key3.db
/etc/ipa/secmod.db
The commit also fixes the file permissions for
/etc/httpd/conf.d/ipa-pki-proxy.conf (644)
during server installation, and the group ownership.
Fixes: https://pagure.io/freeipa/issue/7934
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Some platforms like Debian protect the dbus.service with
RefuseManualStart=True. "systemctl start dbus" fails with operation
refused (it is configured to refuse manual start/stop). On Fedora
"systemctl start dbus" is a no-op when dbus is already running.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
For HSM support, IPA has to pass the token name for CA and subsystem
certificates to certmonger. For now, only the default 'internal' token is
supported.
Related: https://pagure.io/freeipa/issue/5608
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
ipa-server-install now verifies the pki ini override file earlier
Fixes: https://pagure.io/freeipa/issue/5608
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Allow to specify a pki.ini overlay file on the command line. The override
file can be used to override pkispawn settings.
Fixes: https://pagure.io/freeipa/issue/5608
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Note: Some configuration stanzas are deprecated and have been replaced
with new stanzas, e.g. pki_cert_chain_path instead of
pki_external_ca_cert_chain_path.
Fixes: https://pagure.io/freeipa/issue/5608
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Replace get_ldap_uri() + LDAPClient() with new LDAPClient constructors
like LDAPClient.from_realm().
Some places now use LDAPI with external bind instead of LDAP with simple
bind. Although the FQDN *should* resolve to 127.0.0.1 / [::1], there is
no hard guarantee. The draft
https://tools.ietf.org/html/draft-west-let-localhost-be-localhost-04#section-5.1
specifies that applications must verify that the resulting IP is a
loopback API. LDAPI is always local and a bit more efficient, too.
The simple_bind() method also prevents the caller from sending a
password over an insecure line.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
pkispawn sometimes does not run its indextasks. This leads to slow
unindexed filters on attributes such as description, which is used
to log in with a certificate. Explicitly reindex attribute that
should have been reindexed by CA's indextasks.ldif.
See: https://pagure.io/dogtagpki/issue/3083
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
"messagebus" is an old, archaic name for dbus. Upstream dbus has started
to move away from the old name. Let's use the modern term in FreeIPA,
too.
Fixes: https://pagure.io/freeipa/issue/7754
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Don't remove /root/.dogtag/pki-tomcat when performing step 1 of external
CA installation process. Dogtag 10.6.7 changed behavior and no longer
re-creates the client database in step 2.
Fixes: https://pagure.io/freeipa/issue/7742
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
ipaldap's LDAPClient client turns INVALID_CREDENTIAL error into
ACIError. Catch the ACIError and wait until the user has been
replicated.
Apparently no manual or automated test ran into the timeout during
testing.
Fixes: Fixes: https://pagure.io/freeipa/issue/7593
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
DogtagInstance.setup_admin and related methods have multiple LDAP
replication race conditions. The bugs can cause parallel
ipa-replica-install to fail.
The code from __add_admin_to_group() has been changed to use MOD_ADD
ather than search + MOD_REPLACE. The MOD_REPLACE approach can lead to
data loss, when more than one writer changes a group.
setup_admin() now waits until both admin user and group membership have
been replicated to the master peer. The method also adds a new ACI to
allow querying group member in the replication check.
Fixes: https://pagure.io/freeipa/issue/7593
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
The server installer had no console logger set so print
statements were used for communication. Now that a logger
is enabled the extra prints need to be dropped.
A number of logger.info statements have been upgraded
to debug since they do not need to appear on the console
by default.
https://pagure.io/freeipa/issue/6760
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Add absolute_import from __future__ so that pylint
does not fail and to achieve python3 behavior in
python2.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
DogtagInstance.backup_config uses shutil.copy to create a backup of the
config file. The function does not retain owner and group, so it creates a
backup as user and group root:root.
Closes: https://pagure.io/freeipa/issue/7426
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexey Slaykovsky <alexey@slaykovsky.com>
