On a non-HSM, non-renewal-server replica we look in LDAP for
an updated certificate. If the certificates don't match then we
have a new one and write it out. If they match the assumption is
that it hasn't been renewed yet so go into CA_WORKING.
The problem is that for networked HSMs the cert will already be
visible in the database so certmonger will always be in CA_WORKING.
In this case we can assume that if the certs are the same then
that's just fine.
Related: https://pagure.io/freeipa/issue/9273
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Cleanup up no longer used Pylint's disables where possible.
Fixes: https://pagure.io/freeipa/issue/9117
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
* Drop support for python 2
* Only import traceback and syslog when needed
* Only import ipaserver.install.certs when the lock is needed
* Only import ipautil when run is needed
For the unsupported operations case this improves performance by
95%
For the supported operations that don't require a lock the
improvement is about 50%.
For the supported operations that require a lock the improvement
is about 20%
When configuring a CA certmonger calls its helper with the
following operations:
IDENTIFY
FETCH-ROOTS
GET-SUPPORTED-TEMPLATES
GET-DEFAULT-TEMPLATE
GET-NEW-REQUEST-REQUIREMENTS
GET-RENEW-REQUEST-REQUIREMENTS
FETCH-SCEP-CA-CAPS
FETCH-SCEP-CA-CERTS
Only IDENTIFY, FETCH-ROOTS and GET-NEW-REQUEST-REQUIREMENTS are
supported by ipa-submit, along with the request options SUBMIT and
POLL.
Which means every time the IPA CA in certmonger is updated
eight calls to ipa-server-guard are made so the savings are
cumulative.
The savings when executing these eight operations is a 73% decrease
(.7 sec vs 2.5 sec).
https://pagure.io/freeipa/issue/8425
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Commit 49cf5ec64b fixed a bug that
prevented migration from externally-signed to self-signed IPA CA.
But it introduced a subtle new issue: certmonger-initiated renewal
renews an externally-signed IPA CA as a self-signed CA.
To resolve this issue, introduce the `--force-self-signed' flag for
the dogtag-ipa-ca-renew-agent script. Add another certmonger CA
definition that calls this script with the `--force-self-signed'
flag. Update dogtag-ipa-ca-renew-agent to only issue a self-signed
CA certificate if the existing certificate is self-signed or if
`--force-self-signed' was given. Update `ipa-cacert-manage renew'
to supply `--force-self-signed' when appropriate.
As a result of these changes, certmonger-initiated renewal of an
externally-signed IPA CA certificate will not issue a self-signed
certificate.
Fixes: https://pagure.io/freeipa/issue/8176
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
A "cookie" is used with certmonger to track the state of a
request across multiple requests to a CA (in ca-cookie). This
is used with the certmonger POLL operation to submit a request
to the CA for the status of a certificate request. This, along
with the profile, are passed to the certmonger CA helper
scripts via environment variables when a request is made. It is
cleared from the certmonger request once the certificate is
issued.
This CA helper can do a number of things:
- SUBMIT new certicate requests (including the CA)
- POLL for status of an existing certificate request
- For non renewal masters, POLL to see if an updated cert is in
LDAP
A POLL operation requires a cookie so that the state about the
request can be passed to the CA. For the case of retrieving an
updated cert from LDAP there is no state to maintain. It just
checks LDAP and returns either a cert or WAIT_WITH_DELAY if one
is not yet available.
There are two kinds of cookies in operation here:
1. The CERTMONGER_CA_COOKIE environment variable passed via
certmonger to this helper which is a JSON object.
2. The cookie value within the JSON object which contains the
URL to be passed to dogtag.
For the purposes of clarity "cookie" here is the value within
the JSON.
The CERTMONGER_CA_COOKIE is deconstructed and reconstructed as
the request is processed, doing double duty. It initially comes
in as a JSON dict object with two keys: profile and cookie.
In call_handler the CERTMONGER_CA_COOKIE is decomposed into a
python object and the profile compared to the requested profile
(and request rejected if they don't match) and the cookie key
overrides the CERTMONGER_CA_COOKIE environment variable. This is
then reversed at the end of the request when it again becomes a
JSON object containing the profile and cookie.
This script was previously enforcing that a cookie be available on
all POLL requests, whether it is actually required or not. This
patch relaxes that requirement.
The first request of a non-renewal master for an updated certicate
from LDAP is a SUBMIT operation. This is significant because it
doesn't require a cookie: there is no state on a new request. If
there is no updated cert in LDAP then the tracking request goes
into the CA_WORKING state and certmonger will wait 8 hours (as
returned by this script) and try again.
Subsequent requests are done using POLL. This required a cookie
so all such requests would fail with the ca-error
Invalid cookie: u'' as it was empty (because there is no state).
There is no need to fail early on a missing cookie. Enforcement
will be done later if needed (and it isn't always needed). So
if CERTMONGER_CA_COOKIE is an empty string then generate a new
CERTMONGER_CA_COOKIE containing the requested profile and an empty
cookie. It still will fail if certmonger doesn't set a cookie at
all.
An example of a cookie when retrieving a new RA Agent certificate
is:
{"profile": "caServerCert", "cookie": "state=retrieve&requestId=20"}
This will result in this request to the CA:
[09/Jan/2020:14:29:54 -0500] "GET
/ca/ee/ca/displayCertFromRequest?requestId=20&importCert=true&xml=true
HTTP/1.1" 200 9857
For a renewal, the reconstructed cookie will consist of:
{"profile": "caServerCert", "cookie": ""}
https://pagure.io/freeipa/issue/8164
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Helper scripts now use api.bootstrap(log=None) to avoid the creation of
log files. Helper scripts are typically executed from daemons which
perform their own logging. The helpers still log to stderr/stdout.
This also gets rid of some SELinux AVCs when the script tries to write
to /root/.ipa/.
Fixes: https://pagure.io/freeipa/issue/8075
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
On startup certmonger performs a number of options on the
configured CA (IPA, not to be confused with the real dogtag CA)
and the tracking requests.
Break early for operations that are not supported by ipa-submit.
This will save both a fork and a lock call.
https://bugzilla.redhat.com/show_bug.cgi?id=1656519
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Wait until we know a supported operation is being called
(SUBMIT and POLL) before initializing the API, which can be
an expensive operation.
https://bugzilla.redhat.com/show_bug.cgi?id=1656519
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Update the renewal helper to always request a new certificate
("enrollment request") instead of using "renewal request". The
latter is brittle in the face of:
- missing certificate record in database
- missing original request record in database (pointed to by
certificate record)
- "mismatched" certificate or request records (there have been many
cases of this; it is suspected that request/serial range conflicts,
or something similar, may be the cause)
The Dogtag tracking request must know what profile to use, except
where the certificate uses the default profile ("caServerCert" per
'dogtag-ipa-renew-agent' implementation in Certmonger itself).
This part of the puzzle was dealt with in previous commits.
Part of: https://pagure.io/freeipa/issue/7991
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
To aid reader comprehension, use long options instead of short
options when invoking dogtag-ipa-renew-agent.
-N -> --force-new
-O -> --approval-option
Part of: https://pagure.io/freeipa/issue/7991
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Replace the @PYTHONSHEBANG@ substitution with a valid #!/usr/bin/python3
shebang. This turns Python .in files into valid Python files. The files
can now be checked with pylint and IDEs recognize the files as Python
files.
The shebang is still replaced with "#!$(PYTHON) -E" to support
platform-python.
Related: https://pagure.io/freeipa/issue/7984
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Francois Cami <fcami@redhat.com>
The ipa-cert-fix program needs to know where to put shared
certificates. Extract the logic that computes the nickname from
dogtag-ipa-ca-renew-agent to new subroutine
cainstance.get_ca_renewal_nickname().
Part of: https://pagure.io/freeipa/issue/7885
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
When the CA renewal master renews certificates that are shared
across CA replicas, it puts them in LDAP for the other CA replicas
to see. The code to create/update these entries lives in the
dogtag-ipa-ca-renew-agent renewal helper, but it will be useful for
the ipa-cert-fix program too. Extract it to a subroutine in the
cainstance module.
Part of: https://pagure.io/freeipa/issue/7885
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Python scripts are now generated from templates. The scripts are marked
as nodist (no distribution) but install targets. The templates for the
scripts are extra distribution data, no installation (noinst).
Fixes: https://pagure.io/freeipa/issue/7680
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
All Python scripts are now generated from a template with a dynamic
shebang.
ipatests/i18n.py is no longer an executable script with shebang. The
module is not executed as script directly, but rather as
$(PYTHON) ipatests/i18n.py
Fixes: https://pagure.io/freeipa/issue/7680
All Python scripts are now template files with a dynamic shebang line.
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The Python 3 refactoring effort is finishing, it should be safe
to turn all scripts to run in Python 3 by default.
https://pagure.io/freeipa/issue/4985
Reviewed-By: Christian Heimes <cheimes@redhat.com>
The scenario switching from externally signed CA to self-signed CA is
currently failing because the certmonger helper goes through the wrong
code path when the cert is not self-signed.
When the cert is not self-signed but the admin wants to switch to self-signed
a new cert needs to be requested, not retrieved from LDAP.
https://pagure.io/freeipa/issue/7173
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
When running ``ipa-cacert-manage renew --external-ca`` with an IPA
CA having a subject DN that does not correspond to ``CN=Certificate
Authority, {subject-base}``, the CSR for submission to the external
CA does not generated. dogtag-ipa-ca-renew-agent-submit is wrongly
assuming the default form of the CA subject DN.
Update dogtag-ipa-ca-renew-agent-submit to look up the actual
subject DN.
Fixes: https://pagure.io/freeipa/issue/7123
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
During recent refactoring, a workaround was added to make it
possible for OpenSSL backend of python-cryptography to read PEM
certificates returned by dogtag-ipa-renew-agent-submit. This was
fixed in latest certmonger version.
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
This commit fixes requesting certificates via certmonger in Python 3.
This includes dogtag-ipa-ca-renew-agent-submit script and scripts
used during the scripts restarting.
https://pagure.io/freeipa/issue/4985
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
If CERTMONGER_CERTIFICATE is not set in certain scenario, the
code would fail since None cannot be passed to loading certificates.
https://pagure.io/freeipa/issue/4985
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
The recent certificate refactoring assures that ipaldap operations
are able to work with IPACertificate values when communication with
the LDAP server. Use these capabilities and prevent possible bugs.
https://pagure.io/freeipa/issue/4985
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
This was a forgotten part from previous certificate refactoring which
would cause issues since the second part of results throughout the
dogtag-ipa-ca-renew-agent-submit is expected to be a string.
https://pagure.io/freeipa/issue/4985
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
certmonger returns PEM certificates with an additional newline
after the base64 encoded cert, remove it
https://pagure.io/freeipa/issue/4985
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Replace all uses of virtual profiles with `dogtag-ipa-ca-renew-agent-reuse`
and remove profile from the IPA CA certificate tracking request.
This prevents virtual profiles from making their way into CSRs and in turn
being rejected by certain CAs. This affected the IPA CA CSR with Microsoft
CS in particular.
https://pagure.io/freeipa/issue/5799
Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Make sure a CSR is exported for the IPA CA whenever certmonger detects that
the CA certificate is about to expire.
This is a pre-requisite for using the `dogtag-ipa-ca-renew-agent-reuse` CA
instead of the `ipaCSRExport` virtual profile to export the CSR.
https://pagure.io/freeipa/issue/5799
Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Add a switch which makes `dogtag-ipa-ca-renew-agent-submit` reuse the
existing certificate rather than request a new one from the CA while
maintaining LDAP replication of the certificate.
Make this available as a new `dogtag-ipa-ca-renew-agent-reuse` certmonger
CA.
This allows redoing the LDAP replication and reexecuting pre- and post-save
commands of a tracking request without reissuing the certificate.
https://pagure.io/freeipa/issue/5799
Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Use Dogtag's `caCACert` CA certificate profile rather than the
`ipaCACertRenewal` virtual profile for lightweight CA certificates.
The `ipaCACertRenewal` virtual profile adds special handling of externally
signed CA certificates and LDAP replication of issued certificates on top
of `caCACert`, neither of which is relevant for lightweight CA
certificates.
Remove all of the special casing of lightweight CA certificates from
dogtag-ipa-ca-renew-agent-submit.
Make sure existing lightweight CA certmonger tracking requests are updated
on server upgrade.
https://pagure.io/freeipa/issue/5799
Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Do not bypass the renewal master check when a non-virtual profile is used
in dogtag-ipa-ca-renew-agent-submit.
This fixes dogtag-ipa-ca-renew-agent not respecting the CA renewal master
setting for certificates tracked with a real profile. (Note that there
currently aren't any such certificates tracked by us.)
Request the RA certificate using dogtag-submit rather than
dogtag-ipa-ca-renew-agent-submit as the CA renewal master setting is not
available so early in the install process.
https://pagure.io/freeipa/issue/5799
Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Connect to LDAP after kinit is done, otherwise GSSAPI authentication will
fail.
https://pagure.io/freeipa/issue/6757
Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
Fixes an issue where the renew agent uses GSSAPI for LDAP connection but
fails because it is not authenticated.
This reverts commit 7462adec13.
https://pagure.io/freeipa/issue/6757
Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
dogtag-ipa-ca-renew-agent-submit behaves differently depending on the
certificate it needs to renew. For instance, some certificates (such as IPA RA)
are the same on all the hosts and the renewal is actually done only on
the renewal master. On other nodes, the new cert is downloaded from LDAP.
The function is_replicated() is returning the opposite as what it should. If
the cert nickname is IPA RA, it should return that the cert is replicated but
it doesn't, and this leads to a wrong code path to renew the cert.
https://pagure.io/freeipa/issue/6813
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
The "ipaCert" nicknamed certificate is not required to be
in /var/lib/ipa/radb NSSDB anymore as we were keeping a copy
of this file in a separate file anyway. Remove it from there
and track only the file. Remove the IPA_RADB_DIR as well as
it is not required anymore.
https://fedorahosted.org/freeipa/ticket/5695https://fedorahosted.org/freeipa/ticket/6680
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
If an OpenSSL certificate is requested in Certmonger
(CERT_STORAGE == "FILE") the "Subject" field of such Certificate
is ordered as received. However, when an NSS certificate is
requested, the "Subject" field takes the LDAP order
(components get reversed). This is a workaround so that the behavior
stays the same.
The workaround should be removed when
https://pagure.io/certmonger/issue/62 gets fixed.
https://fedorahosted.org/freeipa/ticket/5695
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
This is in preparation for separating out the user under which the
ipa api framework runs as.
This commit also removes certs.NSS_DIR to avoid confusion and replaces
it where appropriate with the correct NSS DB directory, either the old
HTTPD_ALIAS_DIR ot the RA DB IPA_RADB_DIR. In some cases its use is
removed altogether as it was simply not necessary.
https://fedorahosted.org/freeipa/ticket/5959
Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
In addition to replicated certificates (Dogtag certificates, RA
certificate), handle non-replicated certificates in
dogtag-ipa-ca-renew-agent as well.
https://fedorahosted.org/freeipa/ticket/5959
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Some API contexts are used to modify global state (e.g. files in /etc
and /var). These contexts do not support confdir overrides. Initialize
the API with an explicit confdir argument to paths.ETC_IPA.
The special contexts are:
* backup
* cli_installer
* installer
* ipctl
* renew
* restore
* server
* updates
The patch also corrects the context of the ipa-httpd-kdcproxy script to
'server'.
https://fedorahosted.org/freeipa/ticket/6389
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Automake manual section 13 What Gets Cleaned says that make maintainer-clean
should not remove files necessary for subsequent runs of ./configure.
It practically means that all usage of MAINTAINERCLEANFILES were incorrect
so I've removed them.
https://fedorahosted.org/freeipa/ticket/6418
Reviewed-By: Lukas Slebodnik <lslebodn@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
- Temporary modify certmonger dogtag-ipa-ca-renew helper to request the IPA RA
agent cert, using the temp cert created during pkispawn. The cert request
is now processed through certmonger, and the helper arguments are restored
once the agent cert is obtained.
- Modify the installer code creating HTTP and LDAP certificates to use
certmonger's IPA helper with temporary parameters (calling dogtag-submit
instead of ipa-submit)
- Clean-up for the integration tests: sometimes ipa renewal.lock is not
released during ipa-server-uninstall. Make sure that the file is removed
to allow future installations.
https://fedorahosted.org/freeipa/ticket/6433
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
This commit makes sure that dogtag-ipa-ca-renew-agent CA helper can be used
before Kerberos is configured.
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
In the dogtag-ipa-ca-renew-agent-submit certmonger renewal helper,
we currently use our hand-rolled PKCS #10 pyasn1 specification to
parse the friendlyName out of CSRs generated by certmonger (it
contains the NSSDB nickname of the cert).
Use other information from the renewal helper process environment to
determine the nickname and remove our PKCS #10 pyasn1 spec.
Part of: https://fedorahosted.org/freeipa/ticket/6398
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Unused variables may:
* make code less readable
* create dead code
* potentialy hide issues/errors
Enabled check should prevent to leave unused variable in code
Check is locally disabled for modules that fix is not clear or easy or have too many occurences of
unused variables
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Enhance the ipa-certupdate program to add Certmonger tracking
requests for lightweight CA certificates.
Also update the dogtag-ipa-ca-renew-agent-submit to not store or
retrieve lightweight CA certificates, becaues Dogtag clones observe
renewals and update their NSSDBs on their own, and allow the helper
to request non-self-signed certificates.
Part of: https://fedorahosted.org/freeipa/ticket/4559
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
The certmonger renewal helper needs to load server plugins to
operate. Initialise the API with in_server=True.
Fixes: https://fedorahosted.org/freeipa/ticket/5943
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
The ipautil.run function now returns an object with returncode and
output are accessible as attributes.
The stdout and stderr of all commands are logged (unless skip_output is given).
The stdout/stderr contents must be explicitly requested with a keyword
argument, otherwise they are None.
This is because in Python 3, the output needs to be decoded, and that can
fail if it's not decodable (human-readable) text.
The raw (bytes) output is always available from the result object,
as is "leniently" decoded output suitable for logging.
All calls are changed to reflect this.
A use of Popen in cainstance is changed to ipautil.run.
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
The six way of doing this is to replace all occurences of "unicode"
with "six.text_type". However, "unicode" is non-ambiguous and
(arguably) easier to read. Also, using it makes the patches smaller,
which should help with backporting.
Reviewed-By: Petr Viktorin <pviktori@redhat.com>
