IPA DNS zone and forwardzone commands allow to use nameservers with ports
as "SERVER_IP port PORT_NUMBER". bind is supporting this syntax, but the
Resolver in dnspython that is used to verify the list of forwarders
(nameservers) is only allowing to have IP addresses in this list. With
dnspython version 2.20 there is a new validator in dns.resolver.BaseResolver
that ensures this.
Refs:
- https://bind9.readthedocs.io/en/v9_18_4/reference.html#zone-statement-grammar
- https://github.com/rthalley/dnspython/blob/master/dns/resolver.py#L1094
ipapython/dnsutil.DNSResolver derives from dns.resolver.Resolver. The setter
for nameservers has been overloaded in the DNSResolver class to split out
the port numbers into the nameserver_ports dict { SERVER_IP: PORT_NUMBER }.
After the setter for nameservers succeeded, nameserver_ports is set.
nameserver_ports is used in the resolve() method of dns.resolver.Resolver.
Additional tests have been added to verify that nameservers and also
nameserver_ports are properly set and also valid.
Fixes: https://pagure.io/freeipa/issue/9158
Signed-off-by: Thomas Woerner <twoerner@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Cleanup up no longer used Pylint's disables where possible.
Fixes: https://pagure.io/freeipa/issue/9117
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Some plugins, notably dns, modifies a returned entry in order
to compare it to the user-provided info (e.g. dnsrecord-del).
This modification was done on the cached entry directly rather
than a copy which caused unexpected results, mostly
EmptyResult because the cached entry was changed directly so
the next get_entry returned the same modified entry.
In addition, on a hit in the LDAP cache the entire cached entry
was returned regardless of what attributes were requested.
The automember condition add/remove calls only request the
inclusive/exclusive rule attributes and loop over the returned
values to look for duplicates. This was failing because the queried
entry contains attributes that the candidate entry does not contain.
The automember code is:
old_entry = ldap.get_entry(dn, [attr])
for regex in old_entry.keys():
if not isinstance(entry_attrs[regex], (list, tuple)):
old_entry, returned from the cache, contained objectclass, cn,
description, etc. which don't exist in the candidate entry so
entry_attrs[regex] threw a KeyError.
To return a copy of the entry and requested attributes on a
search HIT.
Also be more careful when storing the attributes in the cache entry.
The returned attributes may not match the requested. So store the
attributes we actually have.
This issue was exposed by Ansible which maintains a larger and
longer-lived cache because commands are executed in the server context
one after another, giving the cache a chance to build up.
Adjust the expected test results as well. In test_get_testuser()
the first request asks for all attributes (default) so ensure
that is successful since a user_add gets all attributes in
the post_callback. Next request a subset of the attributes which
is also a hit and confirm that only those requested were returned.
https://pagure.io/freeipa/issue/8897
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
This mostly confirms that when an entry is added, modified or
deleted it is expunged from the cache.
https://pagure.io/freeipa/issue/8798
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Rafael Guterres Jeffman <rjeffman@redhat.com>
Add test to ensure that IPA allows to introduce multiple
permitopen and permitlisten entries.
Signed-off-by: Antonio Torres <antorres@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The string was updated to include the directory the for the database
but this was not reflected in the test and not picked up because
the tests were executed on Fedora 32 which supports dbm so the
test wasn't executed.
https://pagure.io/freeipa/issue/8675
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Upstream dnspython 2.1.0 introduced additional error checking
on SRV values and now rejects invalid priorities.
Remove the sorting test for priority of -1.
https://pagure.io/freeipa/issue/8650
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
The Sleeper class is a helper that makes poll/sleep loops with timeout
easier to write. It takes care of edge cases and does not oversleep
timeout deadline.
Related: https://pagure.io/freeipa/issue/8521
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
New classes for user and group names provide a convenient way to access
the uid and primary gid of a user / gid of a group. The classes also
provide chown() and chgrp() methods to simplify common operations.
The wrappers are subclasses of builtin str type and behave like ordinary
strings with additional features. The pwd and grp structs are retrieved
once and then cached.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
For now FreeIPA handles explicit migration of NSS DB (dbm->sql).
But Mozilla's NSS can be built without the support of legacy database
(DBM). This implies that neither implicit nor explicit DB migration
to SQL will work. So, eventually, this support will be removed from
FreeIPA.
With this patch, the instantiation of NSS with legacy db(if not
supported by NSS) is forbidden.
Fixes: https://pagure.io/freeipa/issue/8474
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The kernel keyrings are not namespaced yet.
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Francois Cami <fcami@redhat.com>
According to datetime.utcfromtimestamp() method documentation[1],
this and similar methods fail for dates past 2038 and can be replaced by
the following expression on the POSIX compliant systems:
datetime(1970, 1, 1, tzinfo=timezone.utc) + timedelta(seconds=timestamp)
Make sure to use a method that at least allows to import the timestamps
properly to datetime objects on 32-bit platforms.
[1] https://docs.python.org/3/library/datetime.html#datetime.datetime.utcfromtimestamp
Fixes: https://pagure.io/freeipa/issue/8378
Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Pylint is a static analysis tool and therefore, couldn't always
analyze dynamic stuff properly. Transformation plugins is a way
to teach Pylint how to handle such cases.
Particularly, with the help of FreeIPA own plugin, it is possible
to tell Pylint about instance fields having a duck-typing nature.
A drawback exposed here is that a static view (Pylint's) of code
should be consistent with an actual one, otherwise, codebase will
be polluted with various skips of pylint checks.
* added missing fields to ipatests.test_integration.base.IntegrationTest
* an attempt is made to clear `no-member` skips for ipatests
* removed no longer needed `pytest` module transformation
Related: https://pagure.io/freeipa/issue/8116
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Even though Pytest supports xunit style setups, unittest and nose
tests, this support is limited and may be dropped in the future
releases. Worst of all is that the mixing of various test
frameworks results in weird conflicts and of course, is not widely
tested.
This is a part of work to remove the mixing of test idioms in the
IPA's test suite:
1) replace unittest.TestCase subclasses
2) replace unittest test controls (SkipTest, fail, etc.)
3) replace unittest assertions
Related: https://pagure.io/freeipa/issue/7989
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Even though Pytest supports xunit style setups, unittest and nose
tests, this support is limited and may be dropped in the future
releases. Worst of all is that the mixing of various test
frameworks results in weird conflicts and of course, is not widely
tested.
This is a part of work to remove the mixing of test idioms in the
IPA's test suite:
1) replace xunit style
2) employ the fixtures' interdependencies
Related: https://pagure.io/freeipa/issue/7989
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
verify_server_cert_validity() and verify_ca_cert_validity() now check
the validity time range of external certificates. The check fails if the
certificate is not valid yet or will expire in less than an hour.
Fixes: https://pagure.io/freeipa/issue/8142
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
With ipa 4.5+, the RA cert is stored in files in
/var/lib/ipa/ra-agent.{key|pem}. The upgrade code handles
the move from /etc/httpd/alias to the files but does not remove
the private key from /etc/httpd/alias.
The fix calls certutil -F -n ipaCert to remove cert and key,
instead of -D -n ipaCert which removes only the cert.
Fixes: https://pagure.io/freeipa/issue/7329
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Equal DNs with multi-valued RDNs can compare inequal if one (or
both) is constructed from a cryptography.x509.Name, because the AVAs
in the multi-valued RDNs are not being sorted.
Sort the AVAs when constructing from Name and add test cases for
equality checks on multi-valued RDNs constructed from inputs with
permuted AVA order.
Part of: https://pagure.io/freeipa/issue/7963
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
When applying DN to a cryptography.x509.Name, multi-valued RDNs get
"flattened" into separate RDNs. Update the constructor to correctly
handle Name values with multi-valued RDNs.
Fixes: https://pagure.io/freeipa/issue/7963
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
python-ldap is a Python package with heavy C extensions. In order to
build python-ldap, not only OpenLDAP development headers are necessary,
but also OpenSSL, Cyrus SASL, and MIT KRB5 development headers.
A fully functional ipaclient doesn't need an LDAP driver. It talks JSON
RPC over HTTPS to a server. python-ldap is only used by ipapython.dn.DN
to convert a string to a DN with ldap_str2dn(). The function is simple
and can be wrapped with ctypes in a bunch of lines.
Related: https://pagure.io/freeipa/issue/6468
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
config_replace_variables() can now also remove lines from a file.
Related: https://pagure.io/freeipa/issue/7860
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The fix for https://pagure.io/freeipa/issue/7868 introduced
a tuple-based OS version management method (osinfo.version_number)
by Christian Heimes.
Convert all occurrences of osinfo.version_id in ipatests to
osinfo.version_number then remove osinfo.version_id.
Related to: https://pagure.io/freeipa/issue/7873
Signed-off-by: François Cami <fcami@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Test code performs comparison to itself in order to verify __eq__ and
__ne__ implementations.
See: https://pagure.io/freeipa/issue/7758
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Python 2 had old style and new style classes. Python 3 has only new
style classes. There is no point to subclass from object any more.
See: https://pagure.io/freeipa/issue/7715
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
tox / pytest is complaining about lots and lots of invalid escape
sequences in our code base. Sprinkle raw strings or backslash escapes
across the code base to fix most occurences of:
DeprecationWarning: invalid escape sequence
There is still one warning that keeps repeating, though:
source:264: DeprecationWarning: invalid escape sequence \d
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Move the /etc/os-release parser and platform detection code out of the
private _importhook module. The ipaplatform module now contains an
osinfo module that provides distribution, os, and vendor information.
See: https://www.freedesktop.org/software/systemd/man/os-release.html
See: https://pagure.io/freeipa/issue/7661
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Add a test for ipautil.run() method called with runas parameter.
The test is using ipautil.run() to execute /usr/bin/id and
checks that the uid/gid are consistent with the runas parameter.
Note that the test needs to be launched by the root user
(non-privileged user may not have the rights to execute ipautil.run()
with runas parameter).
Related to: https://pagure.io/freeipa/issue/7681
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
- Add missing executable bits to all scripts
- Remove executable bits from all files that are not scripts,
e.g. js, html, and Python libraries.
- Remove Python shebang from all Python library files.
It's frown upon to have executable library files in site-packages.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Armando Neto <abiagion@redhat.com>
On multiple occasions, SRV query answers were not properly sorted by
priority. Records with same priority weren't randomized and shuffled.
This caused FreeIPA to contact the same remote peer instead of
distributing the load across all available servers.
Two new helper functions now take care of SRV queries. sort_prio_weight()
sorts SRV and URI records. query_srv() combines SRV lookup with
sort_prio_weight().
Fixes: https://pagure.io/freeipa/issue/7475
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Add absolute_import from __future__ so that pylint
does not fail and to achieve python3 behavior in
python2.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The installer now checks that port 8080 is available and not in use by
any other application.
The port checker has been rewritten to use bind() rather than just
checking if a server responds on localhost. It's much more reliable and
detects more problems.
Original patch by m3gat0nn4ge.
Co-authored-by: Mega Tonnage <m3gat0nn4ge@gmail.com>
Fixes: https://pagure.io/freeipa/issue/7415
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
CertDB no longer makes any assumptions about the default db type of a NSS
DB. Instead it let's certutil decide when dbtype is set to 'auto'. This
makes it much easier to support F27 and F28 from a single code base.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
- Refactor CertDB to look up values from its NSSDatabase.
- Add run_modutil() helpers to support sql format. modutil does not
auto-detect the NSSDB format.
- Add migration helpers to CertDB.
- Add explicit DB format to NSSCertificateDatabase stanza
- Restore SELinux context when migrating NSSDB.
- Add some debugging and sanity checks to httpinstance.
The actual database format is still dbm. Certmonger on Fedora 27 does
neither auto-detect DB format nor support SQL out of the box.
https://pagure.io/freeipa/issue/7354
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
In case of an error, ipapython.ipautil.run() now raises an exception that
contains the error message of the failed command. Before the exception
only contained the command and error code.
The command is no longer collapsed into one string. The error message
and logging output contains the actual command and arguments with intact
quoting.
Example:
CalledProcessError(Command ['/usr/bin/python3', '-c', 'import sys; sys.exit(" ".join(("error", "XXXXXXXX")))'] returned non-zero exit status 1: 'error XXXXXXXX\n')
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
The Python 3 refactoring effort is finishing, it should be safe
to turn all scripts to run in Python 3 by default.
https://pagure.io/freeipa/issue/4985
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Several run() calls used hard-coded paths rather than pre-defined paths
from ipaplatform.paths. The patch fixes all places that I was able to
find with a simple search.
The fix simplifies Darix's port of freeIPA on openSuSE.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
* Replace raise nose.SkipTest with raise unittest.SkipTest
* Replace nose.tools.assert_equal(a, b) with assert a == b
* Replace nose.tools.raises with pytest.raises
* Convert @raises decorator to pytest.raises() but just for relevant
lines.
* Remove nose dependency
I left the nose_compat pytest plugin in place. It can be removed in
another request in case it is no longer used.
https://pagure.io/freeipa/issue/7301
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The new marker needs_ipaapi is used to mark tests that needs an
initialized API (ipalib.api) or some sort of other API services (running
LDAP server) to work. Some packages use api.Command or api.Backend on
module level. They are not marked but rather skipped entirely.
A new option ``skip-ipaapi`` is added to skip all API based tests. With
the option, only simple unit tests are executed. As of now, freeIPA
contains more than 500 unit tests that can be executed in about 5
seconds.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
pytest is setting an env var PYTEST_CURRENT_TEST to the test name + test
parameters. If parameters happen to contain NULL bytes, the putenv()
call fails with "ValueError: embedded null byte". The workaround uses
repr() of test parameters as parameter id.
See https://github.com/pytest-dev/pytest/issues/2957
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Prepare CertDB and NSSDatabase to support sqlite DB format. NSSDatabase
will automatically detect and use either old DBM or new SQL format. Old
databases are not migrated yet.
https://pagure.io/freeipa/issue/7049
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
https://pagure.io/freeipa/issue/7247
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Simo Sorce <ssorce@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
unsupported-assignment-operation is useful at times, make it only
local, not global.
https://pagure.io/freeipa/issue/6874
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
We have only one debug log level and it causes issues with py3.
...
File "/usr/lib/python3.5/site-packages/ipaserver/rpcserver.py", line 490, in marshal
response, version, pretty_print=self.api.env.debug >= 2
TypeError: unorderable types: str() >= int()
https://pagure.io/freeipa/issue/4985
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
This parameter is unused in code. We are no longer testing if IP address
matches an interface in constructor.
https://pagure.io/freeipa/issue/4317
Reviewed-By: David Kupka <dkupka@redhat.com>
The test class depends on a working Kerberos configuration and session.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
