The unit tests execution time within Azure Pipelines(AP) is not
balanced. One test job(Base) takes ~13min, while another(XMLRPC)
~28min. Fortunately, AP supports slicing:
> An agent job can be used to run a suite of tests in parallel. For
example, you can run a large suite of 1000 tests on a single agent.
Or, you can use two agents and run 500 tests on each one in parallel.
To leverage slicing, the tasks in the job should be smart enough to
understand the slice they belong to.
>The step that runs the tests in a job needs to know which test slice
should be run. The variables System.JobPositionInPhase and
System.TotalJobsInPhase can be used for this purpose.
Thus, to support this pytest should know how to split the test suite
into groups(slices). For this, a new internal pytest plugin was added.
About plugin.
- Tests within a slice are grouped by test modules because not all of
the tests within the module are independent from each other.
- Slices are balanced by the number of tests within test module.
- To run some module within its own environment there is a dedicated
slice option (could help with extremely slow tests)
Examples.
- To split `test_cmdline` tests into 2 slices and run the first one:
ipa-run-tests --slices=2 --slice-num=1 test_cmdline
- To split tests into 2 slices, then to move one module out to its own slice
and run the second one:
ipa-run-tests --slices=2 --slice-dedicated=test_cmdline/test_cli.py \
--slice-num=2 test_cmdline
Fixes: https://pagure.io/freeipa/issue/8008
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
This is a sort of rollback to the pre #93c158b05 state with
several improvements.
For now, the nodeids calculation by ipa-run-tests is not stable,
since it depends on current working directory. Nodeids (tests
addresses) are utilized by the other plugins, for example.
Unfortunately, the `pytest_load_initial_conftests` hook doesn't
correctly work with pytest internal paths, since it is called
after the calculation of rootdir was performed, for example.
Eventually, it's simpler to follow the default convention for
Python test discovery.
There is at least one drawback of new "old" implementation.
The ignore rules don't support globs, because pytest 4.3.0+
has the same facility via `--ignore-glob`:
> Add the `--ignore-glob` parameter to exclude test-modules with
> Unix shell-style wildcards. Add the collect_ignore_glob for
> conftest.py to exclude test-modules with Unix shell-style
> wildcards.
Upon switching to pytest4 it will be possible to utilize this.
Anyway, tests for checking current basic facilities of
ipa-run-tests were added.
Fixes: https://pagure.io/freeipa/issue/8007
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
With ipa 4.5+, the RA cert is stored in files in
/var/lib/ipa/ra-agent.{key|pem}. The upgrade code handles
the move from /etc/httpd/alias to the files but does not remove
the private key from /etc/httpd/alias.
The fix calls certutil -F -n ipaCert to remove cert and key,
instead of -D -n ipaCert which removes only the cert.
Fixes: https://pagure.io/freeipa/issue/7329
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Establishing trust with partially unavailable AD hosts require usage
of --server option. The new test checks that both commands trust-add
and trust-fetch-domains properly use this option and also that
trust-add correctly passes the server value when imlicitly invoking
trust-fetch-domains.
Relates to: https://pagure.io/freeipa/issue/7895.
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
A previous refactoring of SELinux tests has have a wrong
assumption about the user field separator within
ipaSELinuxUserMapOrder. That was '$$', but should be just '$'.
Actually, '.ldif' and '.update' files are passed through
Python template string substitution:
> $$ is an escape; it is replaced with a single $.
> $identifier names a substitution placeholder matching
> a mapping key of "identifier"
This means that the text to be substituted on should not be escaped.
The wrong ipaSELinuxUserMapOrder previously set will be replaced on
upgrade.
Fixes: https://pagure.io/freeipa/issue/7996
Fixes: https://pagure.io/freeipa/issue/8005
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Display which dnf repositories were available at the
prepare-build step via metalink.
Also display the fastestmirror cache.
Signed-off-by: François Cami <fcami@redhat.com>
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Running commands with sudo as specific user should succeed
when sudo rule has ipasudorunas field defined with value of that user
and domain-resolution-order is defined in ipa config.
Relates to https://pagure.io/SSSD/sssd/issue/3957
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Sometimes the prepare-build step of azure pipelines fails
with download errors:
"configure: error: Package requirements (nspr) were not met:"
This can be due to fastestmirror not being used to check
mirror availability and sometimes speed. Combined with a
too-low default number of retries, and a high timeout this
can lead to download failures that could be avoided.
Activate fastestmirror, add more download workers, and tune
timeout/retries to make dnf more reliable.
Fixes: https://pagure.io/freeipa/issue/7999
Signed-off-by: François Cami <fcami@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
This exercises the removal of 3DES and RC4 via Samba.
Reviewed-By: Robbie Harwood <rharwood@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
For now, FreeIPA supports SELinux things as they are in RedHat/Fedora.
But different distributions may have their own SELinux customizations.
This moves SELinux configuration out to platform constants:
- SELINUX_MCS_MAX
- SELINUX_MCS_REGEX
- SELINUX_MLS_MAX
- SELINUX_MLS_REGEX
- SELINUX_USER_REGEX
- SELINUX_USERMAP_DEFAULT
- SELINUX_USERMAP_ORDER
and applies corresponding changes to the test code.
Fixes: https://pagure.io/freeipa/issue/7996
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Introduces new utility to configure Samba on an IPA domain member.
The tool sets up Samba configuration and internal databases, creates
cifs/... Kerberos service and makes sure that a keytab for this service
contains the key with the same randomly generated password that is set
in the internal Samba databases.
Samba configuration is created by querying an IPA master about details
of trust to Active Directory configuration. All known identity ranges
added to the configuration to allow Samba to properly handle them
(read-only) via idmap_sss.
Resulting configuration allows connection with both NTLMSSP and Kerberos
authentication for IPA users. Access controls for the shared content
should be set by utilizing POSIX ACLs on the file system under a
specific share.
The utility is packaged as freeipa-client-samba package to allow pulling
in all required dependencies for Samba and cifs.ko (smb3.ko) kernel
module. This allows an IPA client to become both an SMB server and an
SMB client.
Fixes: https://pagure.io/freeipa/issue/3999
Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
By default, the `last` exit code returned from Azure script will be
checked and, if non-zero, treated as a step failure. Luckily,
for Linux script is a shortcut for Bash. Hence errexit/e option
could be applied. But Azure pipelines doesn't set it by default:
https://github.com/microsoft/azure-pipelines-agent/issues/1803
For multiline script this is a problem, unless otherwise designed.
Some of benefits of checking the result of each subcommand:
- preventing subsequent issues (broken packages, container images, etc.)
- time saving (next steps will not run)
- good diagnostics (tells which part of script fails)
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Add a new XMLRPC test with the following scenario:
- ldapadd a user without the posixaccount objectclass
- call ipa stageuser-find <user>
- check that 1 entry is returned
Related: https://pagure.io/freeipa/issue/7983
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
The systemd unit name of `named`(which is actually used) is platform-dependent:
debian - bind9-pkcs11.service
fedora - named-pkcs11.service
redhat - named-pkcs11.service
Other systems may have their own name of `bind` service.
But the default one (`named-pkcs11`) is assumed in many tests.
Of course, these tests fail on such platforms.
This can be easily fixed.
All platforms define well-knownservice `named`, which is linked to
the actually utilized one.
Fixes: https://pagure.io/freeipa/issue/7990
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
For now all the default shells of users and admin are hardcoded in
different parts of the project. This makes it impossible to run the
test suite against the setup, which has the default shell differed
from '/bin/sh'.
The single configuration point for the shell of users and admin is
added to overcome this limitation.
Fixes: https://pagure.io/freeipa/issue/7978
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
"message" parameter of pytest.raises is deprecated since Pytest4.1:
```
It is a common mistake to think this parameter will match the
exception message, while in fact it only serves to provide a custom
message in case the pytest.raises check fails.
```
That was the truth for test_unrecognised_attr_type_raises, which has
wrongly checked an exception message.
Fixes: https://pagure.io/freeipa/issue/7981
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
nfs.py calls "ipa user-add" without inputting the password twice
leading to a timeout. Input password twice then.
Signed-off-by: François Cami <fcami@redhat.com>
Reviewed-By: Alexander Bokovoy <abbra@users.noreply.github.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
This test requires SELinux and fails if selinux is disabled
(because it's calling semanage login -l).
The vagrant images currently in use in the nightly tests
are configured with selinux disabled. Add skipif marker when
selinux is disabled.
Fixes: https://pagure.io/freeipa/issue/7974
Reviewed-By: François Cami <fcami@redhat.com>
The test test_backup_and_restore.py::TestBackupAndRestore
test_full_backup_and_restore_with_selinux_booleans_off
requires SELinux to be enabled because it's using
getsebool command.
Skip the test if SELinux is disabled.
Fixes: https://pagure.io/freeipa/issue/7970
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
The behavior of automember changed with the design
https://www.port389.org/docs/389ds/design/automember-postop-modify-design.html
such that members are "cleaned up" first then re-added. This has
the effect of removing members that no longer apply to a rule.
This was breaking the automember rebuild tests because sometimes
the tests were faster than 389-ds causing memberships to be missed.
This does a client-side wait for the task to finish up so still
exercises the rebuild code.
https://pagure.io/freeipa/issue/7972
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Equal DNs with multi-valued RDNs can compare inequal if one (or
both) is constructed from a cryptography.x509.Name, because the AVAs
in the multi-valued RDNs are not being sorted.
Sort the AVAs when constructing from Name and add test cases for
equality checks on multi-valued RDNs constructed from inputs with
permuted AVA order.
Part of: https://pagure.io/freeipa/issue/7963
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Commit e3f3584 introduced an additional prompt in ipa-server-install
"Do you want to configure chrony with NTP server or pool address?".
The test is building a string passed to stdin in interactive mode
but this string has not been updated with the additional answer for
this new question.
This commit answers 'no' to the question and allows to proceed with
the ipa server installation.
Fixes: https://pagure.io/freeipa/issue/7969
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
The Host.ldap_connect() method uses LDAPClient from ipapython package.
In a3934a21 we started to use secure connection from tests controller to
ipa server. And also 5be9341f changed the LDAPClient.simple_bind method
to forbid password based authentiction over insecure connection.
This makes it imposible to establish ldap connection in some test
configurations where hostnames known to ipa server do not match ones known
to tests controller (i.e. when host.hostname != host.external_hostname)
because TLS certificate is issued for host.hostname and test controller
tries to verify it against host.external_hostname.
A sublass of LDAPClient is provided which allows to skip certificate check.
Fixes: https://pagure.io/freeipa/issue/7960
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Azure tests fail because we couldn't configure docker for IPv6 anymore.
This happened because we weren't able to copy our configuration file to
/etc/docker -- looks like the docker directory does not exist.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Tests added for two scenarios:
1) adding one-way external trust, trust on Windows side is created using
netdom utility.
2) adding one-way forest trust, trust on Windows side is created using
powershell bindings to .Net functions
Tests verify that specified trusts can be established, trust domains can
be fetched and AD user data can be queried by IPA client.
Relates: https://pagure.io/freeipa/issue/6077
Reviewed-By: Kaleemullah Siddiqui <ksiddiqu@redhat.com>
The command
ipa-replica-manage list -v <node>
can display:
last init ended: 1970-01-01 00:00:00+00:00
last init status: None
when called on a node that never had total update.
The fix for 7716 modifies the command so that it doesn't print
those lines when there is no last init status.
This commit adds a new test checking the output of
ipa-replica-manage list -v <node>.
Related to: https://pagure.io/freeipa/issue/7716
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
We recently added validation of externally-signed CA certificate to
ensure certificates signed by external CAs with too-small keys
(according to system crypto policy) are rejected.
Add an integration test that attempts to renew with a 1024-bit
external CA, and asserts failure.
Part of: https://pagure.io/freeipa/issue/7761
Reviewed-By: Christian Heimes <cheimes@redhat.com>
When applying DN to a cryptography.x509.Name, multi-valued RDNs get
"flattened" into separate RDNs. Update the constructor to correctly
handle Name values with multi-valued RDNs.
Fixes: https://pagure.io/freeipa/issue/7963
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
In 2012, ldbm backend in 389-ds started checking entry modification
after running betxnpreop plugins by comparing a number of modifications
before and after. If that number didn't change, it is considered that
plugins didn't modify the list.
ipa-pwd-extop actually removed and re-added modification to ipaNTHash if
it contained 'MagicRegen' value. This did not work since commit
https://pagure.io/389-ds-base/c/6c17ec56076d34540929acbcf2f3e65534060a43
but we were lucky nothing in FreeIPA code actually relied on that except
some code paths in ipasam Samba passdb driver. However, Samba didn't
reach the point where the code was triggered -- until now.
With support to run Samba as a domain member in IPA domain, that code
path is triggered for Kerberos service principals of domain members
(cifs/client.example.test, ...) and NT hash extraction from Kerberos
keys does not work.
Fix ipa-pwd-extop to follow recommendations in
https://pagure.io/389-ds-base/issue/387#comment-120145 and
https://pagure.io/389-ds-base/issue/50369#comment-570696
Fixes: https://pagure.io/freeipa/issue/7953
Reviewed-By: Christian Heimes <cheimes@redhat.com>
In order to test a fix for https://pagure.io/freeipa/issue/7953,
we need to create a keytab with a particular encryption type
(arcfour-hmac) and attempt to request generation of ipaNTHash attribute
from Kerberos keys in LDAP.
Add a test case that performs this operation.
Related: https://pagure.io/freeipa/issue/7953
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Convert use_keytab() function into a context manager to allow additional
operations to be done as part of the test. Also pass proper credentials
cache file to the backend while connecting to LDAP so that right creds
are in use.
This is required to perform actual tests for use of the retrieved keys.
Related: https://pagure.io/freeipa/issue/7953
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Rewrite templates to make test job declarations simpler and easier to
work with.
A test job template can be instantiated this way:
- template: templates/test-jobs.yml
parameters:
jobName: Base
jobTitle: Base tests
testsToRun:
- test_cmdline
- test_install
- test_ipaclient
- test_ipalib
- test_ipaplatform
- test_ipapython
testsToIgnore:
- test_integration
- test_webui
- test_ipapython/test_keyring.py
taskToRun: run-tests
Both 'testsToRun' and 'testsToIgnore' accept arrays of test matches.
Wildcards also supported:
....
testsToRun:
- test_xmlrpc/test_hbac*
....
'taskToRun' specifies a script ipatests/azure/azure-${taskToRun}.sh that
will be executed in the test environment to actually start tests.
Parameters 'testsToRun' and 'testsToIgnore' define TESTS_TO_{RUN,IGNORE}
variables that will be set in the environment of the test script. These
variables will have entries from the parameters separated by a space.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
ipa-run-tests expands arguments passed with their full paths. However,
it doesn't support expanding globs, so targets like 'test_ipa*' cannot
be specified.
Expand the code that replaces '--ignore foo' and 'foo' positional
arguments with support for '--ignore foo*' and 'foo*'.
This allows to reduce a number of additional steps in the CI pipeline
preparation.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Stanislav Levin
Florence Blanc-Renaud
Sergey Orlov
François Cami
Armando Neto
Rob Crittenden
Alexander Bokovoy
Anuja More
Fraser Tweedale
Kaleemullah Siddiqui
Mohammad Rizwan Yusuf