Some tests require its specific Pytest args. With this change
they can be specified in tests definitions.
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Template the autoconf phase.
Fixes: https://pagure.io/freeipa/issue/8768
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The test was temporarily removed because of a known issue
but the issue is now fixed.
Related: https://pagure.io/freeipa/issue/8496
Signed-off-by: Florence Blanc-Renaud <flo@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Sergey Orlov <sorlov@redhat.com>
IPA server's AAAA records at embedded DNS mode depend on result of
`get_server_ip_address` function(`ipaserver.install.installutils`),
which in turn, relies on NSS.
In case of Azure Pipelines, there are neither IPv6 records in
'/etc/hosts' nor external DNS, which may provide such. This leads to
the missing AAAA records for master and missing AAAA records for `ipa-ca`
pointing to master in embedded DNS.
In particular, tests `test_ipa_healthcheck_no_errors`,
`test_ipa_dns_systemrecords_check` fail with:
```
[
{
"source": "ipahealthcheck.ipa.idns",
"check": "IPADNSSystemRecordsCheck",
"result": "WARNING",
"uuid": "b979a88a-6373-4990-bc83-ce724e9730b4",
"when": "20210120055054Z",
"duration": "0.032740",
"kw": {
"msg": "Got {count} ipa-ca AAAA records, expected {expected}",
"count": 1,
"expected": 2
}
}
]
```
where `ipa-ca` record exists only for replica.
Note: since the most of the code in setup_containers was touched it has
been reformatted.
Fixes: https://pagure.io/freeipa/issue/8683
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Linux 5.8 made faccessat2() system call available in August 2020. This
system call is used now by GNU libc to implement more precisely
faccessat() system call. GNU glibc does compile-time check for the
kernel version and uses faccessat2() unconditionally in case it is
available. If kernel responds with ENOSYS error code, GNU libc will
attempt to use older, less flexible, faccessat(() system call.
When running on a system where libseccomp does not know about the new
syscall, the default action in seccomp filters in Docker and other
container runtimes is to respond with EPERM error code. This breaks GNU
libc's implementation of the faccessat() function -- as well as other
newer syscall implementations (e.g. statx()).
libseccomp started to support faccessat2() in July 2020 with
5696c89640
(version 2.5.0: https://github.com/seccomp/libseccomp/releases/tag/v2.5.0)
With Ubuntu 20.04 as a host, use PPA abbra/freeipa-libseccomp which
provides libseccomp 2.5.0 rebuild from Debian Sid.
Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
The test test_dnssec.py::TestInstallDNSSECFirst is failing due to known
issue https://pagure.io/freeipa/issue/8496
currently under investigation by 389ds team.
In the meantime, remove the test from gating to avoid blocking the PRs.
Related: https://pagure.io/freeipa/issue/8496
Reviewed-By: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
FreeIPA uses git in its build process. In the past git was automatically
pulled in. On Fedora 33 builds are failing because git is missing.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Fixes the following error in Azure Pipelines CI after upgrade of Docker
setup:
[2020-09-14 06:50:07] The Compose file './docker-compose.yml' is invalid because:
[2020-09-14 06:50:07] services.client.mem_limit contains an invalid type, it should be a string
Fixes: https://pagure.io/freeipa/issue/8494
Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
This allows to debug issues happened during packages installation:
> -v, --verbose increase verbosity of reporting output.
-vv mode turns off output redirection for package installation,
above level two verbosity flags are passed through to pip (with two less
level) (default: 0)
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Some applications remove their logs on uninstallation.
As a result of this, Azure lost `install` logs.
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Python is provided by the Docker container image and it's no
longer needed to bind mount host's Python into container.
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
- allow override variables template file with an externally
provided one. This allows to add new Azure Pipeline which will
point to a custom platform definition. Note: Azure's WebUI
variables are runtime variables and not available at parsing time,
that's why it's impossible to override template from WebUI in
this case.
- add Rawhide templates
- add Dockerfile for build Rawhie Docker image for tests phase
Note: 'fedora:rawhide' is too old, use for now
'registry.fedoraproject.org/fedora:rawhide'.
See, https://bugzilla.redhat.com/show_bug.cgi?id=1869612
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The kernel keyrings are not namespaced yet.
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Francois Cami <fcami@redhat.com>
`registry.fedoraproject.org/f32/fedora-toolbox` image is used to build
packages on Azure Pipelines.
registry.fedoraproject.org experiences an availability problem and makes
unstable FreeIPA CI.
Fedora also distributes its official images on https://hub.docker.com/_/fedora.
`fedora:32` is already used by FreeIPA CI to build the image for tests.
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Francois Cami <fcami@redhat.com>
389-ds memory autotuning doesn't really work well in containerized
environment as it only looks into host-wide /proc/meminfo. It gets
fooled by 'missing' memory while there is still enough swap space.
This is in particular affects test_commands test suite where
ipa-adtrust-install cannot fully proceed and fails. We plan to rebalance
test containers' memory split but right now just disable test_commands
in Azure CI.
Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Isaac Boukris <iboukris@redhat.com>
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Build container(image registry.fedoraproject.org/f32/fedora-toolbox)
has two more dnf repos enabled compared to Tests container(image
fedora:32). This results in the packages built within the Build
container can have dependencies which are unresolvable(missing)
within Tests container.
This enables updates-testing and updates-testing-modular,
disables fedora-cisco-openh264 for Tests container.
Fixes: https://pagure.io/freeipa/issue/8330
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
* use "developer" in Azure
* fix man page: "development" to "developer"
* list known modes in API bootstrap methods
Other values for mode are still supported to avoid breaking existing
installations.
Fixes: https://pagure.io/freeipa/issue/8313
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Though time namespace support was added in Linux kernel 5.6, it
is not landed on Azure VM (Ubuntu) yet.
The syncing time stuff is required by IPA NTP tests. it's
acceptable for testing 1 IPA environment on 1 Azure VM for such
tests.
Fixes: https://pagure.io/freeipa/issue/8316
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
This allows to override the default seccomp profile.
Custom profile was generated from the default one [0] by adding one
allowed system call 'clock_adjtime'. This one is indirectly used by
chronyd with recent glibc2.31.
[0]: https://github.com/containers/libpod/blob/master/seccomp.json
Fixes: https://pagure.io/freeipa/issue/8316
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Chrony daemon tries to use adjtimex() which doesn't work in the
container we run in Docker environment on Azure Pipelines.
nis-domainname also tries to modify kernel-specific parameter that
doesn't really work in runc-based containers.
Use systemd container detection to avoid starting these services in the
containers.
Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Azure host has 6 GB of physical memory + 7 GB of swap.
FreeIPA CI runs at least 5 masters on each Azure's host.
Thus, swap is intensively used.
Based on the available *physical* memory 389-ds performs db tweaks
and in future may fail to start in case of memory shortage.
Current memory limit for Azure Docker containers(master/replica):
- Physical
$ cat /sys/fs/cgroup/memory/memory.limit_in_bytes
1610612736
- Physical + swap:
$ cat /sys/fs/cgroup/memory/memory.memsw.limit_in_bytes
3221225472
In the meantime, installation of master + ca + kra + dnssec requires:
$ cat /sys/fs/cgroup/memory/memory.max_usage_in_bytes
1856929792
Some test environments require more memory.
For example, 'ipatests.test_integration.test_commands.TestIPACommand':
$ cat /sys/fs/cgroup/memory/memory.memsw.max_usage_in_bytes
2232246272
$ cat /sys/fs/cgroup/memory/memory.max_usage_in_bytes
2232246272
Fixes: https://pagure.io/freeipa/issue/8264
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Use -W with sphinx-build by default to turn warnings into errors.
Signed-off-by: François Cami <fcami@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Applications may crash.
If a crash happens on a remote system during CI run it's sometimes
very hard to understand the reason. The most important means to
analyze such is a stack trace. It's also very important to check
whether there was a core dump or not, even a test passed.
For Docker environment, the core dumps are collected by the host's
systemd-coredump, which knows nothing about such containers (for
now). To build an informative thread stack trace debuginfo packages
should be installed. But they can't be installed on the host OS
(ubuntu), That's why after all the tests completed an additional
container should be up and the host's core dumps and host's journal
should be passed into it.
Even if there weren't enough debuginfo packages at CI-runtime, the
core dump could be analyzed locally later.
Fixes: https://pagure.io/freeipa/issue/8251
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
The platforms may have different Pythons.
But due to [0] the Python installed via the 'UsePythonVersion@0'
task should be compatible with the container's 'libpythonxx.so'.
'AZURE_PYTHON_VERSION' platform variable is introduced to cover
this. So, if your distro has Python3.8, set the mentioned variable
to '3.8', later, this version will be installed by the
'UsePythonVersion@0' Azure task for 'WebUI_Unit_Tests' and 'Tox'
jobs.
To allow tox to run any Python3 environment the 'py3' one is used.
'py3' is the well-known Tox's environment, which utilizes 'python3'
executable.
[0]: https://github.com/microsoft/azure-pipelines-tasks/issues/11070
Fixes: https://pagure.io/freeipa/issue/8254
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abbra@users.noreply.github.com>
Fast mirror selection somehow stopped working. If disabled, the
difference is around 20 seconds for the 'Prepare build environment' step
(2:49 versus 3:09), so while we are saving, currently it is not a lot.
Also remove explicit nodejs stream choice, it seems to be not needed
anymore (again).
Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Armando Neto <abiagion@redhat.com>
An external DNS is not supported yet, but it could be easily
implemented by adding another container with simple DNS server.
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Currently, only one IPA environment is tested within Docker
containers. This is not efficient because Azure's agent gives
6 GB of physical memory and 13 GB of total memory (Feb 2020),
but limits CPU with 2 cores.
Next examples are for 'master-only' topologies.
Let's assume that only one member of github repo simultaneously
run CI. This allows to get the full strength of Azure.
Concurrency results for TestInstallMaster:
------------------------------------------
| job concurrency | time/jobs |
------------------------------------------
| 5 | 40/5 |
| 4 | 34/4 |
| 3 | 25/3 |
| 2 | 19/2 |
| 1 | 17/1 |
------------------------------------------
Results prove the limitation of 2 cores. So, in case of jobs'
number not exceeds the max capacity for parallel jobs(10) the
proposed method couldn't save time, but it reduces the used
jobs number up to 2 times. In other words, in this case CI
could pass 2 x tests.
But what if CI was triggered by several PRs? or jobs' number is
bigger than 10. For example, there are 20 tests to be run.
Concurrency results for TestInstallMaster and 20 input jobs:
------------------------------------------------------------------
| job concurrency | time | jobs used | jobs free |
------------------------------------------------------------------
| 5 | 40 | 4 | 6 |
| 4 | 34 | 5 | 5 |
| 3 | 25 | 7 | 3 |
| 2 | 19 | 10 | 0 |
| 1 | 34 | 20 | 0 |
------------------------------------------------------------------
So, in this case the optimal concurrency would be 4 since it
allows to run two CIs simultaneously (20 tasks on board) and get
results in 34 minutes for both. In other words, two people could
trigger CI from PR and don't wait for each other.
New Azure IPA tests workflow:
+ 1) generate-matrix.py script generates JSON from user's YAML [0]
2) Azure generate jobs using Matrix strategy
3) each job is run in parallel (up to 10) within its own VM (Ubuntu-18.04):
a) downloads prepared Docker container image (artifact) from Azure cloud
(built on Build Job) and loads the received image into local pool
+ b) GNU 'parallel' launch each IPA environment in parallel:
+ 1) docker-compose creates the Docker environment having a required number
of replicas and/or clients
+ 2) setup_containers.py script does the needed container's changes (DNS,
SSH, etc.)
+ 3) launch IPA tests on tests' controller
c) publish tests results in JUnit format to provide a comprehensive test
reporting and analytics experience via Azure WebUI [1]
d) publish regular system logs as artifacts
[0]: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/phases?view=azure-devops&tabs=yaml
Fixes: https://pagure.io/freeipa/issue/8202
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
This log file is collected by azure-run-tests.sh script and then by
Azure 'PublishPipelineArtifact' task. So, the same file gets into
logs artifact.
Fixes: https://pagure.io/freeipa/issue/8202
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
This allows to run IPA tests on Azure using any distro.
To achieve this, one has to do:
1) place a platform specific template on 'ipatests/azure/templates/'
and make a soft link from 'ipatests/azure/templates/variables.yml' to
the new template.
2) place a configuration templates on these paths
3) templates have to answer the questions such as:
a) which Docker image to use to build IPA packages (rpm, deb, etc.)
b) how to prepare Build environment
c) how to build IPA packages
d) how to prepare environment to run Tox tests
e) how to prepare environment to run WebUI unittests
f) which base Docker image to use to build the new image to run
IPA tests within it
Fixes: https://pagure.io/freeipa/issue/8202
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Azure provides Microsoft-hosted agents having tasty resources [0].
For now (Feb 2020),
- (Linux only) Run steps in a cgroup that offers 6 GB of physical memory and
13 GB of total memory
- Provide at least 10 GB of storage for your source and build outputs.
This is enough to set up IPA environments consisted of not only master but also
replicas and clients and thus, run IPA integration tests.
New Azure IPA tests workflow:
+ 1) Azure generate jobs using Matrix strategy
2) each job is run in parallel (up to 10) within its own VM (Ubuntu-18.04):
a) downloads prepared Docker container image (artifact) from Azure cloud
(built on Build Job) and loads the received image into local pool
+ b) docker-compose creates the Docker environment having a required number
of replicas and/or clients
+ c) setup_containers.py script does the needed container's changes (DNS,
SSH, etc.)
+ d) launch IPA tests on tests' controller
e) publish tests results in JUnit format to provide a comprehensive test
reporting and analytics experience via Azure WebUI [1]
f) publish regular system logs as artifacts
[0] https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops
[1] https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/test/publish-test-results?view=azure-devops&tabs=yaml
Fixes: https://pagure.io/freeipa/issue/8202
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
IPA integration tests utilize SSH as a transport to communicate
with IPA hosts. To run such tests Docker environments should
have configured SSH.
Fixes: https://pagure.io/freeipa/issue/8202
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
As for now, a list of tests which will be ignored by Pytest is
mandatory. But actually, a list of tests to run is explicitly set
in yaml config. And thus, 'ignore' list should be an optional field.
This simplifies tests definitions to drop extra stuff.
Fixes: https://pagure.io/freeipa/issue/8202
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
nodejs:12 requires libicu-65.1 while gdb (not direct dependency)
libicu-63.2. As a workaround gdb-minimal [0] could be used.
It's even better as requires less packages to be downloaded
and then installed.
[0] https://fedoraproject.org/wiki/Changes/Minimal_GDB_in_buildroot
Co-authored-by: Stanislav Levin <slev@altlinux.org>
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
* 'fr_FR' locale is utilized in
test_ipaserver/test_i18n_messages.py::test_i18n_messages::test_i18n_consequence_receive
* 'en_US' is a commonly used locale
AP warns regularly:
```
/bin/bash: warning: setlocale: LC_ALL: cannot change locale
(en_US.utf8): No such file or directory
```
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
`test_ipaserver` was lost on refactoring in #c8ef093e56.
Let's run that again.
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Docker utilizes its own way to provide DNS (hostname, hosts, NS).
By default, they are almost the same as the host's ones.
For instance, below is from AP container:
```
cat /etc/hosts
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.0.2 ipa.example.test ipa
cat /etc/resolv.conf
nameserver 168.63.129.16
search hqdv2iuiph0ufpcrhp4amkgzwf.fx.internal.cloudapp.net
```
As a result FreeIPA uses 168.63.129.16 (AP DNS NS [1]) as a DNS forwarder.
It's not desirable to rely on this.
Let's clear test environment.
[1] https://docs.microsoft.com/en-us/azure/virtual-network/what-is-ip-address-168-63-129-16
Related: https://pagure.io/freeipa/issue/8077
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
* switch to Python 3.7 (UsePythonVersion@0 task)
* use "pip install --user"
Fixes: https://pagure.io/freeipa/issue/8030
Signed-off-by: François Cami <fcami@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The azure pipeline defines a "prepare build" step that
installs the FreeIPA development dependencies but the
step does not report failures of the dnf builddep command.
As a consequence, subsequent steps may fail (for instance
because of components not installed such as tox) but are
hard to diagnose.
The fix reports the command failure.
Fixes: https://pagure.io/freeipa/issue/8022
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Azure Pipelines provide counters for running test jobs, these split into
System.StageAttempt and System.PhaseAttempt. Use them to make test
artifacts unique.
For XML test results we don't need to name them differently as they
aren't uploaded as artifacts but rather presented in a separate test
pane.
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Reviewed-By: Stanislav Levin <slev@altlinux.org>
