The test often fails when running in parallel to other tests as very
little memory is left. 389-ds memory autotuning causes database backend
to refuse working in such cases. 389-ds team suggested more memory has
to be made available.
Increate RAM for this test to 2.5GB instead of 2.2GB.
Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Fixes: https://pagure.io/freeipa/issue/9347
Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Francisco Trivino <ftrivino@redhat.com>
With fix for https://pagure.io/freeipa/issue/7951 we started to modify
RPM macros in Azure CI environment. Don't fail if the file does not
exist anymore like it happens now in Fedora.
Fixes: https://pagure.io/freeipa/issue/9347
Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Francisco Trivino <ftrivino@redhat.com>
On fedora 37+ the dnssec tests were broken. The tests
launched for each pull request were disabled or marked
as xfail.
With the bump of bind version, they should now succeed
and can be re-enabled.
Related: https://pagure.io/freeipa/issue/9216
Signed-off-by: Florence Blanc-Renaud <flo@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
While python-ldap is strict dependency of IPA in downstreams, it
is optional for IPA packages published on PyPI.
Openldap 2.6 no longer ships ldap_r-2, that makes
ipapython.dn_ctypes not working against such environments.
Thanks @abbra!
Fixes: https://pagure.io/freeipa/issue/9255
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
The test TestInstallDNSSECFirst is failing because of one of its
dependencies (the most likely suspect is the update of openssl-pkcs11).
Disable the test from azure gating until the issue is solved.
Related: https://pagure.io/freeipa/issue/9216
Signed-off-by: Florence Blanc-Renaud <flo@redhat.com>
Reviewed-By: Francisco Trivino <ftrivino@redhat.com>
With some recent changes the Azure Agent has decreased performance.
For example, `InstallDNSSECFirst_1_to_5` (5 subjobs) job took ~33min
and now it takes ~40min. In the same time there are jobs having only
1 or 2 subjobs and they should be used more.
Fixes: https://pagure.io/freeipa/issue/9207
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
This daemon run clamav which is resource aggressive.
No point to run Windows virus scanner on Ubuntu in Linux-only
environment.
Fixes: https://pagure.io/freeipa/issue/9207
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
With some recent changes on Azure Agent the default DBus call
timeout is not good enough. For example, in case of
`InstallDNSSECFirst_1_to_5` job hostnamectl received reply in ~20sec,
but later it increased to ~30sec (more subjobs - more time to reply).
It's good to raise this timeout to be more protected against minimum
performance times.
https://www.freedesktop.org/software/systemd/man/sd_bus_set_method_call_timeout.html#Description
Fixes: https://pagure.io/freeipa/issue/9207
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Allow npm to install and use latest grunt that is compatible with newer
OpenSSL.
This, in turn, requires ATK interfaces to be present for the chromium
installed by puppeteer.
Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Newer grunt will pull a PhantomJS that is compatible with newer OpenSSL
so the workaround is not needed anymore.
Additionally, OpenSSL 3.0 is more strict and does not tolerate
non-existing default configuration file.
Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Use fedora-toolbox:36 image as it is prepared to work with systemd and
sudo
Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Documentation generator can be run inside Python virtual environment.
This allows to isolate from the system-wide changes and add Sphinx
extensions that aren't packaged in a distribution.
The only exception right now is plantuml package. We rely on plantuml to
generate diagrams and since it is written in Java, it cannot be
installed directly into the Python venv through 'pip' tool.
Fixes: https://pagure.io/freeipa/issue/9148
Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Francisco Trivino <ftrivino@redhat.com>
Cleanup up no longer used Pylint's disables where possible.
Fixes: https://pagure.io/freeipa/issue/9117
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Sometimes tar fails on changed in process files:
```
[2021-09-07 11:03:33] + tar --ignore-failed-read -czf ipaserver_install_logs.tar.gz --warning=no-failed-read /var/log/dirsrv /var/log/httpd2 /var/log/ipa /var/log/ipaclient-install.log /var/log/ipa-custodia.audit.log /var/log/ipaserver-install.log /var/log/krb5kdc.log /var/log/pki /var/log/samba /var/lib/bind/data systemd_journal.log
[2021-09-07 11:03:33] tar: Removing leading `/' from member names
[2021-09-07 11:03:33] tar: Removing leading `/' from hard link targets
[2021-09-07 11:03:33] tar: /var/log/dirsrv/slapd-IPA-TEST/access: file changed as we read it
[2021-09-07 11:03:33] + tests_result=1
```
This is expected failure since processes are not stopped during logs
collection and can flush their logs.
Fixes: https://pagure.io/freeipa/issue/8983
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
- previously, fastlint make's target includes both the Pylint task
and pycodestyle one. The purpose of this target is a fast checking
only for changed Python files. This makes sense for pycodestyle, but
limits Pylint due to a context(file) checking. The clients which
call the code being linted are not checked at all. In Azure Pylint
(for the whole codebase) is run in the Lint task, this makes fastlint
extra for Azure.
- `Quick code style check` task used distro's Pylint, while `Lint`
task PyPI's one. This may cause different results and confuse a
user.
- `Build` task takes time longer than `Lint` one, so this change
doesn't lead to increased CI time.
- all Azure tests depend on Build and Lint tasks. Mostly it's no need
to run tests due to a probably broken code.
Fixes: https://pagure.io/freeipa/issue/8961
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Reviewed-By: François Cami <fcami@redhat.com>
test_cert and test_SubCAkeyReplication are randomly failing.
The suspect for test_SubCAkeyReplication is an nss bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1985061
The reason for test_cert failures was not identified, the only
relevant line in the log contains:
2021-07-22T17:37:21.0873339Z tests: cert, result: 1, time: 30:08.98
2021-07-22T17:37:21.0874172Z Command exited with non-zero status 1
Disable these tests in gating until the NSS bug is fixed and
the related build is available in Fedora.
Related: https://pagure.io/freeipa/issue/8864
Signed-off-by: François Cami <fcami@redhat.com>
Reviewed-By: Michal Polovka <mpolovka@redhat.com>
Reviewed-By: Stanislav Levin <slev@altlinux.org>
test_installation.TestInstallMaster, test_advise,
and test_integration.test_commands.TestIPACommand rely on DNS
forwarders and hit a known BIND bug:
https://gitlab.isc.org/isc-projects/bind9/-/issues/2728
quite often.
This is blocking gating nearly completely.
Disable these tests in gating until the bug is fixed and
the related build is available in Fedora.
Related: https://pagure.io/freeipa/issue/8864
Signed-off-by: François Cami <fcami@redhat.com>
Reviewed-By: Michal Polovka <mpolovka@redhat.com>
Reviewed-By: Stanislav Levin <slev@altlinux.org>
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Otherwise, 'Check for coredumps' task fails with:
```
Verifying : samba-debugsource-2:4.14.4-0.fc34.x86_64 20/20
[Errno 2] No such file or directory: '/var/lib/dnf/rpmdb_lock.pid'
Finishing: Check for coredumps
```
This is due to systemd-tmpfiles(not ready yet).
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The list of installed packages may be useful for checking the
versions of packages for analysis. Previously, only the newly
installed packages can be observed on Build phase.
This is convenient for experienced users of PR-CI.
Note: the read-only access provided for non-master containers
to be able to execute Azure scripts. The logs are still collected
only on controller.
Only RPM-based collection is implemented for Fedora. By default
nothing is collected.
Users may want to override `installed_packages` function
in the corresponding `ipatests/azure/scripts/variables-DISTRO.sh`.
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
There are tons of useless warnings about missing files on collecting
logs, such as:
```
tar: /var/log/ipaserver-kra-install.log: Warning: Cannot stat: No such file or directory
tar: /var/log/ipaepn.log: Warning: Cannot stat: No such file or directory
tar: /etc/NetworkManager/NetworkManager.conf: Warning: Cannot stat: No such file or directory
tar: /var/log/ipabackup.log: Warning: Cannot stat: No such file or directory
tar: /var/log/iparestore.log: Warning: Cannot stat: No such file or directory
...
```
Since `--ignore-failed-read` option is passed to tar the caller
doesn't care about not readable(mostly missing) files and these warnings
may be filtered out.
This improves the readability of test logs.
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
By default pytest reports in summary section about tests failures and errors.
It will be helpful to see skipped, xfailed and xpassed tests.
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The tests in these envs make DNS requests to wild(internet) NSs,
though usually tests assume the opposite making requests to
`test.` zone. This makes CI unstable and dependent on wild
resolvers and logically wrong.
In future there can be tests which may want to check BIND as
resolver(cache) for external networks. In this case such tests
should be placed on not isolated mode.
By default, a test env is not isolated from internet(as it was
before), but it may be a good idea to change this default in
future.
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The nonzero number of memory/memory+Swap usage hits limits may
indicate the possible env instability(crashes, random failures, etc.).
> memory.failcnt # show the number of memory usage hits limits
memory.memsw.failcnt # show the number of memory+Swap hits limits
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
WebUI unit tests fail with:
```
PhantomJS threw an error:ERROR
>> Auto configuration failed 0 [
>> 'Auto configuration failed',
>> '140613066520384:error:25066067:DSO support routines:DLFCN_LOAD:could not load the shared library:dso_dlfcn.c:185:filename(libssl_conf.so): libssl_conf.so: cannot open shared object file: No such file or directory',
>> '140613066520384:error:25070067:DSO support routines:DSO_load:could not load the shared library:dso_lib.c:244:',
>> '140613066520384:error:0E07506E:configuration file routines:MODULE_LOAD_DSO:error loading dso:conf_mod.c:285:module=ssl_conf, path=ssl_conf',
>> '140613066520384:error:0E076071:configuration file routines:MODULE_RUN:unknown module name:conf_mod.c:222:module=ssl_conf'
>> ]
...
Warning: PhantomJS exited unexpectedly with exit code 1. Use --force to continue.
Aborted due to warnings.
```
See https://github.com/wch/webshot/pull/93 for details.
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The initial value of NS of resolv.conf is 127.0.0.11, this
is the embedded NS of docker-compose. The disabling of
this feature is not currently supported by Docker.
On startup systemd-resolved caches the /etc/resolv.conf
(docker-compose version), which is later modified by
setup_containers.py script.
This results in resolving error occurs:
```console
[root@replica1 /]# getent ahosts master1.ipa.test
... can't resolve
[root@replica1 /]# grep 'hosts:' /etc/nsswitch.conf
hosts: files myhostname resolve [!UNAVAIL=return] dns
[root@replica1 /]# resolvectl status
Global
LLMNR setting: resolve
MulticastDNS setting: no
DNSOverTLS setting: no
DNSSEC setting: no
DNSSEC supported: no
Current DNS Server: 127.0.0.11
DNS Servers: 127.0.0.11
Fallback DNS Servers: 1.1.1.1
8.8.8.8
1.0.0.1
8.8.4.4
2606:4700:4700::1111
2001:4860:4860::8888
2606:4700:4700::1001
2001:4860:4860::8844
```
According to docs:
https://www.freedesktop.org/software/systemd/man/systemd-resolved.service.html#/etc/resolv.conf
our case is 4(managed by other packages).
So, restart of systemd-resolved is enough for its re-initialization,
but not for services that already received DNS results. To speed up
the overall process and to no restart each service which wants
internet connection(or wait until service retries connection)
systemd-resolved is masked.
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The calling of systemd's utils during systemd boot may lead to
unpredictable results. For example, if DBus(dbus-broker) service
is not started then DBus request goes nowhere and eventually will
be timeouted. So, it's safer to wait fully booted system.
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
This may speed up boot process.
For example, 'fedora:34' set graphical.target as default,
while multi-user one will be more appropriate.
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
If an error occured while containers setup phase then no logs will
be collected and it is hard(impossible?) to debug such issues on
remote Azure host. With this change in case of such error all the
container's journals will be collected in `systemd_boot_logs`.
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Two, three times a year PR-CI starts failing because tox tests pull in a
newer version of pylint with new warnings. To reduce breakage this
change constraints pylint (and indirectly astroid) to latest tested
minor version. The constraint should be updated when FreeIPA starts to
support a new Fedora version with more recent pylint.
Related: https://pagure.io/freeipa/issue/8818
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Lint task uses PyPI to get the latest Pylint, which may pull in
any other packages as dependencies. For distro isolation the Lint
job should not produce any meaningful artifacts or they should not
be used in subsequent jobs. So, this job have to be isolated from
the others.
Fixes: https://pagure.io/freeipa/issue/8772
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Ubuntu 20.04's docker meets the requirement(4+).
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The security option 'apparmor:unconfined' tells Docker to not
apply AppArmor profiles for containers at all. This will not
replace or remove any existing profile. For example, this happens
on Ubuntu 20.04 which switched to chrony and brings its AppArmor
profile. Container's chronyd get blocked by AppArmor:
fv-az26-252 audit[11304]: AVC apparmor="DENIED" operation="capable" profile="/usr/sbin/chronyd" pid=11304 comm="chronyd" capability=2 capname="dac_read_search"
fv-az26-252 audit[11304]: AVC apparmor="DENIED" operation="capable" profile="/usr/sbin/chronyd" pid=11304 comm="chronyd" capability=1 capname="dac_override"
So, any of AppArmor profiles can block container's processes by
matching executable name. There are two ways:
1) prepare custom AppArmor unconfined profile, load it on Host and
reference it in container's configuration. This requires the
knowledge of profile syntax at least, not to difficult, but
potentially hard to maintain.
2) disable conflicting profile on Host;
Azure will warn about AVC in either case.
The second one was chosen as more simple.
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Azure's VM distro is Ubuntu, which has enabled AppArmor.
This security module sometimes interferes with CI Docker containers,
but to be completely disabled it requires reboot(this is impossible,
at least for now). So, Azure will warn about AVC records in Host's
journal as a possible clue.
It will be equally important to be warned about SECCOMP records to
see possible blocked syscalls(requires SCMP_ACT_LOG as defaultAction in
seccomp profile).
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The journal of Host is useful for AVC/SECCOMP analyzing.
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
