freeipa/ipatests
Stanislav Levin 879855ce70 Azure: Allow to run integration tests
Azure provides Microsoft-hosted agents having tasty resources [0].
For now (Feb 2020),
- (Linux only) Run steps in a cgroup that offers 6 GB of physical memory and
13 GB of total memory
- Provide at least 10 GB of storage for your source and build outputs.

This is enough to set up IPA environments consisted of not only master but also
replicas and clients and thus, run IPA integration tests.

New Azure IPA tests workflow:

+ 1) Azure generate jobs using Matrix strategy
  2) each job is run in parallel (up to 10) within its own VM (Ubuntu-18.04):
    a) downloads prepared Docker container image (artifact) from Azure cloud
       (built on Build Job) and loads the received image into local pool
  + b) docker-compose creates the Docker environment having a required number
       of replicas and/or clients
  + c) setup_containers.py script does the needed container's changes (DNS,
       SSH, etc.)
  + d) launch IPA tests on tests' controller
    e) publish tests results in JUnit format to provide a comprehensive test
       reporting and analytics experience via Azure WebUI [1]
    f) publish regular system logs as artifacts

[0] https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops
[1] https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/test/publish-test-results?view=azure-devops&tabs=yaml

Fixes: https://pagure.io/freeipa/issue/8202
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
2020-02-25 18:02:12 +02:00
..
azure Azure: Allow to run integration tests 2020-02-25 18:02:12 +02:00
man Simplify ipa-run-tests script 2019-07-16 13:23:21 +03:00
prci_definitions Tests for backup-restore when pkg required is missing 2020-02-17 17:02:32 +01:00
pytest_ipa Tests for backup-restore when pkg required is missing 2020-02-17 17:02:32 +01:00
test_cmdline pytest: Migrate unittest/nose to Pytest fixtures 2020-02-12 18:08:32 +02:00
test_install pytest: Migrate unittest/nose to Pytest fixtures 2020-02-12 18:08:32 +02:00
test_integration ipaserver/plugins/hbacrule: Add HBAC to memberservice_hbacsvc* labels 2020-02-24 15:02:24 +01:00
test_ipaclient Fix Pytest4.x warning about message 2019-06-19 19:20:14 +10:00
test_ipalib pylint: Synchronize pylint plugin to ipatests code 2020-02-12 18:08:32 +02:00
test_ipaplatform Don't configure KEYRING ccache in containers 2019-01-18 11:33:11 +01:00
test_ipapython pylint: Synchronize pylint plugin to ipatests code 2020-02-12 18:08:32 +02:00
test_ipaserver ipatests: Properly kill gpg-agent 2020-02-12 18:08:32 +02:00
test_ipatests_plugins ipatests: Allow zero-length arguments 2020-02-14 09:29:20 +02:00
test_webui pytest: Migrate unittest/nose to Pytest fixtures 2020-02-12 18:08:32 +02:00
test_xmlrpc pytest: Migrate unittest/nose to Pytest fixtures 2020-02-12 18:08:32 +02:00
__init__.py Make an ipa-tests package 2013-06-17 19:22:50 +02:00
conftest.py Azure: Allow to run integration tests 2020-02-25 18:02:12 +02:00
create_external_ca.py Test external CA with DNS name constraints 2019-08-06 12:39:46 +02:00
data.py Fix more bytes/unicode issues 2015-10-22 18:34:46 +02:00
i18n.py Sprinkle raw strings across the code base 2018-09-27 10:23:03 +02:00
ipa-run-tests ipatests: Allow zero-length arguments 2020-02-14 09:29:20 +02:00
ipa-test-config Rename pytest_plugins to ipatests.pytest_ipa 2018-08-02 17:07:43 +02:00
ipa-test-task Rename pytest_plugins to ipatests.pytest_ipa 2018-08-02 17:07:43 +02:00
Makefile.am Build: fix distribution of static files for web UI 2016-11-09 13:08:32 +01:00
setup.cfg Port all setup.py to setuptools 2016-10-20 18:43:37 +02:00
setup.py Simplify ipa-run-tests script 2019-07-16 13:23:21 +03:00
test_util.py pylint: Synchronize pylint plugin to ipatests code 2020-02-12 18:08:32 +02:00
util.py pytest: Migrate xunit-style setups to Pytest fixtures 2020-02-12 18:08:32 +02:00