freeipa/ipatests/azure/azure-run-tests.sh

91 lines
3.1 KiB
Bash
Raw Normal View History

#!/bin/bash -ex
# Setup DNS
echo -e '127.0.0.1 localhost\n::1 localhost\n' > /etc/hosts
echo 'nameserver 8.8.8.8' > /etc/resolv.conf
server_realm=EXAMPLE.TEST
server_domain=example.test
server_password=Secret123
# Normalize spacing and expand the list afterwards. Remove {} for the single list element case
tests_to_run=$(eval "echo {$(echo $TESTS_TO_RUN | sed -e 's/[ \t]+*/,/g')}" | tr -d '{}')
tests_to_ignore=$(eval "echo --ignore\ {$(echo $TESTS_TO_IGNORE | sed -e 's/[ \t]+*/,/g')}" | tr -d '{}')
Make use of Azure Pipeline slicing The unit tests execution time within Azure Pipelines(AP) is not balanced. One test job(Base) takes ~13min, while another(XMLRPC) ~28min. Fortunately, AP supports slicing: > An agent job can be used to run a suite of tests in parallel. For example, you can run a large suite of 1000 tests on a single agent. Or, you can use two agents and run 500 tests on each one in parallel. To leverage slicing, the tasks in the job should be smart enough to understand the slice they belong to. >The step that runs the tests in a job needs to know which test slice should be run. The variables System.JobPositionInPhase and System.TotalJobsInPhase can be used for this purpose. Thus, to support this pytest should know how to split the test suite into groups(slices). For this, a new internal pytest plugin was added. About plugin. - Tests within a slice are grouped by test modules because not all of the tests within the module are independent from each other. - Slices are balanced by the number of tests within test module. - To run some module within its own environment there is a dedicated slice option (could help with extremely slow tests) Examples. - To split `test_cmdline` tests into 2 slices and run the first one: ipa-run-tests --slices=2 --slice-num=1 test_cmdline - To split tests into 2 slices, then to move one module out to its own slice and run the second one: ipa-run-tests --slices=2 --slice-dedicated=test_cmdline/test_cli.py \ --slice-num=2 test_cmdline Fixes: https://pagure.io/freeipa/issue/8008 Signed-off-by: Stanislav Levin <slev@altlinux.org> Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
2019-07-03 04:56:32 -05:00
tests_to_dedicate=
[[ -n "$TESTS_TO_DEDICATE" ]] && \
tests_to_dedicate=$(eval "echo --slice-dedicated={$(echo $TESTS_TO_DEDICATE | sed -e 's/[ \t]+*/,/g')}" | tr -d '{}')
systemctl --now enable firewalld
echo "Installing FreeIPA master for the domain ${server_domain} and realm ${server_realm}"
ipa-server-install -U --domain ${server_domain} --realm ${server_realm} \
-p ${server_password} -a ${server_password} \
--setup-dns --setup-kra --auto-forwarders
install_result=$?
tests_result=1
mkdir -p /freeipa/$CI_RUNNER_LOGS_DIR
cd /freeipa/$CI_RUNNER_LOGS_DIR
if [ "$install_result" -eq 0 ] ; then
echo "Run IPA tests"
echo "Installation complete. Performance of individual steps:"
grep 'service duration:' /var/log/ipaserver-install.log | sed -e 's/DEBUG //g'
sed -ri "s/mode = production/mode = development/" /etc/ipa/default.conf
systemctl restart httpd.service
firewall-cmd --add-service={freeipa-ldap,freeipa-ldaps,dns}
echo ${server_password} | kinit admin && ipa ping
mkdir -p ~/.ipa
cp -r /etc/ipa/* ~/.ipa/
echo ${server_password} > ~/.ipa/.dmpw
echo 'wait_for_dns=5' >> ~/.ipa/default.conf
ipa-test-config --help
ipa-test-task --help
ipa-run-tests --help
Make use of Azure Pipeline slicing The unit tests execution time within Azure Pipelines(AP) is not balanced. One test job(Base) takes ~13min, while another(XMLRPC) ~28min. Fortunately, AP supports slicing: > An agent job can be used to run a suite of tests in parallel. For example, you can run a large suite of 1000 tests on a single agent. Or, you can use two agents and run 500 tests on each one in parallel. To leverage slicing, the tasks in the job should be smart enough to understand the slice they belong to. >The step that runs the tests in a job needs to know which test slice should be run. The variables System.JobPositionInPhase and System.TotalJobsInPhase can be used for this purpose. Thus, to support this pytest should know how to split the test suite into groups(slices). For this, a new internal pytest plugin was added. About plugin. - Tests within a slice are grouped by test modules because not all of the tests within the module are independent from each other. - Slices are balanced by the number of tests within test module. - To run some module within its own environment there is a dedicated slice option (could help with extremely slow tests) Examples. - To split `test_cmdline` tests into 2 slices and run the first one: ipa-run-tests --slices=2 --slice-num=1 test_cmdline - To split tests into 2 slices, then to move one module out to its own slice and run the second one: ipa-run-tests --slices=2 --slice-dedicated=test_cmdline/test_cli.py \ --slice-num=2 test_cmdline Fixes: https://pagure.io/freeipa/issue/8008 Signed-off-by: Stanislav Levin <slev@altlinux.org> Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
2019-07-03 04:56:32 -05:00
ipa-run-tests ${tests_to_ignore} \
${tests_to_dedicate} \
--slices=${SYSTEM_TOTALJOBSINPHASE:-1} \
--slice-num=${SYSTEM_JOBPOSITIONINPHASE:-1} \
--verbose --with-xunit '-k not test_dns_soa' ${tests_to_run}
tests_result=$?
else
echo "ipa-server-install failed with code ${save_result}, skip IPA tests"
fi
echo "Potential Python 3 incompatibilities in the IPA framework:"
grep -n -C5 BytesWarning /var/log/httpd/error_log || echo "Good, none detected"
echo "State of the directory server instance, httpd databases, PKI CA database:"
ls -laZ /etc/dirsrv/slapd-*/ /etc/httpd/alias/ /var/lib/ /etc/pki/pki-tomcat/alias/ || true
ls -laZ /var/lib/ipa/certs/ /var/lib/ipa/passwds/ /var/lib/ipa/private/ || true
echo "Uninstall the server"
ipa-server-install --uninstall -U
# second uninstall to verify that --uninstall without installation works
ipa-server-install --uninstall -U
if [ "$install_result" -eq 0 ] ; then
firewall-cmd --remove-service={freeipa-ldap,freeipa-ldaps,dns}
fi
echo "Collect the logs"
journalctl -b --no-pager > systemd_journal.log
tar --ignore-failed-read -cvf var_log.tar \
/var/log/dirsrv \
/var/log/httpd \
/var/log/ipa* \
/var/log/krb5kdc.log \
/var/log/pki \
/var/log/samba \
/var/named/data \
systemd_journal.log
# Final result depends on the exit code of the ipa-run-tests
test "$tests_result" -eq 0 -a "$install_result" -eq 0