Turn multihost config problems into errors

The pytest multihost plugin skips tests, when there is a problem with a
test configuration. Configuration bugs like missing resources are not
considered a problem.

The IPA pytest multihost config object now turns FilterError into a
fatal error, so make_multihost_fixture() fails a test instead of
skipping.

Fixes: https://pagure.io/freeipa/issue/7638
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
Reviewed-By: Ganna Kaihorodova <gkaihoro@redhat.com>

ipatests/pytest_plugins/integration/config.py

@@ -23,6 +23,7 @@
 import logging
 import random
 
+import pytest
 import pytest_multihost.config
 
 from ipapython.dn import DN
@@ -99,6 +100,18 @@ class Config(pytest_multihost.config.Config):
         from ipatests.pytest_plugins.integration.env_config import config_to_env
         return config_to_env(self, **kwargs)
 
+    def filter(self, descriptions):
+        """Destructively filters hosts and orders domains to fit description
+
+        By default make_multihost_fixture() skips a test case, when filter()
+        returns a FilterError. Let's turn FilterError into a fatal error
+        instead.
+        """
+        try:
+            super(Config, self).filter(descriptions)
+        except pytest_multihost.config.FilterError as e:
+            pytest.fail(str(e))
+
 
 class Domain(pytest_multihost.config.Domain):
     """Configuration for an IPA / AD domain"""