The rename tests use names that were not being cleaned up when the
tests fail. Add cleanup steps for them.
Also, use --force so system permissions are removed as well.
Previously the help plugin failed when searching for the docstring
when a topic's module was not found. This can happen when some server
plugins are loaded (e.g. for tests).
Use empty documentation when the topic is not found.
There code assumes that the `conn` in any Connection in the context is
a ServerProxy. This might not always be the case: ldap2 uses a
python-ldap connection here.
The integration tests must wait for replication to happen before checking
results. In some cases, the tests have failed because the checks that
detect completed replication were insufficient.
This fixes the code to:
- Wait for replication to be completed on both servers
- In the case of an error, continue waiting -- it might be the case that
the DS is temporarily unreachable
With old servers, it is possible that xmlclient.connect() succeeds
but commands fail with a Kerberos error.
Verify that commands succeed by sending a ping after connecting.
Follow-up to: https://fedorahosted.org/freeipa/ticket/3931
To make sure the installation works with older servers,
use XML-RPC directly, with a version set explicitly so the request
is not rejected.
RPC was chosen over ldapmodify, because going through the API allows
the server to process the request properly, or even cleanly reject
it if there are incompatible changes in future versions.
https://fedorahosted.org/freeipa/ticket/3931
If an exception is expected, but another one is raised, the CLITestContext
raised a generic AssertionError.
Pass through the original exception instead
To differentiate between individual tests in BeakerLib output,
the argument needs to be added to the test name. Since Nose
doesn't provide a way to get the argument in a plugin,
a `test_argument` attribute must be added to the test function
to support this, simlarly to how `description` is used to set
individual "docstrings".
Add test_argument to the generated tests in the CA-less suite.
Ordered test generators were not announced in plugin hooks, so
e.g. the Beakerlib or collect plugin did not announce them.
Exclude test generators from ordering.
This adds a transport that uses /usr/bin/ssh calls to communicate
with remote hosts.
This transport is a bit slower and buffers output more than paramiko,
so it is only used if paramiko is not available, or forced with an
environment variable.
https://fedorahosted.org/freeipa/ticket/3890
This will make it possible to use a different mechanism for cases like
- Paramiko is not available
- Hosts without SSH servers (e.g. Windows)
Add BaseHost, Transport & Command base classes that define the interface
and common functionality, and Host, ParamikoTransport & SSHCommand with
specific details.
The {get,put}_file_contents methods are left on Host for convenience;
all other Transport methods must be now accessed through the transport.
Part of the work for https://fedorahosted.org/freeipa/ticket/3890
The get_args methods in ipalib.crud and ipalib.plugins.baseldap used
super() calls that skipped some of the classes in the inheritance
chain, and contained code that reimplemented some of the skipped
functionality.
This made it difficult to customize the get_args behavior.
Use proper super() calls.
Updating a CIDict with data like {'A': 1, 'a': 2} would lead to data
loss since only one of the items would get to the CIDict.
This can result in non-obvious bugs similar to this one in python-ldap:
https://bugzilla.redhat.com/show_bug.cgi?id=1007820
Raise an error in this case; any resolution must be done by the caller.
Make the CIDict interface match standard dict (except view* methods).
Add __contains__, __iter__, clear.
Add keyword and iterable support for __init__, update.
Also add values() and itervalues(). Previously the dict versions were
used; the new ones guarantee that the order matches keys().
Mark view* methods as not implemented.
CIDict.copy() now returns a CIDict.
Test the above additions, and fromkeys() which worked but wasn't tested.
Nose doesn't pick up directories that don't begin with 'test'.
Rename ipatests/test_ipaserver/install to test_install so that it's run.
Also, merge test_ipautil.py from ipapython/test into tests/test_ipapython,
so the whole test suite is in one place.
The __all__ list does not cause submodules to be imported, e.g.
one would still have to `import ipaclient.ipachangeconf` rather than
just `import ipaclient` to use `ipaclient.ipachangeconf`.
Even if they did do anything, the lists were incomplete, and (since
`import *` is not used on these modules) unnecessary.
Pylint 1.0 reports undeclared names in __all__ as a warning.
Differences from the test plan at
http://www.freeipa.org/index.php?title=V3/CA-less_install&oldid=6669 are:
- The following tests are included in all applicable positive
install tests, rather than being standalone test cases:
- Verify CA certificate stored in LDAP
- Verify CA PEM file created by IPA server install
- Verify that IPA server install does not configure certmonger
- Verify CA PEM file created by IPA replica install
- Verify that IPA replica install does not configure certmonger
- Verify CA PEM file created by IPA client install
- PKI setup is done only once for each test class
- Master installation is done once for the IPA command tests, and
once for the certinstall tests
- Certificates are compared after base64 decoding to avoid failures
from formatting mismatches
- Minor changes necessary for automation (e.g. adding --unattended
and --password options, correcting error messages)
- Web UI tests are not included here
https://fedorahosted.org/freeipa/ticket/3830
This makes plugin registration easier to read, less error-prone, and,
for many Plugins in a single module, faster to write.
Functionally, the decorator is equivalent to current plugin
registration. However, in the future this style will allow cleaner
semantics.
As an example, and to exercise the new syntax to prevent regressions,
the ping plugin is converted to this style.
The integration testing framework used Paramiko SFTP files as
context managers. This feature is only available in Paramiko 1.10+.
Use an explicit context manager so that we don't rely on the feature.
If stdin is a TTY, ipaserver.install.installutils uses getpass and all
is well. Without a TTY, though, there were two problems:
* The prompt was not printed
* On end of file, an empty string was returned, which caused read_password
to enter an infinite loop.
Fix both problems.
https://fedorahosted.org/freeipa/ticket/3824
Regenerate the POT file and pull new translations from Transifex.
Also, update the Transifex URL in the configuration file:
transifex.net has redirected to transifex.com for some time,
and now its certificate has expired.
Add ldap_connect() method to Host to allow executing querying LDAP from tests.
Use information in the mapping tree to poll until all replication is finished
(or failing) before checking that entries replicated successfully.
This allows a cluster of replicas and clients to be installed
in a named topology.
Several named topologies are available (star, line, complete, tree,
tree2) and new ones can be defined as a simple function.
Logs from Beaker jobs are normally very brief, with the standard
output/error containing detailed information. Make ipa-run-tests
with BeakerLib plugin follow this convention.
Only include INFO and higher level messages in the Beaker logs.
Downgrade several message levels to DEBUG.
Log to console using Python logging instead of showing the Beaker logs.
Since ipa-run-tests sets up its own logging, Nose's own log
handling just causes duplicate messages. Disable it with --nologcapture.
- Use the external hostname when connecting to remote hosts
- Make it possible to specify working directory for remote commands
- Move kinit calls to installation code
This allows tests where installation is done later
- Log at error level when a remote command fails unexpectedly
- Clean up test directory before testing
- Break infinite recursion in mkdir_recursive if dir can't be created
Set up the hostname, /etc/resolv.conf, and /etc/hosts on remote
hosts in the test setup.
Undo the changes in test teardown.
Part of the work for https://fedorahosted.org/freeipa/ticket/3621
Output from IPA's log manager is not captured by Nose's logcapture plugin.
Forward IPA logs to a regular Python logger so that they are shown
on failures.
IPA log messages are also shown on standard error.
Filter out Paramiko logs by default; these are too verbose.
Part of the work for: https://fedorahosted.org/freeipa/ticket/3621
After each test, and after class setups and teardowns, the BeakerLib
integration plugin now downloads log files from the remote masters
and submits them using rlFileSubmit.
Part of the work for: https://fedorahosted.org/freeipa/ticket/3621
Introduce a class inspired by subprocess.Popen that handles
running a command on a remote machine and handling its output.
To separate stdout & stderr streams of a remote command,
they need to be read in parallel, so that one of them doesn't
stall the runner when its buffer fills up. Accomplish this
by using a thread for each stream.
Part of the work for: https://fedorahosted.org/freeipa/ticket/3621
Add methods to run commands and copy files to Host objects.
Adds a base class for integration tests which can currently install
and uninstall IPA in a "star" topology with per-test specified number
of hosts.
A simple test for user replication between two masters is provided.
Log files from the remote hosts can be marked for collection, but the
actual collection is left to a Nose plugin.
Part of the work for: https://fedorahosted.org/freeipa/ticket/3621
Integration tests are configured via environment variables.
Add a framework for parsing these variables and storing them
in easy-to-use objects.
Add an `ipa-test-config` executable that loads the configuration
and prints out variables needed in shell scripts.
Part of the work for https://fedorahosted.org/freeipa/ticket/3621
Tests in test classes decorated by @ipatests.order_plugin.ordered
are sorted by the source line number instead of alphabetically,
if the plugin is enabled.
The ipa-run-tests helper now loads and enables the plugin.
This should make writing integration tests easier.
The plugin hooks into the Nose runner and IPA's logging infrastructure
and calls the appropriate BeakerLib functions (rl*).
IPA's log_manager is extended to accept custom Handler classes.
The ipa-run-tests helper now loads the plugin.
Patr of the work for: https://fedorahosted.org/freeipa/ticket/3621
sys.stdout is buffered by default if redirected to a file.
This may causes automated installation to appear hung.
Flush the stream so that messages are written immediately.
Since we depend on Dogtag 10 now, there is no need to keep code
that installs a Dogtag 9 CA.
Support for upgraded Dogtag-9-style instances is left in.
https://fedorahosted.org/freeipa/ticket/3529
- add missing closing parenthesis in idnsRecord declaration
- remove extra dollar sign from ipaSudoRule declaration
- handle missing/extraneous X-ORIGIN lines in 10-selinuxusermap.update
This does not use the schema updater because the syntax needs to be
fixed in the files themselves, otherwise 389 1.3.2+ will fail
to start.
Older DS versions transparently fix the syntax errors.
The existing ldap-updater directive for ipaSudoRule is fixed
(ldap-updater runs after upgradeconfig).
https://fedorahosted.org/freeipa/ticket/3578
This will convert a master with a selfsign CA to a CA-less one in
ipa-upgradeconfig.
The relevant files are left in place and can be used to manage certs
manually.
Part of the work for: https://fedorahosted.org/freeipa/ticket/3494
ipa <command> -h only showed the summary string, not the full help.
Use the full docstring. Add a custom help formatter that disables
optparse's reformatting.
Test included
https://fedorahosted.org/freeipa/ticket/3543
The CA cert was not loaded, so if it was missing from the PKCS#12 file,
installation would fail.
Pass the cert filename to the server installers and include it in
the NSS DB.
Part of the work for: https://fedorahosted.org/freeipa/ticket/3363
The CertDB class was meant to be a wrapper around NSS databases,
certutil, pk12util, etc. Unfortunately, over time it grew too
dependent on the particular scenarios it is used in.
Introduce a new class that has no knowledge about IPA configuration,
and move generic code to it.
In the future, generic code should be moved to NSSDatabase, code
for the self-signed CA should be removed, and IPA-specific code may
stay in CertDB (which calls NSSDatabase).
We pass names of files with pkcs12 pins to installers which may continue to
use the files after the initial call to create_instance, at which point
the installer has already removed them.
Also, some of the files were not properly removed on failure.
Use ipautil.write_tmp_file for the pin files, which returns a
NamedTemporaryFile object that removes the underlying file when it is
garbage-collected.
Create the files at start of installation. This will allow checking
the pkcs#12 files before the system is modified.
These used ipautil.get_ipa_basedn. Convert that to use the new wrappers.
Beef up the error handling in ipaldap to accomodate the errors we catch
in the server discovery.
Add a DatabaseTimeout exception to errors.py.
These were the last uses of ipautil.convert_ldap_error, remove that.
https://fedorahosted.org/freeipa/ticket/3487https://fedorahosted.org/freeipa/ticket/3446
Petr Viktorin