Commit Graph

40 Commits

Author SHA1 Message Date
Jan Barta
cdecbcd0a1 pylint: fix pointless-statement
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
2016-09-22 16:52:57 +02:00
Martin Basti
71ad8d4fc9 Allow broadcast ip addresses
Currently environments may use prefix /31 on point-to-point connections what
makes IPA validators to fail. IPA should not care if IP address is broadcast
or not. In some cases (when prefix is not specified) IPA cannot decide
properly if broadcast address is really broadcast.

This commit allows usage of broadcast addresses in:
* host plugin
* dns plugin
* server-installer
* client-installer

https://fedorahosted.org/freeipa/ticket/5814

Reviewed-By: David Kupka <dkupka@redhat.com>
2016-09-07 16:22:03 +02:00
Martin Basti
81d64d530c Allow network ip addresses
Currently cloud environments uses heavily prefix /32 (/128) what makes
IPA validators to fail. IPA should not care if IP address is network or not.
This commit allows usage of network addresses in:
* host plugin
* dns plugin
* server-installer
* client-installer

https://fedorahosted.org/freeipa/ticket/5814

Reviewed-By: David Kupka <dkupka@redhat.com>
2016-09-07 16:22:03 +02:00
Florence Blanc-Renaud
bc7eb99a29 Fix session cookies
The CLI was not using session cookies for communication with IPA API.
The kernel_keyring code was expecting the keyname to be a string, but
in python 2 a unicode was supplied (the key is built using
ipa_session_cookie:%principal and principal is a unicode).

The patch fixes the assertions, allowing to store and retrieve the cookie.
It also adds a test with unicode key name.

https://fedorahosted.org/freeipa/ticket/5984

Reviewed-By: Petr Spacek <pspacek@redhat.com>
2016-07-22 16:30:32 +02:00
Martin Babinsky
e6fc8f84d3 Test suite for ipapython/kerberos.py
Low-level unittests checking the correctness principal parsing.

https://fedorahosted.org/freeipa/ticket/3864

Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2016-07-01 09:37:25 +02:00
Martin Basti
d46cd5d956 Pylint: enable reimported check
Fixes current reimports and enables pylint check for them

Reviewed-By: Petr Spacek <pspacek@redhat.com>
Reviewed-By: Lukas Slebodnik <lslebodn@redhat.com>
2016-03-22 10:20:51 +01:00
Martin Basti
491447cc5a pylint: remove bare except
Bare except should not be used.

Reviewed-By: Petr Spacek <pspacek@redhat.com>
Reviewed-By: Lukas Slebodnik <lslebodn@redhat.com>
2016-03-22 10:20:51 +01:00
Petr Vobornik
25c53ba3fe cookie parser: do not fail on cookie with empty value
https://fedorahosted.org/freeipa/ticket/5709

Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-03-01 14:16:08 +01:00
Petr Viktorin
d3851fd76f ipautil.run, kernel_keyring: Encoding fixes for Python 3
https://fedorahosted.org/freeipa/ticket/5638

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2016-02-17 10:41:29 +01:00
Martin Basti
2ce8921fe6 make lint: use config file and plugin for pylint
Our custom implementation of pylint checker is often broken by
incompatible change on pylint side. Using supported solutions (config
file, pylint plugins) should avoid this issue.

The plugin adds missing (dynamic) member to classes in abstract syntax
tree generated for pylint, instead of just ignoring missing members and
all sub-members. This should improve pylint detection of typos and
missing members in api. env and test config.

make-lint python script has been removed, to run pylint execute 'make
lint'

https://fedorahosted.org/freeipa/ticket/5615

Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2016-02-11 11:06:39 +01:00
Jan Cholasta
500ee7e2b1 ipapython: port p11helper C code to Python
This replaces the binary _ipap11helper module with cffi-based Python code.

https://fedorahosted.org/freeipa/ticket/5596

Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-01-21 10:21:32 +01:00
Martin Basti
e1192ebd97 Remove wildcard imports
Wildcard imports should not be used.

Check for wildcard imports has been enabled in pylint.
Pylint note: options 'wildcard-import' causes too much false positive
results, so instead it I used 'unused-wildcard-import' option which has almost
the same effect.

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2015-12-23 07:59:22 +01:00
Martin Basti
e4075b1fe2 Remove unused imports
This patch removes unused imports, alse pylint has been configured to
check unused imports.

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2015-12-23 07:59:22 +01:00
Petr Viktorin
099cf98307 Refactor ipautil.run
The ipautil.run function now returns an object with returncode and
output are accessible as attributes.

The stdout and stderr of all commands are logged (unless skip_output is given).

The stdout/stderr contents must be explicitly requested with a keyword
argument, otherwise they are None.
This is because in Python 3, the output needs to be decoded, and that can
fail if it's not decodable (human-readable) text.

The raw (bytes) output is always available from the result object,
as is "leniently" decoded output suitable for logging.

All calls are changed to reflect this.

A use of Popen in cainstance is changed to ipautil.run.

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2015-12-14 10:54:23 +01:00
Milan Kubik
0a64e9bd70 Applied tier0 and tier1 marks on unit tests and xmlrpc tests
Web UI tests were marked as tier1 tests.

The tier system is intended to be used together with CI system
to make sure the more complicated tests are being run only
when all of the basic functionality is working.

The system is using pytest's marker system. E.g. an invocation of
all tier1 tests with listing will look like:

    $ py.test -v -m tier1 ipatests

or in case of out of tree tests:

    $ ipa-run-tests -m tier1

Reviewed-By: Ales 'alich' Marecek <amarecek@redhat.com>
2015-11-09 11:49:17 +01:00
Petr Viktorin
acf519f5c5 ipapython.secrets: Port to Python 3
StringIO was renamed in Python 3. The import was was unused,
so remove it.

Files need to be opened in binary mode if bytes are written to them.
(For Python 2: on Linux, there's no practical difference between
text and binary mode)

Reviewed-By: Tomas Babej <tbabej@redhat.com>
2015-10-27 17:23:25 +01:00
Simo Sorce
463dda3067 Add ipa-custodia service
Add a customized Custodia daemon and enable it after installation.
Generates server keys and loads them in LDAP autonomously on install
or update.
Provides client code classes too.

Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2015-10-15 14:24:33 +02:00
Petr Viktorin
c44dd40b26 test_keyring: Use str(e) instead of e.message for exceptions
Reviewed-By: Tomas Babej <tbabej@redhat.com>
2015-10-13 14:16:32 +02:00
Petr Viktorin
59d87d53b1 Do not compare types that are not comparable in Python 3
In Python 3, different types are generally not comparable (except for equality),
and None can't be compared to None.
Fix cases of these comparisons.

In ipatest.util, give up on sorting lists if the sorting raises a TypeError.

Reviewed-By: Tomas Babej <tbabej@redhat.com>
2015-10-13 14:16:32 +02:00
Petr Viktorin
dd0bfefae8 ipapython.ssh: Port to Python 3
Sort out the accepted types.
Handle Python 3's stricter separation between bytes and unicode.

Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2015-10-07 10:27:20 +02:00
Petr Viktorin
60d626845d test_dn: Split bytes and unicode
Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2015-10-07 10:27:20 +02:00
Petr Viktorin
ed96f8d9ba ipapython.dn: Use rich comparisons
__cmp__ and cmp were removed from Python 3.

Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2015-10-07 10:27:20 +02:00
Jan Cholasta
23507e6124 Alias "unicode" to "str" under Python 3
The six way of doing this is to replace all occurences of "unicode"
with "six.text_type". However, "unicode" is non-ambiguous and
(arguably) easier to read. Also, using it makes the patches smaller,
which should help with backporting.

Reviewed-By: Petr Viktorin <pviktori@redhat.com>
2015-09-17 11:08:43 +02:00
Petr Viktorin
70d1c71e46 Convert zip() result to list()
In Python 3, zip() returns an iterator. To get a list, it must
be explicitly converted.
In most cases, zip() result is iterated over so this is not
necessary.

Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2015-09-01 11:42:01 +02:00
Petr Viktorin
5178e9a597 Modernize use of range()
In Python 3, range() behaves like the old xrange().
The difference between range() and xrange() is usually not significant,
especially if the whole result is iterated over.

Convert xrange() usage to range() for small ranges.
Use modern idioms in a few other uses of range().

Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2015-09-01 11:42:01 +02:00
Petr Viktorin
3bf91eab25 Use Python3-compatible dict method names
Python 2 has keys()/values()/items(), which return lists,
iterkeys()/itervalues()/iteritems(), which return iterators,
and viewkeys()/viewvalues()/viewitems() which return views.

Python 3 has only keys()/values()/items(), which return views.
To get iterators, one can use iter() or a for loop/comprehension;
for lists there's the list() constructor.

When iterating through the entire dict, without modifying the dict,
 the difference between Python 2's items() and iteritems() is
negligible, especially on small dicts (the main overhead is
extra memory, not CPU time). In the interest of simpler code,
this patch changes many instances of iteritems() to items(),
iterkeys() to keys() etc.

In other cases, helpers like six.itervalues are used.

Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2015-09-01 11:42:01 +02:00
Petr Viktorin
6a741b51da Replace dict.has_key with the 'in' operator
The deprecated has_key method will be removed from dicts in Python 3.

For custom dict-like classes, has_key() is kept on Python 2,
but disabled for Python 3.

Reviewed-By: Tomas Babej <tbabej@redhat.com>
2015-08-12 18:17:23 +02:00
Petr Viktorin
27dabb4528 Modernize 'except' clauses
The 'as' syntax works from Python 2 on, and Python 3 will
drop the "comma" syntax.

Reviewed-By: Tomas Babej <tbabej@redhat.com>
2015-08-12 18:17:23 +02:00
Petr Viktorin
2cafb47ed7 Remove Editable DN and DN component classes
Make all DNs, RDNs and AVAs immutable.
Immutability makes reasoning about DN-handling code easier,
as value objects can't be changed once created.
Instead of mutable DNs, one can use a list (or even a generator)
of RDNs that's converted to a DN on output.

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2015-04-23 11:34:39 +00:00
Petr Vobornik
11bd9d96f1 performance: faster DN implementation
DN code was optimized to be faster if DNs are created from string. This is
the major use case, since most DNs come from LDAP.

With this patch, DN creation is almost 8-10x faster (with 30K-100K DNs).

Second mojor use case - deepcopy in LDAPEntry is about 20x faster - done by
custom __deepcopy__ function.

The major change is that DN is no longer internally composed  of RDNs and
AVAs but it rather keeps the data in open ldap format - the same as output
of str2dn function. Therefore, for immutable DNs, no other transformations
are required on instantiation.

The format is:

DN: [RDN, RDN,...]
RDN: [AVA, AVA,...]
AVA: ['utf-8 encoded str - attr', 'utf-8 encode str -value', FLAG]
FLAG: int

Further indexing of DN object constructs an RDN which is just an encapsulation
of the RDN part of open ldap representation. Indexing of RDN constructs AVA in
the same fashion.

Obtained EditableAVA, EditableRDN from EditableDN shares the respected lists
of the open ldap repr. so that the change of value or attr is reflected in
parent object.

Reviewed-By: Petr Viktorin <pviktori@redhat.com>
2015-04-14 19:31:54 +02:00
Milan Kubik
59f024487e ipatests: port of p11helper test from github
Ported the github hosted [1] script to use pytest's abilities
and included it in ipatests/test_ipapython directory.

[1]: https://github.com/spacekpe/freeipa-pkcs11/blob/master/python/run.py

https://fedorahosted.org/freeipa/ticket/4829

Signed-off-by: Martin Basti <mbasti@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2015-04-02 08:51:27 +00:00
Petr Viktorin
61c4ecccc1 Run pylint on tests
Drop support for pylint < 1.0

Enable ignoring unknown attributes on modules (both nose and pytest
use advanced techniques, support for which only made it to pylint
recently)

Fix some bugs revealed by pylint

Do minor refactoring or add pylint:disable directives where the
linter complains.

Reviewed-By: Tomas Babej <tbabej@redhat.com>
2015-01-14 11:40:28 +01:00
Petr Viktorin
387b8b46b8 test_ipapython: Use functions instead of classes in test generators
pytest's support for Nose-style test generators is not bulletproof;
use a real function to please it.

https://fedorahosted.org/freeipa/ticket/4610

Reviewed-By: Tomas Babej <tbabej@redhat.com>
2014-11-21 12:14:44 +01:00
Petr Viktorin
375e9f7c4b tests: Use PEP8-compliant setup/teardown method names
The setUp/dearDown names are used in the unittest module, but there is no reason
to use them in non-`unittest` test cases.
Nose supports both styles (but mixing them can cause trouble when
calling super()'s methods).
Pytest only supports the new ones.

https://fedorahosted.org/freeipa/ticket/4610

Reviewed-By: Tomas Babej <tbabej@redhat.com>
2014-11-21 12:14:44 +01:00
Xiao-Long Chen
5e96fbc22a Use /usr/bin/python2
Part of the effort to port FreeIPA to Arch Linux,
where Python 3 is the default.

FreeIPA hasn't been ported to Python 3, so the code must be modified to
run /usr/bin/python2

https://fedorahosted.org/freeipa/ticket/3438

Updated by pviktori@redhat.com
2014-01-03 09:46:05 +01:00
Petr Viktorin
a93fc02af6 Raise an error when updating CIDict with duplicate keys
Updating a CIDict with data like {'A': 1, 'a': 2} would lead to data
loss since only one of the items would get to the CIDict.
This can result in non-obvious bugs similar to this one in python-ldap:
https://bugzilla.redhat.com/show_bug.cgi?id=1007820

Raise an error in this case; any resolution must be done by the caller.
2013-09-25 10:13:56 +02:00
Petr Viktorin
0226064bac Add missing dict methods to CIDict
Make the CIDict interface match standard dict (except view* methods).

Add __contains__, __iter__, clear.
Add keyword and iterable support for __init__, update.
Also add values() and itervalues(). Previously the dict versions were
used; the new ones guarantee that the order matches keys().
Mark view* methods as not implemented.
CIDict.copy() now returns a CIDict.

Test the above additions, and fromkeys() which worked but wasn't tested.
2013-09-25 10:13:56 +02:00
Petr Viktorin
468e5e40cc Convert test_ipautil from unittest to nose 2013-09-25 10:13:56 +02:00
Petr Viktorin
3e505fe532 Move tests to test directories
Nose doesn't pick up directories that don't begin with 'test'.
Rename ipatests/test_ipaserver/install to test_install so that it's run.

Also, merge test_ipautil.py from ipapython/test into tests/test_ipapython,
so the whole test suite is in one place.
2013-09-25 10:13:56 +02:00
Petr Viktorin
c60142efda Make an ipa-tests package
Rename the 'tests' directory to 'ipa-tests', and create an ipa-tests RPM
containing the test suite

Part of the work for: https://fedorahosted.org/freeipa/ticket/3654
2013-06-17 19:22:50 +02:00