In Python 3, the keys() method of mappings returns a KeyView object
that reflects the mapping's state. In LDAPEntry, this means that
the collection returned by keys() is case-insensitive and supports
aliases.
Part of the fix for: https://fedorahosted.org/freeipa/ticket/4985
Reviewed-By: Martin Basti <mbasti@redhat.com>
Order of Python dicts/sets was always unreliable, but in Python 3
it's usually different every time. This affects the order in which
values of a LDAP attribute appear.
LDAP values are also specified to be unordered.
Part of the work for: https://fedorahosted.org/freeipa/ticket/4985
Reviewed-By: Martin Basti <mbasti@redhat.com>
When building RPMs with Python 3 support, /usr/bin/ipa will now
use Python 3.
The in-tree ipa command will also run on Python 3.
When building with make install, $(PYTHON) is honored and it will
still default to Python 2.
Part of the work for https://fedorahosted.org/freeipa/ticket/5638
Reviewed-By: Petr Spacek <pspacek@redhat.com>
Python 3 unsets the exception variable at the end of an "except"
block to prevent reference cycles and speed up garbage collection.
Store the exception under a different name in order to use it later.
Part of the work for https://fedorahosted.org/freeipa/ticket/4985
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Petr Spacek <pspacek@redhat.com>
In Python 3, dict.keys() returns a key view. It is not safe to delete
dict keys when iterating over this view.
Convert the keys to list first.
Part of the work for https://fedorahosted.org/freeipa/ticket/4985
Reviewed-By: Martin Basti <mbasti@redhat.com>
In Python 3, str.encode('ascii') converts to bytes, and str()
(nicknamed unicode() in IPA) returns the string representation
of an object, which is b'...' for bytes.
So, unicode('...'.encode('ascii')) results in "b'...'".
Change the code to only call encode() for the error.
Part of the work for https://fedorahosted.org/freeipa/ticket/4985
Reviewed-By: Martin Basti <mbasti@redhat.com>
For historical reasons, the string module contained some functions
that mirror methods of the str type. These are eremoved in Python 3.
Use str methods instead.
Part of the work for https://fedorahosted.org/freeipa/ticket/5638
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
ipacheckldap uses a schema-less connection with decode_attrs=False,
so bytes need to be decoded manually.
This was not a problem in Python2 where bytes and unicode could
be mixed freely.
Part of the work for https://fedorahosted.org/freeipa/ticket/5638
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
In Python 3, the error message from the decimal module is
less clear than before.
(It's apparently the price to pay for speed -- Python3 uses
libmpdec as its Decimal implementation by default.)
Don't check for the exact error message.
https://fedorahosted.org/freeipa/ticket/5638
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Make ipaclient a Python library like ipapython, ipalib, etc.
Use setup.py instead of autotools for installing it.
Move C client tools, Python scripts, and man pages, to client/.
Remove old, empty or outdated, boilerplate files (NEWS, README, AUTHORS).
Remove /setup-client.py (ipalib/setup.py should be used instead).
Update Makefiles and the spec file accordingly.
https://fedorahosted.org/freeipa/ticket/5638
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Pylint can be run with the --py3k switch to detect porting issues.
This is not compatible with regular checking (i.e. to do all checks,
pylint must be run twice, with and without --py3k).
So, do an additional run of pylint in a subprocess for the py3k checks.
Add a --no-py3k switch to skip the additional py3k run.
Also add a --no-lint switch to allow only running the py3 checks.
https://fedorahosted.org/freeipa/ticket/5623
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
- `file` was removed in favor of `open`. Switch to the new spelling.
- `buffer` was removed in favor of a buffer protocol (and memoryview),
and `reload` was moved to importlib.
Both are used in py2-only blocks, so just placate PyLint.
https://fedorahosted.org/freeipa/ticket/5623
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Follow-up to commit 23507e6124
The six way of doing this is to replace all occurences of "unicode"
with "six.text_type". However, "unicode" is non-ambiguous and
(arguably) easier to read. Also, using it makes the patches smaller,
which should help with backporting.
https://fedorahosted.org/freeipa/ticket/5623
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Pylint considers `print` a statement if the __future__ import is
not present, even if it's used like a function with one argument.
Add the __future__ import to files `pylint --py3k` complains about.
https://fedorahosted.org/freeipa/ticket/5623
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Running make with PYTHON=/usr/bin/python3 will build/install the
bits for Python 3.
Executable scripts in ipatests have symlinks Python version suffixes
as per Fedora guidelines. Suffix-less names point to the Python 2 versions.
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
The ipautil.run function now returns an object with returncode and
output are accessible as attributes.
The stdout and stderr of all commands are logged (unless skip_output is given).
The stdout/stderr contents must be explicitly requested with a keyword
argument, otherwise they are None.
This is because in Python 3, the output needs to be decoded, and that can
fail if it's not decodable (human-readable) text.
The raw (bytes) output is always available from the result object,
as is "leniently" decoded output suitable for logging.
All calls are changed to reflect this.
A use of Popen in cainstance is changed to ipautil.run.
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
The client XML-RPC implementation is tied to rpclib internals,
so with a change in Python it needs to be updated. And rpclib
changed in Python 3.
Reviewed-By: Tomas Babej <tbabej@redhat.com>
StringIO was renamed in Python 3. The import was was unused,
so remove it.
Files need to be opened in binary mode if bytes are written to them.
(For Python 2: on Linux, there's no practical difference between
text and binary mode)
Reviewed-By: Tomas Babej <tbabej@redhat.com>
In Python 3, the "message" property was removed in favor of calling str().
Add it to PublicMessage, since IPA code depends on it.
Reviewed-By: Tomas Babej <tbabej@redhat.com>
In Python 3, the base64.b64decode function raises binascii.Error (a ValueError
subclass) when it finds incorrect padding. In Python 2 it raises TypeError.
Callers should usually handle ValueError; unless they are specifically
concerned with handling base64 padding issues).
In some cases, callers should handle ValueError:
- ipalib.pkcs10 (get_friendlyname, load_certificate_request): callers should
handle ValueError
- ipalib.x509 (load_certificate*, get_*): callers should handle ValueError
In other cases ValueError is handled:
- ipalib.parameters
- ipapython.ssh
- ipalib.rpc (json_decode_binary - callers already expect ValueError)
- ipaserver.install.ldapupdate
Elsewhere no error handling is done, because values come from trusted
sources, or are pre-validated:
- vault plugin
- ipaserver.install.cainstance
- ipaserver.install.certs
- ipaserver.install.ipa_otptoken_import
Reviewed-By: Tomas Babej <tbabej@redhat.com>
In Python 2, numbers prfixed with '0' are parsed as octal,
e.g. '020' -> 16. In Python 3, the prefix is '0o'.
Handle the old syntax for IPA's parameter conversion to keep
backwards compatibility.
Reviewed-By: Tomas Babej <tbabej@redhat.com>
Python 3 removes the "message" attribute from exceptions, in favor
of just calling str().
Add it back for IPA's own exception types.
Reviewed-By: Tomas Babej <tbabej@redhat.com>
- Don't encode under Python 3, where shlex would choke on bytes
- Sort the attrs dictionary in export_to_string, so the tests are
deterministic. (The iteration order of dicts was always unspecified,
but was always the same in practice under CPython 2.)
Reviewed-By: Tomas Babej <tbabej@redhat.com>
Python 3 uses plain function objects instead of unbound methods.
So, what was Class.method.__func__ is now just Class.method.
Reviewed-By: Tomas Babej <tbabej@redhat.com>
In Python 3, the variable with the currently handled exception is unset
at the end of the except block. (This is done to break reference
cycles, since exception instances now carry tracebacks, which contain
all locals.)
Fix this in baseldap's error handler.
Use a simpler structure for the ipatests.raises utility that only uses the
exception inside the except block.
Reviewed-By: Tomas Babej <tbabej@redhat.com>
In python 3 , `bytes` has the buffer interface, and `buffer` was removed.
Also, invalid padding in base64-encoded data raises a ValueError rather
than TypeError.
In tests, use pytest.assert_raises for more correct exception assertions.
Also, get rid of unused imports in the tests
Reviewed-By: Tomas Babej <tbabej@redhat.com>
In Python 3, different types are generally not comparable (except for equality),
and None can't be compared to None.
Fix cases of these comparisons.
In ipatest.util, give up on sorting lists if the sorting raises a TypeError.
Reviewed-By: Tomas Babej <tbabej@redhat.com>
Sort out the accepted types.
Handle Python 3's stricter separation between bytes and unicode.
Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
In Python 3, the types module no longer provide alternate names for
built-in types, e.g. `types.StringType` can just be spelled `str`.
NoneType is also removed; it needs to be replaced with type(None)
Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
The StringIO class was moved to the io module.
(In Python 2, io.StringIO is available, but is Unicode-only.)
Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
In Python 3, integers don't have a maximum. The number called
"sys.maxint" is now "sys.maxsize" (defined as larger than the
largest possible list/string index).
The new spelling is also available in Python 2.7.
Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
- Use PyLong instead of PyInt on Python 3
- Use PyBytes instead of PyString for binary strings
- Use Py_TYPE(o) instead of o->py_type, and PyVarObject_HEAD_INIT
instead of PyObject_HEAD_INIT
- Use a helper function instead of Py_BuildValue("s#"), so that
bytes are returned on Python 3
- Use new style module initialization on Python 3
- Use PyModule_AddIntConstant and PyModule_AddIntMacro for adding constants
Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
In Python 3, zip() returns an iterator. To get a list, it must
be explicitly converted.
In most cases, zip() result is iterated over so this is not
necessary.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
In Python 3, range() behaves like the old xrange().
The difference between range() and xrange() is usually not significant,
especially if the whole result is iterated over.
Convert xrange() usage to range() for small ranges.
Use modern idioms in a few other uses of range().
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
The three-argument raise is going away in Python 3. Use the six.reraise
helper instead.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
The form`raise Error, value` is deprecated in favor of `raise Error(value)`,
and will be removed in Python 3.
Use the new syntax.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
In Python 3, `print` is no longer a statement. Call it as a function
everywhere, and include the future import to remove the statement
in Python 2 code as well.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
In Python 3, next() for iterators is a function rather than method.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
In Python 2, map() returns a list; in Python 3 it returns an iterator.
Replace all uses by list comprehensions, generators, or for loops,
as required.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
In Python 3, raw_input() was renamed to input().
Import the function from six.moves to get the right version.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
In Python 3, filter() returns an iterator.
Use list comprehensions instead.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Python 2 has keys()/values()/items(), which return lists,
iterkeys()/itervalues()/iteritems(), which return iterators,
and viewkeys()/viewvalues()/viewitems() which return views.
Python 3 has only keys()/values()/items(), which return views.
To get iterators, one can use iter() or a for loop/comprehension;
for lists there's the list() constructor.
When iterating through the entire dict, without modifying the dict,
the difference between Python 2's items() and iteritems() is
negligible, especially on small dicts (the main overhead is
extra memory, not CPU time). In the interest of simpler code,
this patch changes many instances of iteritems() to items(),
iterkeys() to keys() etc.
In other cases, helpers like six.itervalues are used.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
This feature was removed in Python 3 to ease introspection.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
sys.exc_value is deprecated since Python 1.5, and was removed in Python 3.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Petr Viktorin