Commit Graph

556 Commits

Author SHA1 Message Date
Jan Cholasta
9117a5d5a6 paths: remove DEV_NULL
The platform-specific path to /dev/null is provided by the Python standard
library in os.devnull.

Replace all uses of paths.DEV_NULL with os.devnull and remove DEV_NULL.

https://fedorahosted.org/freeipa/ticket/6474

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2016-11-29 14:50:51 +01:00
Christian Heimes
7fef9cbec7 Fix Python 3 bugs discovered by pylint
In Python 3 exception instances no longer have a message attribute.
For most exceptions, str(e) or string formatting give the same result.

Fix some renamed modules, module members and functions.

https://fedorahosted.org/freeipa/ticket/4985

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-11-25 16:18:22 +01:00
Christian Heimes
38e8719f72 Python3 pylint fixes
Sprinkle 'pylint disable' comments over the code base to silence a bunch
of pylint warnings on Python 3. All silenced warnings are harmless and
not bugs.

https://fedorahosted.org/freeipa/ticket/4985

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-11-25 16:18:22 +01:00
Christian Heimes
2cbaf15604 Replace LooseVersion
pylint is having a hard time with distutils.version in tox's virtual
envs. virtualenv uses some tricks to provide a virtual distutils
package, pylint can't cope with.

https://github.com/PyCQA/pylint/issues/73 suggests to use pkg_resources
instead. pkg_resources' version parser has some more benefits, e.g. PEP
440 conformity. But pkg_resources.parse_version() is a heavy weight solution
with reduced functionality, e.g. no access to major version.

For API_VERSION and plugin version we can use a much simpler and faster
approach.

https://fedorahosted.org/freeipa/ticket/6468

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
2016-11-24 15:46:40 +01:00
Martin Basti
55b14abcb5 remove Knob function
`Knob` function is an old implementation which was replcaed by `knob`
function and currently is unused, so it can be removed

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2016-11-22 17:38:38 +01:00
Christian Heimes
c93bfda594 ipapython and ipatest no longer require lxml
Commits 64af88fe and 9fbd29cc have removed dependency on lxml.

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-11-17 11:29:28 +01:00
Florence Blanc-Renaud
efb3700389 Fix ipa migrate-ds when it finds a search reference
When ipa migrate-ds finds user entries and a search reference, it complains
that the LDAP search did not return any result and does not migrate the
entries or the groups.

The issue comes from LDAPClient._convert_result which returns an empty result
list when the input is a search reference. In turn LDAPClient.find_entries
assumes that the empty result list corresponds to a Search Result Done and
returns without any entry.

The fix examines first the objtype returned by self.conn.result3. If it is
a search result done, then the loop can be exited. Otherwise (referral or
entry), _convert_result is called and the result (if not empty) is appended
to the list of returned entries.

https://fedorahosted.org/freeipa/ticket/6358

Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-11-17 01:01:05 +01:00
Christian Heimes
9102fb3b02 Register entry points of Custodia plugins
With setuptools in place FreeIPA is able to register its Custodia
plugins. Custodia 0.1 ignores the plugins directives. Custodia 0.2 uses
the entry points to discover plugins.

https://fedorahosted.org/freeipa/ticket/6492

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
2016-11-16 23:47:38 +01:00
Christian Heimes
64af88fee4 Port ipapython.dnssec.odsmgr to xml.etree
The module ipapython.dnssec.odsmgr is the only module in ipalib,
ipaclient, ipapython and ipaplatform that uses lxml.etree.

https://fedorahosted.org/freeipa/ticket/6469

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-11-16 23:37:46 +01:00
Christian Heimes
8346e1b067 Add install requirements to Python packages
https://fedorahosted.org/freeipa/ticket/6468

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-11-16 22:58:39 +01:00
Jan Cholasta
714699a81f install: allow specifying verbosity and console log format in CLI
https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-11-11 12:17:25 +01:00
Jan Cholasta
08a446a6bc install: fix subclassing of knob groups
Add new @group decorator to declare an installer class as a knob group
instead of subclassing Group, so that subclassing the installer does not
create duplicates of the original group.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-11-11 12:17:25 +01:00
Jan Cholasta
269ca6c454 install: make knob base declaration explicit
Declare knob bases explicitly using a keyword argument instead of guessing
if the type argument is a base or a type of the knob.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-11-11 12:17:25 +01:00
Jan Cholasta
043c262ce4 install: declare knob CLI names using the argparse convention
Replace cli_name, cli_short_name and cli_positional knob arguments with a
single cli_names argument, which allows defining one or more CLI names
using the argparse convention ("--option" for long option name, "-o" for
short option name and "argument" for positional argument name).

Also replace cli_aliases with cli_deprecated_names which uses the same
convention.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-11-11 12:17:25 +01:00
Jan Cholasta
a929ac3338 install: use standard Python classes to declare knob types
Use type(None) rather than bool to define knobs which are represented as
command line flags. This allows declaring both "--option" and
"--option={0,1}"-style command line options.

Use enum.Enum subclasses instead of set literals to declare enumerations.

Use typing.List[T] instead of (list, T) to declare lists. (Note that a
minimal reimplementation of typing.List is used instead of the Python 2
backport of the typing module due to non-technical reasons.)

Use CheckedIPAddress instead of 'ip' and 'ip-local' to declare IP
addresses.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-11-11 12:17:25 +01:00
Jan Cholasta
9fd1981ae8 install: introduce updated knob constructor
Add new knob() knob constructor. Keep the old Knob() constructor for
backward compatibility with old code.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-11-11 12:17:25 +01:00
Jan Cholasta
be0c1afa74 install: simplify CLI option parsing
Let IPAOptionParser handle parsing of its supported types and use an option
callback only for unsupported types.

Instead of parsing positional arguments manually, parse them using a custom
IPAOptionParser instance, reusing the option parsing code.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-11-11 12:17:25 +01:00
Jan Cholasta
a641e279ff install: improve CLI positional argument handling
Instead of specifying which knobs should be positional arguments in
cli.install_tool(), do it using a flag in knob definition, where the rest
of CLI configuration is.

As a side effect, the usage string for CLI tools can now be generated
automatically.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-11-11 12:17:25 +01:00
Tomas Krizek
a68c95d116 ipaldap: remove do_bind from LDAPClient
Remove do_bind() method that was a relict used in IPAdmin. Replace
its uses with simple / external binds.

https://fedorahosted.org/freeipa/ticket/6461

Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-11-10 17:40:49 +01:00
Florence Blanc-Renaud
808b1436b4 Refactor installer code requesting certificates
- Temporary modify certmonger dogtag-ipa-ca-renew helper to request the IPA RA
agent cert, using the temp cert created during pkispawn. The cert request
is now processed through certmonger, and the helper arguments are restored
once the agent cert is obtained.

- Modify the installer code creating HTTP and LDAP certificates to use
certmonger's IPA helper with temporary parameters (calling dogtag-submit
instead of ipa-submit)

- Clean-up for the integration tests: sometimes ipa renewal.lock is not
released during ipa-server-uninstall. Make sure that the file is removed
to allow future installations.

https://fedorahosted.org/freeipa/ticket/6433

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
2016-11-10 14:15:57 +01:00
Fraser Tweedale
db116f73fe x509: use python-cryptography to process certs
Update x509.load_certificate and related functions to return
python-cryptography ``Certificate`` objects.  Update the call sites
accordingly, including removal of NSS initialisation code.

Also update GeneralName parsing code to return python-cryptography
GeneralName values, for consistency with other code that processes
GeneralNames.  The new function, `get_san_general_names`, and
associated helper functions, can be removed when python-cryptography
provides a way to deal with unrecognised critical extensions.

Part of: https://fedorahosted.org/freeipa/ticket/6398

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
2016-11-10 10:21:47 +01:00
Fraser Tweedale
9522970bfa dn: support conversion from python-cryptography Name
The upcoming change to using python-cryptography for certificate
process will require a way to convert
``cryptography.x509.name.Name`` values to ``ipapython.dn.DN``.
Update the ``DN`` constructor to accept a ``Name``.

Part of: https://fedorahosted.org/freeipa/ticket/6398

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
2016-11-10 10:21:47 +01:00
Petr Spacek
8de11b091f Build: Makefiles for Python packages
This version builds only one version of Python packages. If you want to
build for Python 2 & 3 call configure twice using different --with-python
or specify PYTHON variable when calling make.

dist-hook is using SOURCES.txt file from egg-info.
According to Petr Viktorin this should be enough for our purposes
and avoids need to create plugins for setuptools.

Currently VPATH builds do not work for various reasons.
This should be fixed later on.

Most credit goes to these guys:
 Christian Heimes <cheimes@redhat.com>
 Petr Viktorin <pviktori@redhat.com>
 Kevin Brown <kevin@kevin-brown.com>

https://fedorahosted.org/freeipa/ticket/6418

Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
2016-11-09 13:08:32 +01:00
Petr Spacek
c48e5fd811 Build: move version handling from Makefile to configure
Version information is now in VERSION.m4 instead of VERSION.
Makefile target version-update was minimized and configure can be run
before make. Makefile temporarily contains hardcoded version which has
to match the one specified in VERSION.m4.

This is preparatory step which will allow us to replace hand-made
Makefile with one generated by Automake.

https://fedorahosted.org/freeipa/ticket/6418

Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
2016-11-09 13:08:32 +01:00
Tomas Krizek
5b81dbfda1 ipaldap: merge IPAdmin to LDAPClient
* move IPAdmin methods to LDAPClient
* add extra arguments (cacert, sasl_nocanon) to LDAPClient.__init__()
* add host, port, _protocol to LDAPClient (parsed from ldap_uri)
* create get_ldap_uri() method to create ldap_uri from former
    IPAdmin.__init__() arguments
* replace IPAdmin with LDAPClient + get_ldap_uri()
* remove ununsed function argument hostname from
    enable_replication_version_checking()

https://fedorahosted.org/freeipa/ticket/6461

Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2016-11-07 11:34:03 +01:00
Tomas Krizek
4f1a6a1776 ipaldap: merge gssapi_bind to LDAPClient
* Rename do_sasl_gssapi_bind to gssapi_bind

https://fedorahosted.org/freeipa/ticket/6461

Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2016-11-07 11:34:03 +01:00
Tomas Krizek
60e38ecc7f ipaldap: merge external_bind into LDAPClient
* Rename do_external_bind to external_bind
* Remove user_name argument in  external_bind() and always set it
    to effective user name

https://fedorahosted.org/freeipa/ticket/6461

Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2016-11-07 11:34:03 +01:00
Tomas Krizek
de58a5c605 ipaldap: merge simple_bind into LDAPClient
* Use LDAPClient.simple_bind instead of extra call to IPAdmin.do_simple_bind
* Rename binddn to bind_dn
* Rename bindpw to bind_password
* Explicitly specify bind_dn in all calls

https://fedorahosted.org/freeipa/ticket/6461

Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2016-11-07 11:34:03 +01:00
Tomas Krizek
5760b7e983 ipaldap: remove wait/timeout during binds
Testing whether it is possible to connect to directory server is already done
in RedHatDirectoryService.restart().

https://fedorahosted.org/freeipa/ticket/6461

Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2016-11-07 11:34:03 +01:00
Christian Heimes
91920e7cb4 Add __name__ == __main__ guards to setup.pys
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Petr Spacek <pspacek@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Lenka Doudova <ldoudova@redhat.com>
2016-10-25 18:11:31 +02:00
Jan Cholasta
0d370a959b pylint: enable the import-error check
Check for import errors with pylint to make sure new python package
dependencies are not overlooked.

https://fedorahosted.org/freeipa/ticket/6418

Reviewed-By: Petr Spacek <pspacek@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-10-24 14:11:08 +02:00
Christian Heimes
e12a70a8b1 Remove ipapython/ipa.conf
The file ipapython/ipa.conf is no longer used and not installed.

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Petr Spacek <pspacek@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-10-20 18:43:37 +02:00
Christian Heimes
4cd83fb51c Port all setup.py to setuptools
All setup.py files are now using setuptools through a common file
ipasetup.py. The file is auto-generated and contain all common
settings.

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Petr Spacek <pspacek@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-10-20 18:43:37 +02:00
David Kupka
fb85230e25 UnsafeIPAddress: Implement __(g|s)etstate__ and to ensure proper (un)pickling
Missing attributes in instance created by pickle.load cause AttributeError in
second part of ipa-server-install --external-ca.

https://fedorahosted.org/freeipa/ticket/6385

Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2016-10-12 10:42:24 +02:00
Martin Babinsky
71f642f751 do not use keys() method when iterating through dictionaries
pylint-1.6.4-1.fc26.noarch reports "C0201(consider-iterating-dictionary)" when
building FreeIPA, we have to fix these errors

https://fedorahosted.org/freeipa/ticket/6391

Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-10-12 10:38:52 +02:00
Martin Babinsky
29829cc55a remove trailing newlines form python modules
pylint-1.6.4-1.fc26.noarch reports these, hence they should be fixed in order
to build FreeIPA with this version

https://fedorahosted.org/freeipa/ticket/6391

Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-10-12 10:38:52 +02:00
Martin Basti
4628522c53 Pylint: fix the rest of unused local variables
Reviewed-By: Pavel Vomacka <pvomacka@redhat.com>
2016-10-11 16:50:32 +02:00
Martin Basti
d937588146 Pylint: remove unused variables from installers and scripts
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2016-10-06 10:43:36 +02:00
Christian Heimes
ac94d32c4f Add iSecStore.span
In the future Custodia is going to make CSStore.span an abstract method.

Closes: https://fedorahosted.org/freeipa/ticket/6365
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-10-06 10:39:15 +02:00
Martin Babinsky
4d994bee60 Move character escaping function to ipautil
Functions `escape_seq` and `unescape_seq` have a generic use-case so it makes
sense to move them from `kerberos` to ipautil module so that other modules can
reuse them more readily.

https://fedorahosted.org/freeipa/ticket/5809

Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
2016-10-03 13:42:34 +02:00
Martin Basti
45e3aee352 Pylint: enable check for unused-variables
Unused variables may:
* make code less readable
* create dead code
* potentialy hide issues/errors

Enabled check should prevent to leave unused variable in code

Check is locally disabled for modules that fix is not clear or easy or have too many occurences of
unused variables

Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2016-09-27 13:35:58 +02:00
Martin Basti
0f88f8fe88 Remove unused variables in the code
This commit removes unused variables or rename variables as "expected to
be unused" by using "_" prefix.

This covers only cases where fix was easy or only one unused variable
was in a module

Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2016-09-27 13:35:58 +02:00
Martin Babinsky
f7764cda68 Make Continuous installer continuous only during execution phase
`common.Continuous` class is a basis for uninstallers, which should execute
all the steps regardless of occuring errors. However, we would like the
installer to raise exceptions and return non-zero exit code during validation
phase when some preconditions are not met.

Add a separate exception handler which catches exceptions and logs them as
errors during execution phase only.

https://fedorahosted.org/freeipa/ticket/5725

Reviewed-By: Petr Spacek <pspacek@redhat.com>
2016-09-26 18:38:37 +02:00
Martin Babinsky
347f5ca0e1 use separate exception handlers for executors and validators
installer framework has been modified to allow for different error handling
during validation and execution phases.

https://fedorahosted.org/freeipa/ticket/5725

Reviewed-By: Petr Spacek <pspacek@redhat.com>
2016-09-26 18:38:37 +02:00
Tomas Krizek
07ff1f619c Update man/help for --server option
The --server option now specifically mentions that
it expects the FQDN of the IPA server.

https://fedorahosted.org/freeipa/ticket/6202

Reviewed-By: Petr Spacek <pspacek@redhat.com>
2016-09-23 15:04:53 +02:00
Jan Barta
71b3352ad0 pylint: fix bad-mcs-method-argument
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
2016-09-22 16:52:57 +02:00
Jan Barta
8420d04f38 pylint: fix bad-mcs-classmethod-argument
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
2016-09-22 16:52:57 +02:00
Jan Barta
9bc57a01e1 pylint: fix old-style-class
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
2016-09-22 16:52:57 +02:00
Jan Barta
36484e8672 pylint: fix simplifiable-if-statement warnings
fix inefficient if statements, enable pylint check

Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
2016-09-22 16:52:57 +02:00
Tomas Krizek
ddf48f2fef Show error message for invalid IPs in client install
Re-raise the thrown exception to get an error message
instead of a traceback during ipa-client-install with
invalid IP address.

https://fedorahosted.org/freeipa/ticket/6340

Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
2016-09-21 10:26:24 +02:00