Commit Graph

793 Commits

Author SHA1 Message Date
Christian Heimes
964a9bdcec Py3: Replace six.string_types with str
In Python 3, six.string_types is just an alias for str.

See: https://pagure.io/freeipa/issue/7715
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
2018-09-27 16:11:18 +02:00
Christian Heimes
b431e9b684 Py3: Remove subclassing from object
Python 2 had old style and new style classes. Python 3 has only new
style classes. There is no point to subclass from object any more.

See: https://pagure.io/freeipa/issue/7715
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
2018-09-27 11:49:04 +02:00
Christian Heimes
4a58adf79e Sprinkle raw strings across the code base
tox / pytest is complaining about lots and lots of invalid escape
sequences in our code base. Sprinkle raw strings or backslash escapes
across the code base to fix most occurences of:

  DeprecationWarning: invalid escape sequence

There is still one warning that keeps repeating, though:

  source:264: DeprecationWarning: invalid escape sequence \d

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
2018-09-27 10:23:03 +02:00
Christian Heimes
21f14e9893 Remove Python 2 support and packages
Remove Python 2 related code and configuration from spec file, autoconf
and CI infrastructure. From now on, FreeIPA 4.8 requires at least Python
3.6. Python 2 packages like python2-ipaserver or python2-ipaclient are
no longer available. PR-CI, lint, and tox aren't testing Python 2
compatibility either.

See: https://fedoraproject.org/wiki/Changes/FreeIPA_Python_2_Removal
Fixes: https://pagure.io/freeipa/issue/7568
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
2018-09-06 17:39:00 +02:00
Florence Blanc-Renaud
f0228fa649 uninstall -v: remove Tracebacks
ipa-server-install --uninstall -v -U prints Traceback in its log file.
This issue happens because it calls subprocess.Popen with close_fds=True
(which closes all file descriptors in the child process)
but it is trying to use the file logger in the child process
(preexec_fn is called in the child just before the child is executed).
The fix is using the logger only in the parent process.

Fixes: https://pagure.io/freeipa/issue/7681
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
2018-08-23 12:06:45 +02:00
Alexander Scheel
fc0f6b4351 Add missing docstrings to kernel_keyring.py
Signed-off-by: Alexander Scheel <ascheel@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
2018-08-20 17:58:16 -04:00
Alexander Scheel
3322aad7da Add docstring to verify_kdc_cert_validity
Signed-off-by: Alexander Scheel <ascheel@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
2018-08-20 17:58:16 -04:00
Stanislav Laznicka
d473278621
ipatests: add installer framework testing
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
2018-07-19 08:42:33 +02:00
Armando Neto
3ccd512dab Disable Pylint 2.0 violations
Globally disabling the following violations:

- `assignment-from-no-return` (E1111):
  Assigning to function call which doesn't return. Used when an
  assignment is done on a function call but the inferred function
  doesn't return anything.

- `keyword-arg-before-vararg` (W1113):
  Keyword argument before variable positional arguments list in the
  definition of %s function When defining a keyword argument before
  variable positional arguments, one can end up in having multiple
  values passed for the aforementioned parameter in case the method is
  called with keyword arguments.

Locally disabling the following:

- `subprocess-popen-preexec-fn` (W1509):
  Using preexec_fn keyword which may be unsafe in the presence of
  threads The preexec_fn parameter is not safe to use in the presence
  of threads in your application. The child process could deadlock
  before exec is called. If you must use it, keep it trivial! Minimize
  the number of libraries you call into.
  https://docs.python.org/3/library/subprocess.html#popen-constructor

Fixed violations:

- `bad-mcs-classmethod-argument` (C0204):
  Metaclass class method %s should have %s as first argument Used when
  a metaclass class method has a first argument named differently than
  the value specified in valid-metaclass-classmethod-first-arg option
  (default to "mcs"), recommended to easily differentiate them from
  regular instance methods.
  - Note: Actually `cls` is the default first arg for `__new__`.

- `consider-using-get` (R1715):
  Consider using dict.get for getting values from a dict if a key is
  present or a default if not Using the builtin dict.get for getting a
  value from a dictionary if a key is present or a default if not, is
  simpler and considered more idiomatic, although sometimes a bit slower

Issue: https://pagure.io/freeipa/issue/7614

Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
2018-07-16 17:03:35 +02:00
Armando Neto
d13571942e Fix Pylint 2.0 violations
Fix the following violations aiming to support Pylint 2.0

- `unneeded-not` (C0113):
  Consider changing "not item in items" to "item not in items" used
  when a boolean expression contains an unneeded negation.

- `useless-import-alias` (C0414):
  Import alias does not rename original package Used when an import
  alias is same as original package.e.g using import numpy as numpy
  instead of import numpy as np

- `raising-format-tuple` (W0715):
  Exception arguments suggest string formatting might be intended Used
  when passing multiple arguments to an exception constructor, the
  first of them a string literal containing what appears to be
  placeholders intended for formatting

- `bad-continuation` (C0330):
  This was already included on the disable list, although with current
  version of pylint (2.0.0.dev2) violations at the end of the files
  are not being ignored.
  See: https://github.com/PyCQA/pylint/issues/2278

- `try-except-raise` (E0705):
  The except handler raises immediately Used when an except handler
  uses raise as its first or only operator. This is useless because it
  raises back the exception immediately. Remove the raise operator or
  the entire try-except-raise block!

- `consider-using-set-comprehension` (R1718):
  Consider using a set comprehension Although there is nothing
  syntactically wrong with this code, it is hard to read and can be
  simplified to a set comprehension.Also it is faster since you don't
  need to create another transient list

- `dict-keys-not-iterating` (W1655):
  dict.keys referenced when not iterating Used when dict.keys is
  referenced in a non-iterating context (returns an iterator in
  Python 3)

- `comprehension-escape` (W1662):
  Using a variable that was bound inside a comprehension Emitted when
  using a variable, that was bound in a comprehension handler, outside
  of the comprehension itself. On Python 3 these variables will be
  deleted outside of the comprehension.

Issue: https://pagure.io/freeipa/issue/7614

Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
2018-07-14 12:04:19 +02:00
Christian Heimes
f89e501ee1 Handle races in replica config
When multiple replicas are installed in parallel, two replicas may try
to create the cn=replica entry at the same time. This leads to a
conflict on one of the replicas. replica_config() and
ensure_replication_managers() now handle conflicts.

ipaldap now maps TYPE_OR_VALUE_EXISTS to DuplicateEntry(). The type or
value exists exception is raised, when an attribute value or type is
already set.

Fixes: https://pagure.io/freeipa/issue/7566
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Thierry Bordaz <tbordaz@redhat.com>
2018-07-12 15:26:25 +02:00
Armando Neto
ba954efafd Fix pylint 2.0 conditional-related violations
In order to support pylint 2.0 the following violations must be fixed:

- `chained-comparison` (R1716):
  Simplify chained comparison between the operands This message is
  emitted when pylint encounters boolean operation like
  "a < b and b < c", suggesting instead to refactor it to "a < b < c".

- `consider-using-in` (R1714):
  Consider merging these comparisons with "in" to %r To check if a
  variable is equal to one of many values,combine the values into a
  tuple and check if the variable is contained "in" it instead of
  checking for equality against each of the values.This is faster
  and less verbose.

Issue: https://pagure.io/freeipa/issue/7614

Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
2018-07-12 08:49:43 +02:00
Armando Neto
b4ad0d19a2 Fix pylint 2.0 return-related violations
Aiming to support pylint 2.0 some functions and methods must have their
return statements updated in order to fix two new violations:

- `useless-return` (R1711):
  Useless return at end of function or method Emitted when a single
  "return" or "return None" statement is found at the end of function
  or method definition. This statement can safely be removed because
  Python will implicitly return None

- `inconsistent-return-statements` (R1710):
  Either all return statements in a function should return an
  expression, or none of them should. According to PEP8, if any return
  statement returns an expression, any return statements where no value
  is returned should explicitly state this as return None, and an
  explicit return statement should be present at the end of the
  function (if reachable)

Issue: https://pagure.io/freeipa/issue/7614

Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
2018-07-11 10:11:38 +02:00
Armando Neto
b274da726b Replace file.flush() calls with flush_sync() helper
Calls to `os.fsync(f.fileno())` need to be accompained by `f.flush()`.

Commit 8bbeedc93f introduces the helper
`ipapython.ipautil.flush_sync()`, which handles all calls in the right
order.

However, `flush_sync()` takes as parameter a file object with fileno
and name, where name must be a path to the file, this isn't possible
in some cases where file descriptors are used.

Issue: https://pagure.io/freeipa/issue/7251

Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
2018-07-07 10:20:01 +02:00
Christian Heimes
9c86d35a3f Cleanup shebang and executable bit
- Add missing executable bits to all scripts
- Remove executable bits from all files that are not scripts,
  e.g. js, html, and Python libraries.
- Remove Python shebang from all Python library files.

It's frown upon to have executable library files in site-packages.

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Armando Neto <abiagion@redhat.com>
2018-07-05 19:46:42 +02:00
Christian Heimes
198a2c6112 Import ABCs from collections.abc
Python 3 has moved all collection abstract base classes to
collections.abc. Python 3.7 started to deprecate the old aliases.

The whole import block needs to be protected with import-error and
no-name-in-module, because Python 2 doesn't have collections.abc module and
collections.abc.Mapping, while Python 3 doesn't have collections.Mapping.

Fixes: https://pagure.io/freeipa/issue/7609
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
2018-07-05 19:45:10 +02:00
Christian Heimes
6896c90eb2 Extend Sub CA replication test
Test more scenarios like replication replica -> master. Verify that master
and replica have all expected certs with correct trust flags and all keys.

See: https://pagure.io/freeipa/issue/7590
See: https://pagure.io/freeipa/issue/7589
Fixes: https://pagure.io/freeipa/issue/7611
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2018-07-04 09:32:54 +02:00
Rob Crittenden
b96906156b
Improve console logging for ipa-server-install
The server installation and uninstallation overlaps both the
server and client installers. The output could be confusing
with a server uninstall finishing with the message:

The ipa-client-install command was successful

This was in part due to the fact that the server was not
configured with a console format and verbose was False which
meant that no logger messages were displayed at all.

In order to suppress client installation errors and avoid
confusion add a list of errors to ignore. If a server install
was not successful and hadn't gotten far enough to do the
client install then we shouldn't complain loudly about it.

https://pagure.io/freeipa/issue/6760

Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
2018-06-20 08:38:03 +02:00
Christian Heimes
f90e137a17 Sort and shuffle SRV record by priority and weight
On multiple occasions, SRV query answers were not properly sorted by
priority. Records with same priority weren't randomized and shuffled.
This caused FreeIPA to contact the same remote peer instead of
distributing the load across all available servers.

Two new helper functions now take care of SRV queries. sort_prio_weight()
sorts SRV and URI records. query_srv() combines SRV lookup with
sort_prio_weight().

Fixes: https://pagure.io/freeipa/issue/7475
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
2018-06-19 08:56:46 +02:00
Christian Heimes
0a87de5ed5 Backport gzip.decompress for Python 2
Python 2 doesn't have gzip.decompress(data: bytes) -> bytes function.
Backport the two line function from Python 3.6.

Fixes: https://pagure.io/freeipa/issue/7563
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
2018-05-30 15:09:55 +02:00
Fraser Tweedale
1da3eddf56 Handle compressed responses from Dogtag
We currently accept compressed responses for some Dogtag resources,
via an 'Accept: gzip, deflate' header.  But we don't decompress the
received data.  Inspect the response Content-Encoding header and
decompress the response body according to its value.

The `gzip.decompress` function is only available on Python 3.2 or
later.  In earlier versions, it is necessary to use StringIO and
treat the compressed data as a file.  This commit avoids this
complexity.  Therefore it should only be included in Python 3 based
releases.

Fixes: https://pagure.io/freeipa/issue/7563
Reviewed-By: Christian Heimes <cheimes@redhat.com>
2018-05-30 15:09:55 +02:00
Stanislav Laznicka
f47d86c719 Move config directives handling code
Move config directives handling code:
        ipaserver.install.installutils -> ipapython.directivesetter

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
2018-05-29 17:03:56 +02:00
Christian Heimes
9a9c8ced30 Use sane default settings for ldap connections
LDAP connections no longer depend on sane settings in global ldap.conf
and use good default settings for cert validation, CA, and SASL canonization.

https://pagure.io/freeipa/issue/7418

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
2018-05-29 15:30:37 +02:00
Alexander Bokovoy
c61151f6aa pylint3: workaround false positives reported for W1662
Pylint3 falsely reports warning W1662: using a variable that was bound
inside a comprehension for the cases where the same name is reused for a
loop after the comprehension in question.

Rename the variable in a loop to avoid it.

If the code looks like the following:

  arr = [f for f in filters if callable(f)]
  for f in arr:
      result = result + f()

pylint3 would consider 'f' used outside of comprehension. Clearly, this
is a false-positive warning as the second 'f' use is completely
independent of the comprehension's use of 'f'.

Reviewed-By: Aleksei Slaikovskii <aslaikov@redhat.com>
2018-05-17 16:55:42 -04:00
Rob Crittenden
648d7c0d38 Disable message about log in ipa-backup if IPA is not configured
Introduce server installation constants similar to the client
but only tie in SERVER_NOT_CONFIGURED right now.

For the case of not configured don't spit out the "See <some log>
for more information" because no logging was actually done.

In the case of ipa-backup this could also be confusing if the
--log-file option was also passed in because it would not be
used.

https://pagure.io/freeipa/issue/6843

Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
2018-05-03 17:34:45 +02:00
Rob Crittenden
41352ef938 Revert run_pk12util part of 807a5cbe7c
Only certutil creates files in the local directory. Changing the
directory for pk12util breaks ipa-server-certinstall if the
PKCS#12 file is not passed in as an absolute path.

https://pagure.io/freeipa/issue/7489

Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Alexey Slaykovsky <alexey@slaykovsky.com>
2018-04-20 08:51:37 -04:00
Stanislav Laznicka
b5bdd07bc5
Add absolute_import future imports
Add absolute_import from __future__ so that pylint
does not fail and to achieve python3 behavior in
python2.

Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
2018-04-20 09:43:37 +02:00
Christian Heimes
807a5cbe7c certdb: Move chdir into subprocess call
According to a comment, certutil may create files in the current working
directory. Rather than changing the cwd of the current process,
FreeIPA's certutil wrapper now changes cwd for the subprocess only.

See: https://pagure.io/freeipa/issue/7416
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
2018-04-10 17:35:17 +02:00
Christian Heimes
6aca027ecc Fix installer CA port check for port 8080
The installer now checks that port 8080 is available and not in use by
any other application.

The port checker has been rewritten to use bind() rather than just
checking if a server responds on localhost. It's much more reliable and
detects more problems.

Original patch by m3gat0nn4ge.

Co-authored-by: Mega Tonnage <m3gat0nn4ge@gmail.com>
Fixes: https://pagure.io/freeipa/issue/7415
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
2018-04-04 08:58:48 +02:00
Rob Crittenden
64fca87a52 Remove the Continuous installer class, it is unused
https://pagure.io/freeipa/issue/7330

Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2018-03-19 17:38:41 +01:00
Rob Crittenden
68c7b03689 Return a value if exceptions are raised in server uninstall
The AdminTool class purports to "call sys.exit() with the return
value" but most of the run implementations returned no value, or
the methods they called returned nothing so there was nothing to
return, so this was a no-op.

The fix is to capture and bubble up the return values which will
return 1 if any exceptions are caught.

This potentially affects other users in that when executing the
steps of an installer or uninstaller the highest return code
will be the exit value of that installer.

Don't use the Continuous class because it doesn't add any
value and makes catching the exceptions more difficult.

https://pagure.io/freeipa/issue/7330

Signed-off-by: Rob Crittenden rcritten@redhat.com
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2018-03-19 17:38:41 +01:00
Christian Heimes
df99af4a68 Remove unused modutils wrappers from NSS/CertDB
The disable system trust feature is no longer used.

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2018-02-23 11:04:10 +01:00
Christian Heimes
66a32d8931 NSS: Force restore of SELinux context
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2018-02-23 11:04:10 +01:00
Christian Heimes
492e3c9b1e NSSDB: Let certutil decide its default db type
CertDB no longer makes any assumptions about the default db type of a NSS
DB. Instead it let's certutil decide when dbtype is set to 'auto'. This
makes it much easier to support F27 and F28 from a single code base.

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2018-02-23 11:04:10 +01:00
Christian Heimes
c5fb6c8502 Prepare migration of mod_nss NSSDB to sql format
- Refactor CertDB to look up values from its NSSDatabase.
- Add run_modutil() helpers to support sql format. modutil does not
  auto-detect the NSSDB format.
- Add migration helpers to CertDB.
- Add explicit DB format to NSSCertificateDatabase stanza
- Restore SELinux context when migrating NSSDB.
- Add some debugging and sanity checks to httpinstance.

The actual database format is still dbm. Certmonger on Fedora 27 does
neither auto-detect DB format nor support SQL out of the box.

https://pagure.io/freeipa/issue/7354

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2018-02-23 11:04:10 +01:00
Christian Heimes
9c2c3df0ab Add better CalledProcessError and run() logging
In case of an error, ipapython.ipautil.run() now raises an exception that
contains the error message of the failed command. Before the exception
only contained the command and error code.

The command is no longer collapsed into one string. The error message
and logging output contains the actual command and arguments with intact
quoting.

Example:
CalledProcessError(Command ['/usr/bin/python3', '-c', 'import sys; sys.exit(" ".join(("error", "XXXXXXXX")))'] returned non-zero exit status 1: 'error XXXXXXXX\n')

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
2018-02-20 13:03:01 +01:00
Stanislav Laznicka
f31797c70a Have all the scripts run in python 3 by default
The Python 3 refactoring effort is finishing, it should be safe
to turn all scripts to run in Python 3 by default.

https://pagure.io/freeipa/issue/4985

Reviewed-By: Christian Heimes <cheimes@redhat.com>
2018-02-15 18:43:12 +01:00
Christian Heimes
1b0c55a3b3 Unified ldap_initialize() function
Replace all ldap.initialize() calls with a helper function
ldap_initialize(). It handles cacert and cert validation correctly. It
also provides a unique place to handle python-ldap 3.0 bytes warnings in
the future.

Fixes: https://pagure.io/freeipa/issue/7411
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
2018-02-15 18:32:17 +01:00
Fraser Tweedale
4daac52ddd ipaldap: allow GetEffectiveRights on individual operations
Allow caller to specify that the GetEffectiveRights server control
should be used on a per-operation basis.  Also update
ldap2.get_effective_rights to use this new API.

Part of: https://pagure.io/freeipa/issue/6609

Reviewed-By: Christian Heimes <cheimes@redhat.com>
2018-02-09 08:57:41 +01:00
Christian Heimes
2391c75e3d Replace hard-coded paths with path constants
Several run() calls used hard-coded paths rather than pre-defined paths
from ipaplatform.paths. The patch fixes all places that I was able to
find with a simple search.

The fix simplifies Darix's port of freeIPA on openSuSE.

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
2018-02-08 09:32:12 +01:00
Christian Heimes
7924dae6ae Fix pylint error in ipapython/dn.py
ipapython/dn.py:1324: [R1710(inconsistent-return-statements), DN.__contains__]
Either all return statements in a function should return an expression, or none of them should.)

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
2018-01-23 17:10:16 +01:00
Christian Heimes
dc599e0797 LGTM: Remove redundant assignment
https://pagure.io/freeipa/issue/7344

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
2018-01-09 07:53:28 +01:00
Christian Heimes
a4f36eec0a LGTM: Name unused variable in loop
For loop variable '_nothing' is not used in the loop body. The name
'unused' is used to indicate that a variable is unused.

https://pagure.io/freeipa/issue/7344

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
2018-01-09 07:53:28 +01:00
Christian Heimes
3c59cf5728 Require python-ldap 3.0.0b2
Use new LDAPBytesWarning to ignore python-ldap's bytes warnings. New
build is available in @freeipa/freeipa-master.

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Felipe Volpone <fbarreto@redhat.com>
2017-12-19 14:05:29 +01:00
Christian Heimes
b567f3afea Use pylint 1.7.5 with fix for bad python3 import
Closes: https://pagure.io/freeipa/issue/7315
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Petr Cech <pcech@redhat.com>
2017-12-19 13:28:06 +01:00
Christian Heimes
8cb756a229 Fix pylint warnings inconsistent-return-statements
Add consistent return to all functions and methods that are covered by
tox -e pylint[23]. I haven't checked if return None is always a good
idea or if we should rather raise an error.

See: https://pagure.io/freeipa/issue/7326
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
2017-12-18 11:51:14 +01:00
Christian Heimes
1505922c2b NSSDB: use preferred convert command
After further testing, Kai Engert proposed to use -N with -f -@ to
convert a NSSDB from DBM to SQL format.

https://fedoraproject.org/wiki/Changes/NSSDefaultFileFormatSql#Upgrade.2Fcompatibility_impact

https://pagure.io/freeipa/issue/7049

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
2017-12-07 16:46:10 +01:00
Christian Heimes
c1f275f9eb Update to python-ldap 3.0.0
Replace python3-pyldap with python3-ldap.

Remove some old code for compatibility with very old python-ldap.

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
2017-12-06 16:54:04 +01:00
Florence Blanc-Renaud
19138c5ba3 Fix ca less IPA install on fips mode
When ipa-server-install is run in fips mode and ca-less, the installer
fails when the keys are provided with --{http|dirsrv|pkinit}-cert-file
in a separate key file.

The installer transforms the key into PKCS#8 format using
openssl pkcs8 -topk8
but this command fails on a fips-enabled server, unless the options
-v2 aes256 -v2prf hmacWithSHA256
are also provided.

Fixes:
https://pagure.io/freeipa/issue/7280

Reviewed-By: Christian Heimes <cheimes@redhat.com>
2017-11-27 17:51:03 +01:00
Christian Heimes
9e640190ee
Run tox tests for PyPI packages on Travis
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
2017-11-20 17:01:59 +01:00
Christian Heimes
0071744929 Support sqlite NSSDB
Prepare CertDB and NSSDatabase to support sqlite DB format. NSSDatabase
will automatically detect and use either old DBM or new SQL format. Old
databases are not migrated yet.

https://pagure.io/freeipa/issue/7049

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
2017-11-16 12:17:01 +01:00
Christian Heimes
a48f6511f6 Use namespace-aware meta importer for ipaplatform
Instead of symlinks and build-time configuration the ipaplatform module
is now able to auto-detect platforms on import time. The meta importer
uses the platform 'ID' from /etc/os-releases. It falls back to 'ID_LIKE'
on platforms like CentOS, which has ID=centos and ID_LIKE="rhel fedora".

The meta importer is able to handle namespace packages and the
ipaplatform package has been turned into a namespace package in order to
support external platform specifications.

https://fedorahosted.org/freeipa/ticket/6474

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
2017-11-15 14:17:24 +01:00
Christian Heimes
8bbeedc93f
Backup ipa-custodia conf and keys
https://pagure.io/freeipa/issue/7247

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Simo Sorce <ssorce@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
2017-11-13 18:10:54 +01:00
Felipe Barreto
405da071d1 Warning the user when using a loopback IP as forwarder
Changing the --forwarder option to accept a loopback IP.
Previously, an error would be raised, now we just show a
warning message.

Fixes: https://pagure.io/freeipa/issue/5801
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-11-09 09:24:03 -02:00
Christian Heimes
b29db07c3b Use os.path.isfile() and isdir()
Replace custom file_exists() and dir_exists() functions with proper
functions from Python's stdlib.

The change also gets rid of pylint's invalid bad-python3-import error,
https://github.com/PyCQA/pylint/issues/1565

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2017-10-20 12:27:19 +02:00
Fraser Tweedale
c5afee964e cli: simplify parsing of arbitrary types
Add the 'constructor' type to IPAOption to allow parsing arbitrary
types.

When using this type, supply the 'constructor' attribute with the
constructor of the type.  The checker for the 'constructor' type
attempts to construct the data, returning if successful else raising
OptionValueError.

The 'knob' interface remains unchanged but now accepts arbitrary
constructors.

This feature subsumes the '_option_callback' mechanism, which has
been refactored away.

This feature also subsumes the "dn" type in IPAOption, but this
refactor is deferred.

Part of: https://pagure.io/freeipa/issue/6858

Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
2017-10-04 10:09:18 +02:00
Stanislav Laznicka
fab589d7f5 pylint: disable __hash__ for some classes
pylint requires all classes implementing __eq__ to also implement
__hash__. We disable hashing for the classes that miss the ability,
should they ever be required to use it, it can be implemented then.

https://pagure.io/freeipa/issue/6874

Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
2017-09-08 15:42:07 +02:00
Stanislav Laznicka
ae0bd124f5 install.util: disable no-value-for-parameter
InnerClassMeta is rather magical and seems to work as-is. There's a
reason not to always send all parameters to the methods since they
really don't have to be able to handle all the parameters all the
time.

https://pagure.io/freeipa/issue/6874

Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
2017-09-08 15:42:07 +02:00
Stanislav Laznicka
7a86ff5d9b Fix cert file creation during CA-less installation
When writing extracted certs and keys to the file, we opened the
same file at a different spot but the original file position
indicator would not be moved when the certificate is written there.
The result is that the certificate gets rewritten by the private
key.  This commit fixes it.

Fixes: https://pagure.io/freeipa/issue/7118
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
2017-08-25 09:48:02 +02:00
Fraser Tweedale
477b3dca80 Fix incorrect 'with' statement in CA-less installation
Part of: https://pagure.io/freeipa/issue/7118

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
2017-08-25 09:48:02 +02:00
Stanislav Laznicka
b1fbdbefdb
ipautil: port host_port_open() to python 3
socket.send() expects `Bytes` instance, not string

https://pagure.io/freeipa/issue/4985

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2017-08-18 11:48:51 +02:00
Stanislav Laznicka
dbeb41efd6 kerberos: fix sorting Principal objects
When service-find was issued under Python 3, the command fails
because it tried to sort a list of Principal objects which was not
possible.

https://pagure.io/freeipa/issue/4985

Reviewed-By: Felipe Volpone <fbarreto@redhat.com>
2017-08-17 09:56:01 +02:00
Petr Vobornik
038d192065
log progress of wait_for_open_ports
To know what to focus on when some check fail. E.g. to detect that
IPv6 address or its resolution for localhost is misconfigured.

https://pagure.io/freeipa/issue/7083

Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
2017-08-11 13:40:38 +02:00
Petr Vobornik
cc72db67e2
control logging of host_port_open from caller
host_port_open copied logging behavior of ipa-replica-conncheck utility
which doesn't make it much reusable.

Now log level can be controlled from caller so other callers might use
other logging level without host_port_open guessing what was the
intention.

https://pagure.io/freeipa/issue/7083

Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
2017-08-11 13:40:38 +02:00
Stanislav Laznicka
1521296297 x509,certdb: handle certificates as bytes
Certificates, both in PEM and DER format, should be handled as bytes
in Python 3.

https://pagure.io/freeipa/issue/4985

Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-07-27 10:28:58 +02:00
Stanislav Laznicka
5a44ca6383 Create a Certificate parameter
Up until now, Bytes parameter was used for certificate parameters
throughout the framework. However, the Bytes parameter does nothing
special for certificates, like validation, so this had to be done
for each of the parameters which were supposed to represent a
certificate.

This commit introduces a special Certificate parameter which takes
care of certificate validation so this does not have to be done
separately. It also makes sure that the certificates represented by
this parameter are always converted to DER format so that we can work
with them in a unified manner throughout the framework.

This commit also makes it possible to pass bytes directly during
instantiation of the Certificate parameter and they are still
represented correctly after their conversion in the _convert_scalar()
method.

https://pagure.io/freeipa/issue/4985

Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-07-27 10:28:58 +02:00
Stanislav Laznicka
b5732efda6 x509: Make certificates represented as objects
https://pagure.io/freeipa/issue/4985

Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-07-27 10:28:58 +02:00
Stanislav Laznicka
4375ef860f Split x509.load_certificate() into PEM/DER functions
Splitting the load_certificate() function into two separate helps
us word the requirements for the input explicitly. It also makes
our backend similar to the one of python-cryptography so eventually
we can swap python-cryptography for IPA x509 module.

https://pagure.io/freeipa/issue/4985

Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-07-27 10:28:58 +02:00
Tomas Krizek
ba5f7afedc logging: make sure logging level is set to proper value
During py.test initialization, the value 'debug' is passed instead
of logging.DEBUG.

Signed-off-by: Tomas Krizek <tkrizek@redhat.com>
Reviewed-By: David Kupka <dkupka@redhat.com>
2017-07-26 15:57:56 +02:00
Jan Cholasta
07229c8ff6 logging: do not use ipa_log_manager to create module-level loggers
Replace all `ipa_log_manager.log_mgr.get_logger` calls to create
module-level loggers with `logging.getLogger` calls and deprecate
`ipa_log_manager.log_mgr.get_logger`.

Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-07-14 15:55:59 +02:00
Jan Cholasta
7a482b7c72 logging: do not log into the root logger
Deprecate `ipa_log_manager.root_logger` and replace all calls to it with
module-level logger calls.

Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-07-14 15:55:59 +02:00
Jan Cholasta
ab9d1e75fc logging: do not reference loggers in arguments and attributes
Remove logger arguments in all functions and logger attributes in all
objects, with the exception of API object logger, which is now deprecated.
Replace affected logger calls with module-level logger calls.

Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-07-14 15:55:59 +02:00
Jan Cholasta
ffadcb0414 logging: remove object-specific loggers
Remove all object-specific loggers, with the exception of `Plugin.log`,
which is now deprecated. Replace affected logger calls with module-level
logger calls.

Deprecate object-specific loggers in `ipa_log_manager.get_logger`.

Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-07-14 15:55:59 +02:00
Jan Cholasta
9d19654cbd logging: use the actual root logger as the root logger
Use the actual root logger (`logging.getLogger()`) rather than the `ipa`
logger (or the `root` logger in case of ipa-ods-exporter) as the root
logger.

Always configure logging on the root logger.

Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-07-14 15:55:59 +02:00
Jan Cholasta
f62a0fdb90 logging: port to standard Python logging
Use the standard `logging` module to configure logging instead of the
in-house `ipapython.log_manager` module and remove `ipapython.log_manager`.

Disable the logging-not-lazy and logging-format-interpolation pylint
checks.

Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-07-14 15:55:59 +02:00
Jan Cholasta
464516489f logging: do not configure any handlers by default
Do not configure stderr handler by default and let the application do the
configuration.

Fix ipa-dnskeysync-replica and ipa-dnskeysyncd not to add stderr handler
twice.

Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-07-14 15:55:59 +02:00
Aleksei Slaikovskii
c39da523ad
ipapython/graph.py redundant variable fix
Changed deletion of edges in remove_vertex method because there's no
need to store redundant variable in memory.

Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-07-14 09:16:13 +02:00
Aleksei Slaikovskii
fe913b9c9f
ipapython/graph.py String formatting
Changed string formatting for Value Errors raise.

Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-07-14 09:16:13 +02:00
Aleksei Slaikovskii
52a435978b
ipapython/graph.py complexity optimization
Hi!

I've just read the code and I saw that graph bfs uses not optimal
for Python solution. So I've edited it with more optimal one.

https://pagure.io/freeipa/issue/7051

Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-07-14 09:16:13 +02:00
Martin Basti
f18ce01355
IPAOptionParser: fix dict comprehension
The statement can be simplified and be more resources friendly

Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
2017-07-14 09:15:00 +02:00
Jan Cholasta
4736fef6bb ldap2: remove URI argument from ldap2 constructor
LDAPClient should be used for ad-hoc connections, so the argument is not
necessary, and currently also unused.

Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
2017-07-04 12:06:33 +02:00
Martin Basti
0487f993fd py3: ipaldap: fix encoding of datetime objects
datetime objects were converted to string instead fo bytes.

https://pagure.io/freeipa/issue/4985

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2017-06-21 10:24:39 +02:00
Martin Basti
db01767acf py3: LDAPClient: remove __del__ method
in py3 we are receiving ugly tracebacks due __del__ method that
should be used very carefully or never. Due tracebacks, this doesn't
work and context manager should be used for reliable connection
termination.

Exception ignored in: <bound method LDAPClient.__del__ of ipaserver.plugins.ldap2.ldap2()>
Traceback (most recent call last):
  File "/usr/lib/python3.5/site-packages/ipapython/ipaldap.py", line 1057, in __del__
  File "/usr/lib/python3.5/site-packages/ipaserver/plugins/ldap2.py", line 123, in close
  File "/usr/lib/python3.5/site-packages/ipalib/backend.py", line 94, in isconnected
NameError: name 'hasattr' is not defined
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2017-06-21 10:24:39 +02:00
Martin Basti
7c163c90b8 LDAPEntry: rename _orig to _orig_raw
It was hard to detect what is supposed to be in self._orig variable.
Renaming to _orig_raw makes clear for future generations that it
contains bytes.

https://pagure.io/freeipa/issue/4985

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2017-06-21 10:24:39 +02:00
Martin Basti
52b43c7168 python-netifaces: update to reflect upstream changes
python-netifaces now provides IPv6 netmask in format mask/prefix. It
breaks freeipa as it is unexpected format for python-netaddr. We must
split netmask and provide only prefix for netaddr.

https://pagure.io/freeipa/issue/7021

Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
Reviewed-By: Petr Vobornik <pvoborni@redhat.com>
2017-06-20 13:52:10 +02:00
Martin Basti
f9cba7d161 Remove ip_netmask from option parser
ipa-dns-install uses ip_netmask=False --> parse_netmask=False, other installers uses default (parse_netmask=True).
Use this consistent accross all installers.

Also this option is unused (and shouldn't be used).

https://pagure.io/freeipa/issue/4317

Reviewed-By: David Kupka <dkupka@redhat.com>
2017-06-20 11:29:41 +02:00
Martin Basti
6024165101 CheckedIPAddress: remove match_local param
This parameter is unused in code. We are no longer testing if IP address
matches an interface in constructor.

https://pagure.io/freeipa/issue/4317

Reviewed-By: David Kupka <dkupka@redhat.com>
2017-06-20 11:29:41 +02:00
Martin Basti
0b69e44f16 refactor CheckedIPAddress class
Make methods without side effects (setting mask)

https://pagure.io/freeipa/issue/4317

Reviewed-By: David Kupka <dkupka@redhat.com>
2017-06-20 11:29:41 +02:00
Martin Basti
82ad586f6c Fix local IP address validation
Previously bf9886a84393d1d1546db7e49b102e08a16a83e7 match_local has
undesirable side effect that CheckedIPAddress object has set self._net
from local interface.

However with the recent changes, match_local is usually set to False,
thus this side effect stops happening and default mask per address class
is used. This causes validation error because mask on interface and mask
used for provided IP addresses differ (reporducible only with classless
masks).

FreeIPA should compare only IP addresses with local addresses without masks

https://pagure.io/freeipa/issue/4317

Reviewed-By: David Kupka <dkupka@redhat.com>
2017-06-20 11:29:41 +02:00
Stanislav Laznicka
bee3c1eccd More verbose error message on kdc cert validation
KDC cert validation was added but provides rather non-descriptive
error should there be something wrong with a certificate. Pass
the error message from the `openssl` tool in such cases.

https://pagure.io/freeipa/issue/6945

Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
2017-06-16 16:04:00 +02:00
Stanislav Laznicka
f827fe0f19 cert-validate: keep all messages in cert validation
Previous attempt to improve error messages during certificate
validation would only work in English locale so we're keeping
the whole NSS messages for all cases.

https://pagure.io/freeipa/issue/6945

Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
2017-06-16 16:04:00 +02:00
Stanislav Laznicka
d665224a85 session_storage: Correctly handle string/byte types
In session_storage.py, store_data() stores data as the bytes data
type but get_data() is returning a string. Have get_data() return
bytes as well.

https://pagure.io/freeipa/issue/4985

Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-06-09 16:59:53 +02:00
Martin Basti
6637980af6 Only warn when specified server IP addresses don't match intf
In containers local addresses differ from public addresses and we need
a way to provide only public address to installers.

https://pagure.io/freeipa/issue/2715
https://pagure.io/freeipa/issue/4317

Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
2017-06-06 18:10:33 +02:00
Fraser Tweedale
5f0e13ce9c ca-add: validate Subject DN name attributes
If the Subject DN is syntactically valid but contains unrecognised
name attributes, FreeIPA accepts it but Dogtag rejects it, returning
status 400 and causing the framework to raise RemoteRetrieveError.

Update the ca-add command to perform some additional validation on
the user-supplied Subject DN, making sure that we recognise all the
attributes.

Fixes: https://pagure.io/freeipa/issue/6987
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Felipe Volpone <felipevolpone@gmail.com>
2017-06-01 09:28:36 +02:00
Martin Basti
27f8f9f03d py3: ipaldap: encode Boolean as bytes
Python LDAP requires bytes

https://fedorahosted.org/freeipa/ticket/4985

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2017-06-01 09:24:24 +02:00
Fraser Tweedale
bc6d499514 Add Subject Key Identifier to CA cert validity check
CA certificates MUST have the Subject Key Identifier extension to
facilitiate certification path construction.  Not having this
extension on the IPA CA certificate will cause failures in Dogtag
during signing; it tries to copy the CA's Subject Key Identifier to
the new certificate's Authority Key Identifier extension, which
fails.

When installing an externally-signed CA, check that the Subject Key
Identifier extension is present in the CA certificate.

Fixes: https://pagure.io/freeipa/issue/6976
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2017-05-30 12:39:15 +02:00
Jan Cholasta
3b5dbf7cdb server install: fix KDC certificate validation in CA-less
Verify that the provided certificate has the extended key usage and subject
alternative name required for KDC.

https://pagure.io/freeipa/issue/6831
https://pagure.io/freeipa/issue/6869

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
2017-05-19 12:31:24 +02:00
David Kupka
b9fd123d61 ipapython.ipautil.run: Add option to set umask before executing command
https://pagure.io/freeipa/issue/6831

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
2017-05-19 12:31:24 +02:00
Jan Cholasta
01a7416d30 install: trust IPA CA for PKINIT
Trust IPA CA to issue PKINIT KDC and client authentication certificates in
the IPA certificate store.

https://pagure.io/freeipa/issue/6831

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
2017-05-19 12:31:24 +02:00
Jan Cholasta
52730c786f certdb: use custom object for trust flags
Replace trust flag strings with `TrustFlags` objects. The `TrustFlags`
class encapsulates `certstore` key policy and has an additional flag
indicating the presence of a private key.

https://pagure.io/freeipa/issue/6831

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
2017-05-19 12:31:24 +02:00