* Make jinja2 an optional dependency and csrgen an optional plugin
* Make otptoken_yubikey an optional plugin
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
The server-side plugin for IPA Vault relied on the fact that the default
oid for encryption algorithm is 3DES in CBC mode (DES-EDE3-CBC). Dogtag
10.4 has changed the default from 3DES to AES. Pass the correct
algorithm OID to KeyClient.archive_encrypted_data().
Closes: https://pagure.io/freeipa/issue/6899
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Add tox infrastructure to test client wheel packages workflow:
* build client packages
* install client packages
* ipa-run-tests --ipaclient-unittests under Python 2 and 3
* pylint of client packages under Python 2 and 3
* placeholder packages work as expected
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Parallel make or flags like IPA_OMIT_INSTALL and IPA_SERVER_WHEELS could
lead to bad packages for PyPI. Only build the packages we want with
correct flags.
Placeholder packages from 'make pypi_package' conflict with
'make wheel_bundle' packages. Use a separate destination directory for
PyPI packages.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
mod_wsgi has no way to import a WSGI module by dotted module name. A new
kdcproxy.wsgi script is used to import kdcproxy from whatever Python
version mod_wsgi is compiled against. This will simplify moving FreeIPA
to Python 3 and solves an import problem on Debian.
Resolves: https://pagure.io/freeipa/issue/6834
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Fix problem with hard-coded shebang in ipa command line tool by using
a proper setuptools entry point for the console script. ipaclient is now
an executable Python package, too.
```
$ python -m ipaclient ping
```
is equivalent to
```
$ ipa ping
```
Related: https://pagure.io/freeipa/issue/6653
Closes: https://pagure.io/freeipa/issue/6850
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
To create a wheel bundle with ipaserver and its dependencies:
make wheel_bundle IPA_SERVER_WHEELS=1
To include additional dependencies:
make wheel_bundle IPA_EXTRA_WHEELS=ipatests[webui]
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
ipaserver did not have extra_requires to state additional dependencies.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
The pyhbac module is part of SSSD. It's not available as stand-alone
PyPI package. It would take a lot of effort to package it because the
code is deeply tight into SSSD.
Let's follow the example of other SSSD Python packages and make the
import of pyhbac conditionally. It's only necessary for caacl and
hbactest plugins.
I renamed convert_to_ipa_rule() to _convert_to_ipa_rule() because it
does not check for presence of pyhbac package itself. The check is
performed earlier in execute(). The prefix indicates that it is an
internal function and developers have to think twice before using it
in another place.
This makes it much easier to install ipaserver with instrumented build
of Python with a different ABI or in isolated virtual envs to profile
and debug the server.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
The test class depends on a working Kerberos configuration and session.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Ticket 6604 makes pylint and jsl optional dependencies. The change
is controversal, because some developers prefer that pylint and jsl
should be required unless explicitly disabled.
`make devcheck` is my answer to address the concerns. It's a superior
solution to `make lint` as pre-commit check. It combines several
additional checks under a single, easy rememberable and convenient make
target:
* build all
* acilint, apiclient, jslint, polint
* make check
* pylint under Python 2 and 3
* subset of unit test suite
https://fedorahosted.org/freeipa/ticket/6604
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Fixes TypeError: bytes or integer address expected instead of str instance
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Doctests of ipatests.util fail under Python 3.
The old test scenario does no longer work on Python 3 since u'how are you'
and 'how are you' have identical type, but u'how are you' != b'how are you'.
It works with int / float on all Python versions.
Python 2 has <type 'int'> while Python 3 uses <class 'int'>.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
In-memory cache causes problem in forking servers. A file based cache is
good enough. It's easier to understand and avoids performance regression
and synchronization issues when cert becomes out-of-date.
https://pagure.io/freeipa/issue/6787
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
pytest 3.x does no longer support plain pytest.skip() on module level.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
The presence of IPA packages on PyPI revealed an interesting issue with
make wheel_bundle. pip gives final releases a higher precedence than our
development packages. make wheel_bundle downloads ipa 4.5.0 from PyPI
instead of using our own wheels.
Use a constraint file to enforce correct versions.
https://pagure.io/freeipa/issue/6468
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
The changes made to ipa-run-tests script in fd1b4f6 broke the
ipa-test-config and ipa-test-task scripts which are not executed
via pytest.
To fix the issue, all helper code and dependencies of the integration
plugin are moved out of ipatests.test_integration and into the
integration plugin. As first step ipatests.pytest_plugins.integration
is turned into a package.
https://pagure.io/freeipa/issue/6798
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Milan Kubik <mkubik@redhat.com>
Apache has a default keep alive timeout of 5 seconds. That's too low for
interactive commands, e.g. password prompts. 30 seconds sounds like a
good compromise.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
Do not forcefully close the connection after every request. This enables
HTTP connection keep-alive, also known as persistent TCP and TLS/SSL
connection. Keep-alive speed up consecutive HTTP requests by 15% (for
local, low-latency network connections to a fast server) to multiple
times (high latency connections or remote peers).
https://pagure.io/freeipa/issue/6641
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
ctypes can only handle bytes, not text. Encode and decode all incoming
and outgoing text from UTF-8 to bytes.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Simo Sorce <ssorce@redhat.com>
************* Module ipaserver.install.ipa_kra_install
ipaserver/install/ipa_kra_install.py:25: [W0402(deprecated-module), ] Uses of a deprecated module 'optparse')
************* Module ipapython.install.core
ipapython/install/core.py:163: [E1101(no-member), _knob] Module 'types' has no 'TypeType' member)
************* Module ipatests.test_ipapython.test_dn
ipatests/test_ipapython/test_dn.py:1205: [W1505(deprecated-method), TestDN.test_x500_text] Using deprecated method assertEquals())
************* Module ipa-ca-install
install/tools/ipa-ca-install:228: [E1101(no-member), install_master] Instance of 'ValueError' has no 'message' member)
install/tools/ipa-ca-install:232: [E1101(no-member), install_master] Instance of 'ValueError' has no 'message' member)
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Simo Sorce <ssorce@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Debian packages should be installed under dist-packages, not
site-packages. Debian has patched distutils and setuptools to add a new
flag '--install-layout'. For --with-ipaplatform=debian,
PYTHON_INSTALL_EXTRA_OPTIONS is set to '--install-layout=deb'.
https://pagure.io/freeipa/issue/6764
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Timo Aaltonen <tjaalton@debian.org>
Reviewed-By: Martin Basti <mbasti@redhat.com>
./configure no longer fails when pylint or jsl are not available. The
make targets for pylint and jsl are no longer defined without the tools.
Rational:
pylint and jsl are not required to build FreeIPA. Both are useful
developer tools. It's more user friendly to make both components
optionally with default config arguments. There is no reason to
fail building on a build system without development tools.
It's still possible to enforce dependency checks with --with-jslint and
--enable-pylint.
https://fedorahosted.org/freeipa/ticket/6604
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
First try custom location, then csrgen subdir in confdir and finally
fall back to package data.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Ben Lipton <blipton@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
csrgen broke packaging of ipaclient for PyPI. All csrgen related
resources are now package data of ipaclient package. Package data is
accessed with Jinja's PackageLoader() or through pkg_resources.
https://pagure.io/freeipa/issue/6714
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Ben Lipton <blipton@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
* use with statement to open/close files
* prefer fchmod/fchown when a file descriptor is available
* set permission before data is written to file
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
automake was complaining about duplicated definitions of pkginstall. It
was defined to true in Makefile.python.am only to be overriden in some
Makefile.am.
Now we assume that pkginstall is implicit true and only skip
installation when pkginstall is explicitly set to false.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
pylint gets confused by duplicated package names, e.g. ipaplatform and
pypi/ipaplatform.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Some calls to setup.py specified a build base, some did not. This can
lead to issues, e.g. build, clean and install are using different build
directories.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
I also renamed the base directory to pypi and added a new build target
pypi_packages.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
The ipa and freeipa packages are placeholders to prevent PyPI squashing
attacks and reserve the names for future use. `pip install ipa` installs
ipaclient.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
test_ldap just imported nss.nss to call nss_init_nodb(). It should be
safe to remove the call. Let's see what CI has to say.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
It's not necesary to restore the context of newly created files. SELinux
ensures that new files have the correct permission. An explicit
restore_context() is only required when either policies have changed or
the context was changed manually.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
* add missing default_backend
* unpad encrypted data
* use cryptography's hashes and HMAC construct
* remove hard dependency on python-nss from setup.py
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
The schema cache used a BytesIO buffer to read/write schema cache before
it got flushed to disk. Since the schema cache is now loaded in one go,
the temporary buffer is no longer needed.
File locking has been replaced with a temporary file and atomic rename.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: David Kupka <dkupka@redhat.com>
It's inefficient to open a zip file over and over again. By loading all
members of the schema cache file at once, the ipa CLI script starts
about 25 to 30% faster for simple cases like help and ping.
Before:
$ time for i in {1..20}; do ./ipa ping >/dev/null; done
real 0m13.608s
user 0m10.316s
sys 0m1.121s
After:
$ time for i in {1..20}; do ./ipa ping >/dev/null; done
real 0m9.330s
user 0m7.635s
sys 0m1.146s
https://fedorahosted.org/freeipa/ticket/6690
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: David Kupka <dkupka@redhat.com>
Fix "implicit declaration of function ‘strlen’" in ipa_pwd_ntlm.c,
credits to Lukas.
Add -Werror=implicit-function-declaration to CFLAGS to point developers
to missing includes. It causes compilation to fail when a developer
forgets to add a required include. The problem is no longer hidden in a
massive wall of text from make.
Silence a harmless error from 389-DS slapi.h until the bug is fixed in
downstream, https://pagure.io/389-ds-base/issue/48979
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
--without-ipatests skips building and installation of the ipatests
package. By default the ipatests package is always build and installed
by make install.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Simo Sorce <ssorce@redhat.com>
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
The new development server depends on werkzeug instead of paste. The
werkzeug WSGI server comes with some additional features, most
noticeable multi-processing server. The IPA framework is not compatible
with threaded servers. Werkzeug can serve static files easily and has a
fast auto-reloader.
The new lite-server implementation depends on PR 314 (privilege
separation). For Python 3 support, it additionally depends on PR 393.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Some tests assume that JSON deserializier returns tuples instead of
lists. I don't think it is necessary but let's pass the tests for now.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Improve performance of FreeIPA's JSON serializer and deserializer.
* Don't indent and sort keys. Both options trigger a slow path in
Python's json package. Without indention and sorting, encoding
mostly happens in optimized C code.
* Replace O(n) type checks with O(1) type lookup and eliminate
the use of isinstance().
* Check each client capability only once for every conversion.
* Use decoder's obj_hook feature to traverse the object tree once and
to eliminate calls to isinstance().
Closes: https://fedorahosted.org/freeipa/ticket/6655
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Avoid comparison of bytes with int in _is_null() check. b'' == 0
triggers a BytesWarning.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Print api.env, uname, euid/egid, cwd and Python version when tests are
run with -v (e.g. ipa-run-tests -v).
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
In tree runs of make check leave some artifacts around. The patch adds
them to make clean and .gitignore.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
install-exec and bdist_wheel also depend on version.py. Let's ensure
that version.py is correctly generated when installing or building
packages.
Yes, make is clever and correctly merges dependencies with rules from
included make files.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Milan Kubik <mkubik@redhat.com>
1e06a5195b removed pytest.ini. Without the
ini file, pytest 3.x has suboptimal settings and no longer picks up all
test functions and test files.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Milan Kubik <mkubik@redhat.com>
Let's replace some ugly hacks with proper pytest conftest.py hooks.
Test initialization of ipalib.api is now handled in
pytest_cmdline_main(). Pytest plugins, markers and ignores are also
moved into conftest.py. Additional guards make it possible to run tests
without ipaserver installed.
I added confcutdir to ensure that pytest does not leave our project
space. Pytest used pytest.ini or setup.py before but pytest.ini is gone.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Milan Kubik <mkubik@redhat.com>
Let's relax the check for .git from directory to exists in order to
support freeipa in a git submodule. Submodules have a .git file with
content like
gitdir: ../.git/modules/freeipa
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Petr Spacek <pspacek@redhat.com>
The environment variable IPA_CONFDIR overrides the default confdir path.
The value of the environment variable must be an absolute path to an existing
directory. The new variable makes it much simpler to use the 'ipa'
command and ipalib with a local configuration directory.
Some scripts (e.g. servers, installers, and upgrades) set the confdir
explicitly and do not support the env var.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Some API contexts are used to modify global state (e.g. files in /etc
and /var). These contexts do not support confdir overrides. Initialize
the API with an explicit confdir argument to paths.ETC_IPA.
The special contexts are:
* backup
* cli_installer
* installer
* ipctl
* renew
* restore
* server
* updates
The patch also corrects the context of the ipa-httpd-kdcproxy script to
'server'.
https://fedorahosted.org/freeipa/ticket/6389
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
python-cryptography versions < 1.3 no longer compile with recent OpenSSL
1.0.2 versions. In order to build wheels, a more recent version of
cryptography is required. 1.3.1 is the oldest well tested version (RHEL
7.3) that is known to work with FreeIPA.
Bump up in freeipa.spec is not required for technical reasons. The
problem only affects PyPI packages. It's policy to keep
requirements in sync.
https://fedorahosted.org/freeipa/ticket/6468
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
* make wheel_bundle no longer bundles ipaplatform
* ipaclient and ipalib use a consistent extra tag for the install
subpackage. `pip install ipalib[ipalib.install]` looks a bit silly.
https://fedorahosted.org/freeipa/ticket/6474
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
The PyPI package for python-gssapi 1.1.x has a packaging bug. It depends on
enum34 for Python 3 although it is only required for 2.7. 1.2.0 is the
oldest version that has been tested at length by QE. It's know to work.
Bump up in freeipa.spec is not required for technical reasons. The
packaging bug only affects PyPI packages. It's policy to keep
requirements in sync.
https://fedorahosted.org/freeipa/ticket/6468
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
Christian Heimes