A new test case was not picking up ``ipa-run-tests`` script.
Fixes: https://pagure.io/freeipa/issue/8307
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Sphinx is extensible with plugins that can add new syntax, roles,
directives, domains, and more.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Run sphinx-builder with -W (fail on error), --keep-going, and -j auto.
Auto-job scaling speeds up sphinx-builder a LOT.
Add make lint target to doc/Makefile. The -E and -a option ensure that
all files are always re-read and rewritten.
Add option to run sphinx-builder from a virtual env that mimics RTD
builds closer than Fedora packages.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: François Cami <fcami@redhat.com>
Add doclint to fastcheck so that documentation syntax issues
are caught sooner (before they hit CI).
Signed-off-by: François Cami <fcami@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
'makeapi' and 'makeaci' has to be run in a particular environment that
forces IPA Python modules from the source tree used instead of what
might be installed system-wide.
Create 'make api' and 'make aci' targets to provide easy access to them.
Make sure we run Python interpreter with PYTHONPATH set to force use of
the source tree.
Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
The SELinux policy defines file contexts that are also used by clients,
e.g. /var/log/ipa/. Make freeipa-selinux a dependency of freeipa-common.
Related: https://pagure.io/freeipa/issue/6891
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Hook up the new policy to autoconf and automake.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
> Pylint is a tool that checks for errors in Python code, tries to enforce a
> coding standard and looks for code smells. It can also look for certain type
> errors, it can recommend suggestions about how particular blocks can be
> refactored and can offer you details about the code's complexity.
Fixes: https://pagure.io/freeipa/issue/8202
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
For now, a list of YAML files' paths is hardcoded (even after
globbing) into Makefile.am. Moreover, Azure templates are not
checked at all until Azure triggered.
With this change, the list of YAMLs is populated automatically
on yamllinting.
Jinja templates are not parseable by a regular yaml module, to
skip such the YAML_TEMPLATE_FILES is utilized.
Fixes: https://pagure.io/freeipa/issue/8202
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Removing Travis CI in favour of Azure Pipelines.
All tests previously tested by Travis are also configured in Azure.
Repos [1] and [2] were used to build Docker images for Travis, thus
they are no longer required for branches master and ipa-4-8.
Branches ipa-4-7 and ipa-4-6 don't have Azure pipelines configured,
so Travis will continue to be used by them.
1 - https://github.com/freeipa/ipa-docker-test-runner
2 - https://github.com/freeipa/freeipa-builder
Related: https://pagure.io/freeipa/issue/7323
Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Francisco Trivino <ftrivino@redhat.com>
This is a sort of rollback to the pre #93c158b05 state with
several improvements.
For now, the nodeids calculation by ipa-run-tests is not stable,
since it depends on current working directory. Nodeids (tests
addresses) are utilized by the other plugins, for example.
Unfortunately, the `pytest_load_initial_conftests` hook doesn't
correctly work with pytest internal paths, since it is called
after the calculation of rootdir was performed, for example.
Eventually, it's simpler to follow the default convention for
Python test discovery.
There is at least one drawback of new "old" implementation.
The ignore rules don't support globs, because pytest 4.3.0+
has the same facility via `--ignore-glob`:
> Add the `--ignore-glob` parameter to exclude test-modules with
> Unix shell-style wildcards. Add the collect_ignore_glob for
> conftest.py to exclude test-modules with Unix shell-style
> wildcards.
Upon switching to pytest4 it will be possible to utilize this.
Anyway, tests for checking current basic facilities of
ipa-run-tests were added.
Fixes: https://pagure.io/freeipa/issue/8007
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Previously fastlint would only process .py files.
Detect which .in files are in fact Python files and add
them to the list of files to process during the fastlint
Makefile target.
Original change suggested by Alexander Bokovoy.
Fixes: https://pagure.io/freeipa/issue/7984
Signed-off-by: François Cami <fcami@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The make target `pylint` hasn't a full list of its dependencies.
This leads to problems like:
- different build results
- PR tests just do not run pylint over some Python scripts.
The new build target (`python_scripts`) was implemented.
It's intended for building all Python scripts (files, containing
@PYTHONSHEBANG@ as a shebang placeholder).
The make `pylint` should require it.
Fixes: https://pagure.io/freeipa/issue/7921
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Build all scripts in install/tools/ to check them with pylint, so that
``make pylint`` always checks all scripts. The script files are
generated by make.
Please note that fastlint does not check script files.
See: https://pagure.io/freeipa/issue/7772
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Thomas Woerner <twoerner@redhat.com>
Add "make cov-scan" to automate Coverity scan builds. cov-build requires
extra quirks to work with recent versions of GCC on Fedora.
The make target requires a token and Coverity's build chain. Both are
available for privileged project owners on
https://scan.coverity.com/projects/freeipa-freeipa .
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Let's catch broken YAML files (Travis, PR-CI) and spec file early.
- Use rpmlint to detect syntax errors in spec file early
- Attempt to parse all YAML files with PyYAML
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Remove Python 2 related code and configuration from spec file, autoconf
and CI infrastructure. From now on, FreeIPA 4.8 requires at least Python
3.6. Python 2 packages like python2-ipaserver or python2-ipaclient are
no longer available. PR-CI, lint, and tox aren't testing Python 2
compatibility either.
See: https://fedoraproject.org/wiki/Changes/FreeIPA_Python_2_Removal
Fixes: https://pagure.io/freeipa/issue/7568
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
Python scripts are now generated from templates. The scripts are marked
as nodist (no distribution) but install targets. The templates for the
scripts are extra distribution data, no installation (noinst).
Fixes: https://pagure.io/freeipa/issue/7680
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Move logic for installing just the Python packages out of the spec file
and into our root Makefile. It removes code duplication to simplify a
spec file that supports building without Python 2.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
when any file from FreeIPA tree has been deleted there was
a failure like:
pylint
------
************* Module ipaserver/install/ntpinstance.py
ipaserver/install/ntpinstance.py:1: [F0001(fatal), ] No module named ipaserver/install/ntpinstance.py)
Adding --diff-filter to fastlint will not list deleted files
in git diff --names-only output to not include not existing
files to checklist.
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
`git diff` shows also context lines by default. When passed to pycodestyle
it can produce errors unrelated to changed lines. It prevents running of
subsequent checks.
Limiting context to 0 lines by `git diff -U0` enables to test only the
modified lines and allows to run subsequent checks.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Python 3's regular expression default to full range of unicode
characters. Restrict \w matches to ASCII and drop \b suffix check to fix
a problem with validation the Chinese translation zh_CN.
Co-Authored-By: Stanislav Laznicka <slaznick@redhat.com>
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Make it possible to run API, ACI, and potests under Python 3.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
- Check pycodestyle before pylint. pycodestyle takes seconds while
pylint can easily take half a minute or more.
- Fix exit, needs two $
- Add some newlines to make output more readable
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
Fast linting only needs modified files with pylint and diff with
pycodestyle. It's good enough to detect most code errors very fast. It
typically takes less than 10 seconds. A complete full pylint run uses
all CPU cores for several minutes. PEP 8 violations are typically
reported after 30 minutes to several hours on Travis CI.
Fast lintings uses git diff and git merge-base to find all modified
files in a branch or working tree. There is no easy way to find the
branch source. On Travis the information is provided by Travis. For
local development it's a new variable IPA_GIT_BRANCH in VERSION.m4.
Fast testing execute all unit tests that do not depend on ipalib.api.
In total it takes about 30-40 seconds (!) to execute linting, PEP 8 checks
and unittests for both Python 2 and 3.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Instead of symlinks and build-time configuration the ipaplatform module
is now able to auto-detect platforms on import time. The meta importer
uses the platform 'ID' from /etc/os-releases. It falls back to 'ID_LIKE'
on platforms like CentOS, which has ID=centos and ID_LIKE="rhel fedora".
The meta importer is able to handle namespace packages and the
ipaplatform package has been turned into a namespace package in order to
support external platform specifications.
https://fedorahosted.org/freeipa/ticket/6474
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
ignore_import_errors was added in 9b534238 to build FreeIPA ACI/API with
some dependencies missing. It turns out that the import hook doesn't
play nice with other meta importers or Cython-generated code like lxml:
./makeaci: ipaserver/plugins/dogtag.py:246: ignoring ImportError: No module named lxml.re
Traceback (most recent call last):
File "./makeaci", line 134, in <module>
main(options)
File "./makeaci", line 107, in main
api.finalize()
File "ipalib/plugable.py", line 733, in finalize
self.__do_if_not_done('load_plugins')
File "ipalib/plugable.py", line 425, in __do_if_not_done
getattr(self, name)()
File "ipalib/plugable.py", line 614, in load_plugins
self.add_package(package)
File "ipalib/plugable.py", line 641, in add_package
module = importlib.import_module(name)
File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "ipaserver/plugins/dogtag.py", line 246, in <module>
from lxml import etree
File "src/lxml/etree.pyx", line 93, in init lxml.etree
File "src/lxml/_elementpath.py", line 58, in init lxml._elementpath
AttributeError: 'FailedImport' object has no attribute 'compile'
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
This patchset returns back js directory with its symlinks into
the repository so it does not need to be created during build.
https://pagure.io/freeipa/issue/6447
Reviewed-By: Petr Vobornik <pvoborni@redhat.com>
pip install foo foo[more] does not install the extra dependencies 'more'
of foo. It's a known bug in pip, see
https://github.com/pypa/pip/issues/4391#issuecomment-290712930 and
https://github.com/pypa/pip/issues/988
The same bug applies to pip wheel. As a workaround pip wheel first
builds extra dependencies, then wheel dependencies. This ensures that
ipaclient[otptoken_yubikey] dependencies get built properly.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Add tox infrastructure to test client wheel packages workflow:
* build client packages
* install client packages
* ipa-run-tests --ipaclient-unittests under Python 2 and 3
* pylint of client packages under Python 2 and 3
* placeholder packages work as expected
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Parallel make or flags like IPA_OMIT_INSTALL and IPA_SERVER_WHEELS could
lead to bad packages for PyPI. Only build the packages we want with
correct flags.
Placeholder packages from 'make pypi_package' conflict with
'make wheel_bundle' packages. Use a separate destination directory for
PyPI packages.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
To create a wheel bundle with ipaserver and its dependencies:
make wheel_bundle IPA_SERVER_WHEELS=1
To include additional dependencies:
make wheel_bundle IPA_EXTRA_WHEELS=ipatests[webui]
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Ticket 6604 makes pylint and jsl optional dependencies. The change
is controversal, because some developers prefer that pylint and jsl
should be required unless explicitly disabled.
`make devcheck` is my answer to address the concerns. It's a superior
solution to `make lint` as pre-commit check. It combines several
additional checks under a single, easy rememberable and convenient make
target:
* build all
* acilint, apiclient, jslint, polint
* make check
* pylint under Python 2 and 3
* subset of unit test suite
https://fedorahosted.org/freeipa/ticket/6604
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
The presence of IPA packages on PyPI revealed an interesting issue with
make wheel_bundle. pip gives final releases a higher precedence than our
development packages. make wheel_bundle downloads ipa 4.5.0 from PyPI
instead of using our own wheels.
Use a constraint file to enforce correct versions.
https://pagure.io/freeipa/issue/6468
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
./configure no longer fails when pylint or jsl are not available. The
make targets for pylint and jsl are no longer defined without the tools.
Rational:
pylint and jsl are not required to build FreeIPA. Both are useful
developer tools. It's more user friendly to make both components
optionally with default config arguments. There is no reason to
fail building on a build system without development tools.
It's still possible to enforce dependency checks with --with-jslint and
--enable-pylint.
https://fedorahosted.org/freeipa/ticket/6604
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
Add new pylint AST checker plugin which implements a check for imports
forbidden in IPA. Which imports are forbidden is configurable in pylintrc.
Provide default forbidden import configuration and disable the check for
existing forbidden imports in our code base.
Reviewed-By: Martin Basti <mbasti@redhat.com>
So that it will be nicely formatted on FreeIPA Pagure landing page.
https://pagure.io/freeipa
Some links were updated as other projects also moved to Pagure.io.
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
pylint gets confused by duplicated package names, e.g. ipaplatform and
pypi/ipaplatform.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
I also renamed the base directory to pypi and added a new build target
pypi_packages.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
The ipa and freeipa packages are placeholders to prevent PyPI squashing
attacks and reserve the names for future use. `pip install ipa` installs
ipaclient.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
--without-ipatests skips building and installation of the ipatests
package. By default the ipatests package is always build and installed
by make install.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Simo Sorce <ssorce@redhat.com>
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
The new development server depends on werkzeug instead of paste. The
werkzeug WSGI server comes with some additional features, most
noticeable multi-processing server. The IPA framework is not compatible
with threaded servers. Werkzeug can serve static files easily and has a
fast auto-reloader.
The new lite-server implementation depends on PR 314 (privilege
separation). For Python 3 support, it additionally depends on PR 393.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Let's replace some ugly hacks with proper pytest conftest.py hooks.
Test initialization of ipalib.api is now handled in
pytest_cmdline_main(). Pytest plugins, markers and ignores are also
moved into conftest.py. Additional guards make it possible to run tests
without ipaserver installed.
I added confcutdir to ensure that pytest does not leave our project
space. Pytest used pytest.ini or setup.py before but pytest.ini is gone.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Milan Kubik <mkubik@redhat.com>
configure option --with/without-pylint was replaced by
--enable/disable-pylint. Pylint is always called as $(PYTHON) -m python.
If you need to override Pylint version, use command "make pylint PYTHON=xxx".
https://fedorahosted.org/freeipa/ticket/157
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
* make wheel_bundle no longer bundles ipaplatform
* ipaclient and ipalib use a consistent extra tag for the install
subpackage. `pip install ipalib[ipalib.install]` looks a bit silly.
https://fedorahosted.org/freeipa/ticket/6474
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
AC_CONFIG_FILES in configure.ac works well only with Makefiles.
Other files have to be handled by Makefile.am so depedencies
are tracked properly.
There is a problem that Python sub-directories depend on ipasetup.py
which is one level above the sub-directory. This means that depedencies
are the other way around that expected. This is being worked around
using hack from
http://lists.gnu.org/archive/html/automake/2009-03/msg00011.htmlhttps://fedorahosted.org/freeipa/ticket/6498
Reviewed-By: Martin Basti <mbasti@redhat.com>
