Move logic for installing just the Python packages out of the spec file
and into our root Makefile. It removes code duplication to simplify a
spec file that supports building without Python 2.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
when any file from FreeIPA tree has been deleted there was
a failure like:
pylint
------
************* Module ipaserver/install/ntpinstance.py
ipaserver/install/ntpinstance.py:1: [F0001(fatal), ] No module named ipaserver/install/ntpinstance.py)
Adding --diff-filter to fastlint will not list deleted files
in git diff --names-only output to not include not existing
files to checklist.
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
`git diff` shows also context lines by default. When passed to pycodestyle
it can produce errors unrelated to changed lines. It prevents running of
subsequent checks.
Limiting context to 0 lines by `git diff -U0` enables to test only the
modified lines and allows to run subsequent checks.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Python 3's regular expression default to full range of unicode
characters. Restrict \w matches to ASCII and drop \b suffix check to fix
a problem with validation the Chinese translation zh_CN.
Co-Authored-By: Stanislav Laznicka <slaznick@redhat.com>
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Make it possible to run API, ACI, and potests under Python 3.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
- Check pycodestyle before pylint. pycodestyle takes seconds while
pylint can easily take half a minute or more.
- Fix exit, needs two $
- Add some newlines to make output more readable
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
Fast linting only needs modified files with pylint and diff with
pycodestyle. It's good enough to detect most code errors very fast. It
typically takes less than 10 seconds. A complete full pylint run uses
all CPU cores for several minutes. PEP 8 violations are typically
reported after 30 minutes to several hours on Travis CI.
Fast lintings uses git diff and git merge-base to find all modified
files in a branch or working tree. There is no easy way to find the
branch source. On Travis the information is provided by Travis. For
local development it's a new variable IPA_GIT_BRANCH in VERSION.m4.
Fast testing execute all unit tests that do not depend on ipalib.api.
In total it takes about 30-40 seconds (!) to execute linting, PEP 8 checks
and unittests for both Python 2 and 3.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Instead of symlinks and build-time configuration the ipaplatform module
is now able to auto-detect platforms on import time. The meta importer
uses the platform 'ID' from /etc/os-releases. It falls back to 'ID_LIKE'
on platforms like CentOS, which has ID=centos and ID_LIKE="rhel fedora".
The meta importer is able to handle namespace packages and the
ipaplatform package has been turned into a namespace package in order to
support external platform specifications.
https://fedorahosted.org/freeipa/ticket/6474
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
ignore_import_errors was added in 9b534238 to build FreeIPA ACI/API with
some dependencies missing. It turns out that the import hook doesn't
play nice with other meta importers or Cython-generated code like lxml:
./makeaci: ipaserver/plugins/dogtag.py:246: ignoring ImportError: No module named lxml.re
Traceback (most recent call last):
File "./makeaci", line 134, in <module>
main(options)
File "./makeaci", line 107, in main
api.finalize()
File "ipalib/plugable.py", line 733, in finalize
self.__do_if_not_done('load_plugins')
File "ipalib/plugable.py", line 425, in __do_if_not_done
getattr(self, name)()
File "ipalib/plugable.py", line 614, in load_plugins
self.add_package(package)
File "ipalib/plugable.py", line 641, in add_package
module = importlib.import_module(name)
File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "ipaserver/plugins/dogtag.py", line 246, in <module>
from lxml import etree
File "src/lxml/etree.pyx", line 93, in init lxml.etree
File "src/lxml/_elementpath.py", line 58, in init lxml._elementpath
AttributeError: 'FailedImport' object has no attribute 'compile'
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
This patchset returns back js directory with its symlinks into
the repository so it does not need to be created during build.
https://pagure.io/freeipa/issue/6447
Reviewed-By: Petr Vobornik <pvoborni@redhat.com>
pip install foo foo[more] does not install the extra dependencies 'more'
of foo. It's a known bug in pip, see
https://github.com/pypa/pip/issues/4391#issuecomment-290712930 and
https://github.com/pypa/pip/issues/988
The same bug applies to pip wheel. As a workaround pip wheel first
builds extra dependencies, then wheel dependencies. This ensures that
ipaclient[otptoken_yubikey] dependencies get built properly.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Add tox infrastructure to test client wheel packages workflow:
* build client packages
* install client packages
* ipa-run-tests --ipaclient-unittests under Python 2 and 3
* pylint of client packages under Python 2 and 3
* placeholder packages work as expected
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Parallel make or flags like IPA_OMIT_INSTALL and IPA_SERVER_WHEELS could
lead to bad packages for PyPI. Only build the packages we want with
correct flags.
Placeholder packages from 'make pypi_package' conflict with
'make wheel_bundle' packages. Use a separate destination directory for
PyPI packages.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
To create a wheel bundle with ipaserver and its dependencies:
make wheel_bundle IPA_SERVER_WHEELS=1
To include additional dependencies:
make wheel_bundle IPA_EXTRA_WHEELS=ipatests[webui]
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Ticket 6604 makes pylint and jsl optional dependencies. The change
is controversal, because some developers prefer that pylint and jsl
should be required unless explicitly disabled.
`make devcheck` is my answer to address the concerns. It's a superior
solution to `make lint` as pre-commit check. It combines several
additional checks under a single, easy rememberable and convenient make
target:
* build all
* acilint, apiclient, jslint, polint
* make check
* pylint under Python 2 and 3
* subset of unit test suite
https://fedorahosted.org/freeipa/ticket/6604
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
The presence of IPA packages on PyPI revealed an interesting issue with
make wheel_bundle. pip gives final releases a higher precedence than our
development packages. make wheel_bundle downloads ipa 4.5.0 from PyPI
instead of using our own wheels.
Use a constraint file to enforce correct versions.
https://pagure.io/freeipa/issue/6468
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
./configure no longer fails when pylint or jsl are not available. The
make targets for pylint and jsl are no longer defined without the tools.
Rational:
pylint and jsl are not required to build FreeIPA. Both are useful
developer tools. It's more user friendly to make both components
optionally with default config arguments. There is no reason to
fail building on a build system without development tools.
It's still possible to enforce dependency checks with --with-jslint and
--enable-pylint.
https://fedorahosted.org/freeipa/ticket/6604
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
Add new pylint AST checker plugin which implements a check for imports
forbidden in IPA. Which imports are forbidden is configurable in pylintrc.
Provide default forbidden import configuration and disable the check for
existing forbidden imports in our code base.
Reviewed-By: Martin Basti <mbasti@redhat.com>
So that it will be nicely formatted on FreeIPA Pagure landing page.
https://pagure.io/freeipa
Some links were updated as other projects also moved to Pagure.io.
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
pylint gets confused by duplicated package names, e.g. ipaplatform and
pypi/ipaplatform.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
I also renamed the base directory to pypi and added a new build target
pypi_packages.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
The ipa and freeipa packages are placeholders to prevent PyPI squashing
attacks and reserve the names for future use. `pip install ipa` installs
ipaclient.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
--without-ipatests skips building and installation of the ipatests
package. By default the ipatests package is always build and installed
by make install.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Simo Sorce <ssorce@redhat.com>
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
The new development server depends on werkzeug instead of paste. The
werkzeug WSGI server comes with some additional features, most
noticeable multi-processing server. The IPA framework is not compatible
with threaded servers. Werkzeug can serve static files easily and has a
fast auto-reloader.
The new lite-server implementation depends on PR 314 (privilege
separation). For Python 3 support, it additionally depends on PR 393.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Let's replace some ugly hacks with proper pytest conftest.py hooks.
Test initialization of ipalib.api is now handled in
pytest_cmdline_main(). Pytest plugins, markers and ignores are also
moved into conftest.py. Additional guards make it possible to run tests
without ipaserver installed.
I added confcutdir to ensure that pytest does not leave our project
space. Pytest used pytest.ini or setup.py before but pytest.ini is gone.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Milan Kubik <mkubik@redhat.com>
configure option --with/without-pylint was replaced by
--enable/disable-pylint. Pylint is always called as $(PYTHON) -m python.
If you need to override Pylint version, use command "make pylint PYTHON=xxx".
https://fedorahosted.org/freeipa/ticket/157
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
* make wheel_bundle no longer bundles ipaplatform
* ipaclient and ipalib use a consistent extra tag for the install
subpackage. `pip install ipalib[ipalib.install]` looks a bit silly.
https://fedorahosted.org/freeipa/ticket/6474
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
AC_CONFIG_FILES in configure.ac works well only with Makefiles.
Other files have to be handled by Makefile.am so depedencies
are tracked properly.
There is a problem that Python sub-directories depend on ipasetup.py
which is one level above the sub-directory. This means that depedencies
are the other way around that expected. This is being worked around
using hack from
http://lists.gnu.org/archive/html/automake/2009-03/msg00011.htmlhttps://fedorahosted.org/freeipa/ticket/6498
Reviewed-By: Martin Basti <mbasti@redhat.com>
AC_CONFIG_FILES in configure.ac works well only with Makefiles.
Other files have to be handled by Makefile.am so depedencies
are tracked properly.
https://fedorahosted.org/freeipa/ticket/6498
Reviewed-By: Martin Basti <mbasti@redhat.com>
AC_CONFIG_FILES in configure.ac works well only with Makefiles.
Other files have to be handled by Makefile.am so depedencies
are tracked properly.
https://fedorahosted.org/freeipa/ticket/6498
Reviewed-By: Martin Basti <mbasti@redhat.com>
AC_CONFIG_FILES in configure.ac works well only with Makefiles.
Other files have to be handled by Makefile.am so depedencies
are tracked properly.
https://fedorahosted.org/freeipa/ticket/6498
Reviewed-By: Martin Basti <mbasti@redhat.com>
The Makefile test targets were lost when gettextize infrastructure was
introduced. Now it is re-added in its modernized form which counts with
generated .pot files.
ipatests/i18n.py is now explicitly setting character encoding in files
it generates. According to gettext manual chapter "Filling in the Header Entry"
the Content-Type header is language-specific so it does not make sense
to fill it in in .pot file.
https://fedorahosted.org/freeipa/ticket/6418
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Lukas Slebodnik <lslebodn@redhat.com>
The target was added to top-level Makefile.am as well so the maintainer
does not need to jump between directories when doing Zanata pull/push
and strip-po.
https://fedorahosted.org/freeipa/ticket/6418
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Lukas Slebodnik <lslebodn@redhat.com>
We now use standard framework generatedby "gettextize" utility.
It has two limitations which I do not consider sufficiently important
to invest into hand-made solution:
1. It can automatically gather strings only from files which have some
file extension like .c or .py. Right now we do not have any
translatable strings in Python files without extensions. Given that these
files will be removed from source tree and replaced with entry points
from setuptools I do not see a reason to invest into supporting this.
2. It does not automatically strip untranslated strings from po files.
This is a manual step in mainteiner's in workflow anyway so I will
add separate Makefile target for it later on.
This commit contains gettextize instrastructure + filled-in files
Makevars and POTFILES.in.
https://fedorahosted.org/freeipa/ticket/6418
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Lukas Slebodnik <lslebodn@redhat.com>
Exclude rpmbuild from pylint checks when make lint is executed.
Clean up the current find expression.
https://fedorahosted.org/freeipa/ticket/6418
Reviewed-By: Petr Spacek <pspacek@redhat.com>
This is a huge hack. rpms target will touch VERSION.m4 file. This change
is then detected by automake Makefiles which subsequently re-execute configure
and make.
We have to workaround fact that variables in new make targets
(executed after new configure) are different than original ones.
Also, we have to 'bake-in' precise snapshot version from Git to
VERSION.m4 inside of RPM tarball so the RPM does not depend on git
anymore.
All this magic slows build down a bit.
Do not enable IPA_VERSION_IS_GIT_SNAPSHOT if you want fastest possible builds.
The option IPA_VERSION_IS_GIT_SNAPSHOT is now enabled by default as it
was before we started the build system refactoring effort.
https://fedorahosted.org/freeipa/ticket/6418
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
make lint and make dist were generating files which were not removed by
make clean.
https://fedorahosted.org/freeipa/ticket/6418
Reviewed-By: Christian Heimes <cheimes@redhat.com>
make rpms and ./makerpms.sh will produce the same RPM packages. The
advantage of makerpms.sh is that it will take care of initial
autoreconf & configure phases as needed.
rpm-build-4.13.0-1.fc24.x86_64 broke parallel build of RPMs.
If you get error
INTERNAL: Exiting with 1 jobserver tokens available; should be 8!
undefine the MAKEFLAGS variable and do not specify neither -j nor -l.
https://fedorahosted.org/freeipa/ticket/6418
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Makefile in doc subdirectory should be integrated into the main build
system but I do not have time to do it now. For now it is enough
to distribute everything.
https://fedorahosted.org/freeipa/ticket/6418
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
At the same time, I've renamed tmpfilesd config file to static name
"ipa.conf" instead of using package-specific name. It had no purpose
and just complicated build and packaging.
Variable substitution into configuration has to be done in Makefile
and not in Autoconf as documented in:
Autoconf v2.69 manual chapter 4.8.2 Installation Directory Variables:
... Most of these variables have values that rely on prefix or
exec_prefix. ... Similarly, you should not rely on AC_CONFIG_FILES
to replace bindir and friends in your shell scripts and other files;
instead, let make manage their replacement.
https://fedorahosted.org/freeipa/ticket/6418
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
BASH completion is now installed by make install.
https://fedorahosted.org/freeipa/ticket/6418
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
