ignore_import_errors was added in 9b534238 to build FreeIPA ACI/API with
some dependencies missing. It turns out that the import hook doesn't
play nice with other meta importers or Cython-generated code like lxml:
./makeaci: ipaserver/plugins/dogtag.py:246: ignoring ImportError: No module named lxml.re
Traceback (most recent call last):
File "./makeaci", line 134, in <module>
main(options)
File "./makeaci", line 107, in main
api.finalize()
File "ipalib/plugable.py", line 733, in finalize
self.__do_if_not_done('load_plugins')
File "ipalib/plugable.py", line 425, in __do_if_not_done
getattr(self, name)()
File "ipalib/plugable.py", line 614, in load_plugins
self.add_package(package)
File "ipalib/plugable.py", line 641, in add_package
module = importlib.import_module(name)
File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "ipaserver/plugins/dogtag.py", line 246, in <module>
from lxml import etree
File "src/lxml/etree.pyx", line 93, in init lxml.etree
File "src/lxml/_elementpath.py", line 58, in init lxml._elementpath
AttributeError: 'FailedImport' object has no attribute 'compile'
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Derive the default value of `xmlrpc_uri` and `ldap_uri` from `server`.
Derive the default value of `basedn` from `domain`.
Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Domain, realm, basedn, xmlrpc_uri, ldap_uri do not have any reasonable default.
This patch removes hardcoded default so the so the code which depends
on these values blows up early and does not do crazy stuff
with default values instead of real ones.
This should help to uncover issues caused by improper ipalib
initialization.
Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Sprinkle 'pylint disable' comments over the code base to silence a bunch
of pylint warnings on Python 3. All silenced warnings are harmless and
not bugs.
https://fedorahosted.org/freeipa/ticket/4985
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Add import hook to makeapi and makeaci which makes them ignore import
errors in modules in our source tree and instead print a warning.
This makes it possible to build IPA without having to have most of our
runtime dependencies installed.
https://fedorahosted.org/freeipa/ticket/6418
Reviewed-By: Petr Spacek <pspacek@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
This commit removes unused variables or rename variables as "expected to
be unused" by using "_" prefix.
This covers only cases where fix was easy or only one unused variable
was in a module
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
This patch introduces a separate Principal parameter that allows the framework
to syntactically validate incoming/outcoming principals by using a single
shared codebase.
https://fedorahosted.org/freeipa/ticket/3864
Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Use the default context rather the server context for code not running
inside the server.
This prevents the affected code from attempting to initialize the session
manager.
https://fedorahosted.org/freeipa/ticket/5988
Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
Allow multiple incompatible versions of a plugin using the same name. The
current plugins are assumed to be version '1'.
The unique identifier of plugins was changed from plugin name to plugin
name and version. By default, the highest version available at build time
is used. If the plugin is an unknown remote plugin, version of '1' is used
by default.
https://fedorahosted.org/freeipa/ticket/4427
Reviewed-By: David Kupka <dkupka@redhat.com>
Capture the server API rather than client API in API.txt. Client API may be
affected by client-side plugins and thus may not correspond to what is
transmitted over the wire.
https://fedorahosted.org/freeipa/ticket/4739
Reviewed-By: David Kupka <dkupka@redhat.com>
Dynamically create plugin package for the remote server with modules and
commands based on the API schema when client API is finalizes. For in-tree
API instances, use ipalib.plugins directly.
https://fedorahosted.org/freeipa/ticket/4739
Reviewed-By: David Kupka <dkupka@redhat.com>
Add new Param keyword argument cli_metavar to specify the stand-in for CLI
option arguments in command help text. Uppercase class name is used by
default.
https://fedorahosted.org/freeipa/ticket/4739
Reviewed-By: David Kupka <dkupka@redhat.com>
Implement the `name`, `doc` and `summary` Plugin attributes as properties
to allow them to be overriden in sub-classes.
Always use .doc rather than .__doc__ to access plugin documentation.
Remove the mostly unused `module`, `fullname`, `bases` and `label`
attributes.
https://fedorahosted.org/freeipa/ticket/4739
Reviewed-By: David Kupka <dkupka@redhat.com>
Iterate over all plugin packages defined in the API to find the given
topic module. The last module found has priority.
This will allow topics to be defined in client-side plugins.
https://fedorahosted.org/freeipa/ticket/4739
Reviewed-By: David Kupka <dkupka@redhat.com>
Specify module topic by name rather than by name and summary. A topic
module of the topic name must exist. Summary is extracted from the
docstring of the topic module.
This changes makes topic handling more generic and consistent between
modules and commands.
https://fedorahosted.org/freeipa/ticket/4739
Reviewed-By: David Kupka <dkupka@redhat.com>
Help topic can now be specified in the 'topic' class attribute of command
plugins. Default value is the name of the module where the command is
defined.
This allows defining a command outside of the topic module.
https://fedorahosted.org/freeipa/ticket/4739
Reviewed-By: David Kupka <dkupka@redhat.com>
Change Param formatting to:
* always use quantified names rather than the `required` and `multivalue`
kwargs,
* ignore kwargs with default value,
* ignore kwargs related to validation, as validation is now strictly
server-side,
* ignore the `attribute` and `primary_key` kwargs, as they are relevant
only on object params,
* ignore the `include` and `exclude` kwargs, as makeapi takes into account
only params available in the 'cli' context,
* ignore the unused `csv` kwarg.
Format optional Output arguments as kwargs.
https://fedorahosted.org/freeipa/ticket/4739
Reviewed-By: David Kupka <dkupka@redhat.com>
In Python 3, `print` is no longer a statement. Call it as a function
everywhere, and include the future import to remove the statement
in Python 2 code as well.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
This feature was removed in Python 3 to ease introspection.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
This object will allow splitting large translatable strings into more
pieces, so translators don't have to re-translate the entire text
when a small part changes.
https://fedorahosted.org/freeipa/ticket/3587
The API version the client sends can now be used to check what the client
expects or is capable of.
All version tests IPA does will be be named and listed in one module,
ipalib.capabilities, which includes a function to test a specific capability
against an API version.
Similarly to Python's __future__ module, capabilities.py also serves as
documentation of backwards-incompatible changes to the API.
The first capability to be defined is "messages". Recent enough clients can
accept a list of warnings or other info under the "messages" key in the
result dict.
If a JSON client does not send the API version, it is assumed this is a testing
client (e.g. curl from the command line). Such a client "has" all capabilities,
but it will always receive a warning mentioning that forward compatibility
is not guaranteed.
If a XML client does not send the API version, it is assumed it uses the API
version before capabilities were introduced. (This is to keep backwards
compatibility with clients containing bug https://fedorahosted.org/freeipa/ticket/3294)
Whenever a capability is added, the API version must be incremented.
To ensure that, capabilities are written to API.txt and checked by
`makeapi --validate`.
Design page: http://freeipa.org/page/V3/Messages
Ticket: https://fedorahosted.org/freeipa/ticket/2732
Python does not guarantee dict order. Our reliance on it in `makeapi`
is technically incorrect, even if it doesn't matter yet in any of our
developers' environments.
This patch sorts the options by name instead of relying on dict order.
As an added benefit, future API.txt diffs should be be neater.
Move the code for encoding boolean values to LDAP boolean syntax from the
Parameter class to the Encoder class, where the rest of LDAP encoding takes
place. Remove encoding code from the Parameter class altogether, as all LDAP
encoding should be done in the Encoder class.
Replace all occurences of create_default with equivalent default_from
and remove create_default from the framework. This is needed for
proper parameter validation, as there is no way to tell which
parameters to validate prior to calling create_default, because
create_default does not provide information about which parameters are
used for generating the default value.
This patch changes the way plugins are initialized. Instead of
finalizing all the plugins at once, plugins are finalized only after
they are accessed (currently applies to Command, Object and
Attribute subclasses, only in CLI by default).
This change provides significant performance boost, as only the
plugins that are actually used are finalized.
ticket 1336
Some Param or Output attributes do not cause API incompatibility
(e.g. doc, label or callables) and does not need to be included
in API.txt. When these attributes are modified, a lot of bogus
changes may get in API.txt - making the real API changes less
detectable.
https://fedorahosted.org/freeipa/ticket/2107
Iterate over all API commands and perform the following validation:
* Every command must have documentation
and it must be marked for international translation
* Every module hosting a command must have documentation
and it must be marked for international translation
* Every module topic must be marked for international translation
For every error found emit a diagnostic.
Emit a summary of total errors found.
Return error flag if errors found, zero otherwise.
Doc parts are not removed from the API completely. This leads to
unnecessary updates to API.txt when the option/argument documentation
is changed.
This patch replaces unreliable doc stripping function with a regular
expression. It works for all current doc strings (simple string or
GetText). The only limitation is that the RE supports only up to
2 levels of nested parentheses in doc string.
https://fedorahosted.org/freeipa/ticket/1057
Doc parts are not removed from the API completely. This leads to
unnecessary updates to API.txt when the option/argument documentation
is changed.
This patch replaces unreliable doc stripping function with a regular
expression. It works for all current doc strings (simple string or
GetText). The only limitation is that the RE supports only up to
2 levels of nested parentheses in doc string.
https://fedorahosted.org/freeipa/ticket/1057
makeapi script is used to check if ipalib API is consistent with the
known state in API.txt. When the API is changed, major API version
should be updated. However, when new options/arguments/outputs were
added to an ipalib command, `makeapi --validate' call did not capture
this.
This patch fixes this issue and ensures that also the last command
in API.txt is checked (it was not before this patch).
https://fedorahosted.org/freeipa/ticket/868
The API does a fair number of self tests and locking to assure that the
registered commands are consistent and will work. This does not need
to be done on a production system and adds additional overhead causing
somewhere between a 30 and 50% decrease in performance.
Because makeapi is executed when a build is done ensure that it is
executed in developer mode to ensure that the framework is ok.
ticket 751
This patch contains 2 parts.
The first part is a small utility to create and validate the current
API. To do this it needs to load ipalib which on a fresh system
introduces a few problems, namely that it relies on a python plugin
to set the default encoding to utf8. For our purposes we can skip that.
It is also important that any optional plugins be loadable so the
API can be examined.
The second part is a version exchange between the client and server.
The version has a major and a minor version. The major verion is
updated whenever existing API changes. The minor version is updated when
new API is added. A request will be rejected if either the major versions
don't match or if the client major version is higher than then server
major version (though by implication new API would return a command not
found if allowed to proceed).
To determine the API version of the server from a client use the ping
command.
ticket 584
