ipa rpc server did set the LANG environment variable on each
request and it was not thread safe which led to unpredictable
mixed languages output. Also, there were mistakes regarding
setting the Accept-Language HTTP header.
Now on each request we're setting the "languages" property
in the context thread local variable and client is setting
the Accept-Language HTTP header correctly.
Also, as the server is caching the schema and the schema can
be generated for several languages it's good to store different
schema fingerprint for each language separately.
pagure: https://pagure.io/freeipa/issue/7238
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
The `module.register` member is added just a few lines
before pylint warns there's none such thing.
https://pagure.io/freeipa/issue/6874
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
If a server is running under py2 it could return a bytes value for
the 'topic_topic' field in the schema response. A py3 client fails
to handle this (in one place it applies 'str' to it, which raises
BytesWarning; in other places it tries to serialise the schema to
JSON which fails because of the bytes value).
Handle the case where 'topic_topic' is not unicode, and handle bytes
values when serialising the schema to JSON.
Fixes: https://pagure.io/freeipa/issue/6809
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Up until now, Bytes parameter was used for certificate parameters
throughout the framework. However, the Bytes parameter does nothing
special for certificates, like validation, so this had to be done
for each of the parameters which were supposed to represent a
certificate.
This commit introduces a special Certificate parameter which takes
care of certificate validation so this does not have to be done
separately. It also makes sure that the certificates represented by
this parameter are always converted to DER format so that we can work
with them in a unified manner throughout the framework.
This commit also makes it possible to pass bytes directly during
instantiation of the Certificate parameter and they are still
represented correctly after their conversion in the _convert_scalar()
method.
https://pagure.io/freeipa/issue/4985
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Replace all `ipa_log_manager.log_mgr.get_logger` calls to create
module-level loggers with `logging.getLogger` calls and deprecate
`ipa_log_manager.log_mgr.get_logger`.
Reviewed-By: Martin Basti <mbasti@redhat.com>
Signed-off-by: David Kreitschmann <david@kreitschmann.de>
Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
Signed-off-by: David Kreitschmann <david@kreitschmann.de>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Cache the KRA transport certificate on disk (in ~/.cache/ipa) as well as
in memory.
https://fedorahosted.org/freeipa/ticket/6652
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Add new pylint AST checker plugin which implements a check for imports
forbidden in IPA. Which imports are forbidden is configurable in pylintrc.
Provide default forbidden import configuration and disable the check for
existing forbidden imports in our code base.
Reviewed-By: Martin Basti <mbasti@redhat.com>
The schema cache used a BytesIO buffer to read/write schema cache before
it got flushed to disk. Since the schema cache is now loaded in one go,
the temporary buffer is no longer needed.
File locking has been replaced with a temporary file and atomic rename.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: David Kupka <dkupka@redhat.com>
It's inefficient to open a zip file over and over again. By loading all
members of the schema cache file at once, the ipa CLI script starts
about 25 to 30% faster for simple cases like help and ping.
Before:
$ time for i in {1..20}; do ./ipa ping >/dev/null; done
real 0m13.608s
user 0m10.316s
sys 0m1.121s
After:
$ time for i in {1..20}; do ./ipa ping >/dev/null; done
real 0m9.330s
user 0m7.635s
sys 0m1.146s
https://fedorahosted.org/freeipa/ticket/6690
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: David Kupka <dkupka@redhat.com>
The `methods` argument of `batch` and `dnsrecords` attribute of `dnsrecord`
were incorrectly defined as `Str` instead of `Any`.
https://fedorahosted.org/freeipa/ticket/6647
Reviewed-By: Martin Basti <mbasti@redhat.com>
As a part of CLI schema cache some data about each previously contacted server
are stored in simple JSON file. The file may get corrupted and became
undecodable for various reasons (parallel access, file system error,
tampering). Since the data are not necessary we should just warn an continue.
https://fedorahosted.org/freeipa/ticket/6578
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
In client-only installations the ipaserver package is not available.
Additional guards prevent pylint to complain about missing ipaserver
package.
https://fedorahosted.org/freeipa/ticket/6468
Reviewed-By: Martin Basti <mbasti@redhat.com>
pylint is having a hard time with distutils.version in tox's virtual
envs. virtualenv uses some tricks to provide a virtual distutils
package, pylint can't cope with.
https://github.com/PyCQA/pylint/issues/73 suggests to use pkg_resources
instead. pkg_resources' version parser has some more benefits, e.g. PEP
440 conformity. But pkg_resources.parse_version() is a heavy weight solution
with reduced functionality, e.g. no access to major version.
For API_VERSION and plugin version we can use a much simpler and faster
approach.
https://fedorahosted.org/freeipa/ticket/6468
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
Cache CommandError answer to schema command to avoid sending the command
to pre-schema servers every time. This information expires after some
time (1 hour) in order to start using schema as soon as the server is
upgraded.
https://fedorahosted.org/freeipa/ticket/6095
Signed-off-by: Jan Cholasta <jcholast@redhat.com>
Signed-off-by: David Kupka <dkupka@redhat.com>
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
Read whole cache into memory and keep it there for lifetime of api
object. This removes the need to repetitively open/close the cache and
speeds up every access to it.
https://fedorahosted.org/freeipa/ticket/6048
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Do not open/close the file with every access to plugins. Extensive
access to filesystem may cause significant slowdown.
https://fedorahosted.org/freeipa/ticket/6048
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
To avoid the need to read all data for a plugin from cache and actualy
use the separately stored help data it must be requested and returned
separately.
https://fedorahosted.org/freeipa/ticket/6048
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Fingerprint can be obtained from schema filename of from ServerInfo
instance. Use FORMAT in path to avoid openening schema just to read its
format.
https://fedorahosted.org/freeipa/ticket/6048
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Once dirty flag is set to True it must not be set back to False.
Otherwise changes are not written back to file.
https://fedorahosted.org/freeipa/ticket/6048
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Whether a parameter is treated like password is determined by the
`password` class attribute defined in the Param class. Whether the CLI will
asks for confirmation of a password parameter depends on the value of the
`confirm` kwarg of the Password class.
Move the `confirm` kwarg from the Password class to the Param class, so
that it can be used by any Param subclass which has the `password` class
attribute set to True.
This fixes confirmation of the --key option of otptoken-add, which is a
Bytes subclass with `password` set to True.
https://fedorahosted.org/freeipa/ticket/6174
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: David Kupka <dkupka@redhat.com>
When client comunicates with server that doesn't support 'schema'
command it needs to determine its api version to be able to use the
right compat code. Storing information about server version reduces the
need to call 'env' or 'ping' command only to first time the server is
contacted.
https://fedorahosted.org/freeipa/ticket/6069
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Store name, summary, topic_topic and exclude in single entry in cache
for all commands. These data are needed for help and storing and
loading them together allows fast help response.
https://fedorahosted.org/freeipa/ticket/6048
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Information about schema cache format is stored in every cache item.
When schema cache format changes in incompatible way format will be
increased. When format stored in cache doesn't match currently used
format the entry in cache is ignored.
https://fedorahosted.org/freeipa/ticket/6048
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Avoid need to instantiate all commands just to get information for
displaying help.
https://fedorahosted.org/freeipa/ticket/6048
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Check presence of schema in cache (and download it if necessary) on
__init__ instead of with each __getitem__ call. Prefill internal
dictionary with empty record for each command to be able to quickly
determine if requested command exist in schema or not. Rest of schema
data are read from cache on first attempt to retrive them.
https://fedorahosted.org/freeipa/ticket/6048https://fedorahosted.org/freeipa/ticket/6069
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Copy & paste accident caused the ping command to be called with an unwanted
argument, which results in an exception.
Remove the argument to fix it.
https://fedorahosted.org/freeipa/ticket/6129
This patch introduces a separate Principal parameter that allows the framework
to syntactically validate incoming/outcoming principals by using a single
shared codebase.
https://fedorahosted.org/freeipa/ticket/3864
Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Bundle remote plugin interface definitions for servers which lack API
schema support. These server API versions are included:
* 2.49: IPA 3.1.0 on RHEL/CentOS 6.5+,
* 2.114: IPA 4.1.4 on Fedora 22,
* 2.156: IPA 4.2.0 on RHEL/CentOS 7.2 and IPA 4.2.4 on Fedora 23,
* 2.164: IPA 4.3.1 on Fedora 23.
For servers with other API versions, the closest lower API version is used.
https://fedorahosted.org/freeipa/ticket/4739
Reviewed-By: David Kupka <dkupka@redhat.com>
Store schema in per user cache. Together with schemas also information
about mapping between server and fingerprint is stored to reduce traffic.
https://fedorahosted.org/freeipa/ticket/4739
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Allow multiple incompatible versions of a plugin using the same name. The
current plugins are assumed to be version '1'.
The unique identifier of plugins was changed from plugin name to plugin
name and version. By default, the highest version available at build time
is used. If the plugin is an unknown remote plugin, version of '1' is used
by default.
https://fedorahosted.org/freeipa/ticket/4427
Reviewed-By: David Kupka <dkupka@redhat.com>
Advertise param's default value even when `autofill` is False. When
`autofill` is False, set `alwaysask` to True in the schema, as it is
semantically equivallent and removes redundancy.
This fixes default value disappearing in CLI for some params.
https://fedorahosted.org/freeipa/ticket/4739
Reviewed-By: David Kupka <dkupka@redhat.com>
Instead, support excluding commands from specified contexts and exclude
commands with NO_CLI set from the 'cli' context.
https://fedorahosted.org/freeipa/ticket/4739
Reviewed-By: David Kupka <dkupka@redhat.com>
Remove the `autofill` kwarg from param schema. On the server, include
default value only if autofill is set. On the client, set autofill if param
has a default value.
Remove the `deprecated_cli_aliases`, `hint` and `sortorder` kwargs, and the
`dnsrecord_extra`, `dnsrecord_part` and `suppress_empty` flags from param
schema, as they are now handled exclusively on the client.
Replace the `no_option` and `no_output` flags in param schema with
exclusion of the param in 'cli' and 'webui' contexts.
Remove the `no_display` flag from output schema, as it is now handled
exclusively on the client.
https://fedorahosted.org/freeipa/ticket/4739
Reviewed-By: David Kupka <dkupka@redhat.com>
Rather than having args and options separately in command schema, merge
them together and use new `positional` param flag to differentiate between
them.
https://fedorahosted.org/freeipa/ticket/4739
Reviewed-By: David Kupka <dkupka@redhat.com>
