Commit Graph

137 Commits

Author SHA1 Message Date
Aleksei Slaikovskii
6c5a7464b2
Fixing translation problems
ipa rpc server did set the LANG environment variable on each
request and it was not thread safe which led to unpredictable
mixed languages output. Also, there were mistakes regarding
setting the Accept-Language HTTP header.

Now on each request we're setting the "languages" property
in the context thread local variable and client is setting
the Accept-Language HTTP header correctly.

Also, as the server is caching the schema and the schema can
be generated for several languages it's good to store different
schema fingerprint for each language separately.

pagure: https://pagure.io/freeipa/issue/7238
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
2018-01-31 16:03:19 +01:00
Christian Heimes
d3f43a6737 LGTM: Membership test with a non-container
Silence false positive by using isinstance(value, dict).

Also clean up and optimize most common cases.

https://pagure.io/freeipa/issue/7344

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
2018-01-09 07:53:28 +01:00
Christian Heimes
8cb756a229 Fix pylint warnings inconsistent-return-statements
Add consistent return to all functions and methods that are covered by
tox -e pylint[23]. I haven't checked if return None is always a good
idea or if we should rather raise an error.

See: https://pagure.io/freeipa/issue/7326
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
2017-12-18 11:51:14 +01:00
Rob Crittenden
17bda0b1a5 Use the CA chain file from the RPC context
The value can be passed in the create_connection() call but
wasn't used outside that call. It already defaults to
api.env.tls_ca_cert so the context.ca_certfile should be used
instead so the caller can override the cert chain on a
per-connection basis. This may be handy in the future when
there is IPA-to-IPA trust, or for IPA-to-IPA migration.

https://pagure.io/freeipa/issue/7145

Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2017-12-04 10:29:19 -05:00
Stanislav Laznicka
61605d28d8 parameters: introduce CertificateSigningRequest
Previously, CSRs were handled as a Str parameter which brought
trouble to Python 3 because of its more strict type requirements.
We introduce a CertificateSigningRequest parameter which allows to
use python-cryptography x509.CertificateSigningRequest to represent
CSRs in the framework.

https://pagure.io/freeipa/issue/7131
2017-10-25 09:44:37 +02:00
Stanislav Laznicka
5acd484090
rpc: don't decode cookie_string if it's None
This removes an ugly debug message from client installation

Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
2017-09-20 12:58:33 +02:00
Stanislav Laznicka
e09b6c2602 rpc: don't encode bytes
bytes.encode() appeared in rpc.py by a mistake, should have been
bytes.decode()

https://pagure.io/freeipa/issue/4985

Reviewed-By: Felipe Volpone <fbarreto@redhat.com>
2017-08-30 12:44:46 +02:00
Stanislav Laznicka
5a44ca6383 Create a Certificate parameter
Up until now, Bytes parameter was used for certificate parameters
throughout the framework. However, the Bytes parameter does nothing
special for certificates, like validation, so this had to be done
for each of the parameters which were supposed to represent a
certificate.

This commit introduces a special Certificate parameter which takes
care of certificate validation so this does not have to be done
separately. It also makes sure that the certificates represented by
this parameter are always converted to DER format so that we can work
with them in a unified manner throughout the framework.

This commit also makes it possible to pass bytes directly during
instantiation of the Certificate parameter and they are still
represented correctly after their conversion in the _convert_scalar()
method.

https://pagure.io/freeipa/issue/4985

Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-07-27 10:28:58 +02:00
Jan Cholasta
7a482b7c72 logging: do not log into the root logger
Deprecate `ipa_log_manager.root_logger` and replace all calls to it with
module-level logger calls.

Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-07-14 15:55:59 +02:00
Jan Cholasta
ffadcb0414 logging: remove object-specific loggers
Remove all object-specific loggers, with the exception of `Plugin.log`,
which is now deprecated. Replace affected logger calls with module-level
logger calls.

Deprecate object-specific loggers in `ipa_log_manager.get_logger`.

Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-07-14 15:55:59 +02:00
Stanislav Laznicka
d665224a85 session_storage: Correctly handle string/byte types
In session_storage.py, store_data() stores data as the bytes data
type but get_data() is returning a string. Have get_data() return
bytes as well.

https://pagure.io/freeipa/issue/4985

Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-06-09 16:59:53 +02:00
Stanislav Laznicka
e1f8684e85 rpc: avoid possible recursion in create_connection
There was a recursion in RPCClient.create_connection() which under rare
circumstances would not have an ending condition. This commit removes
it and cleans up the code a bit as well.

https://pagure.io/freeipa/issue/6796

Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
2017-06-02 16:43:02 +02:00
Stanislav Laznicka
79d1752577 rpc: preparations for recursion fix
Made several improvements to coding style:
 - same use of KerberosError throughout the module
 - removed some unused variables
 - moved code from try-except blocks if it didn't have to be there
 - preparations for putting most of RPCClient.create_connection()
   to loop

https://pagure.io/freeipa/issue/6796

Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
2017-06-02 16:43:02 +02:00
Stanislav Laznicka
81a808caeb Avoid possible endless recursion in RPC call
This commit removes recursion in RPCClient.forward() which may lack
end condition.

https://pagure.io/freeipa/issue/6796

Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
2017-06-02 16:43:02 +02:00
Simo Sorce
d63326632b
Prevent churn on ccaches
We slice down the received cookie so that just the content that matter
is preserved. Thi is ok because servers can't trust anything else anyway
and will accept a cookie with the ancillary data missing.

By removing variable parts like the expiry component added by
mod_session or the Expiration or Max-Age metadata we keep only the part
of the cookie that changes only when a new session is generated.

This way when storing the cookie we actually add a new entry in the
ccache only when the session actually changes, and this prevents churn
on FILE based ccaches.

Related https://pagure.io/freeipa/issue/6775

Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
2017-03-28 13:36:30 +02:00
Simo Sorce
fbbeb132bf
Handle failed authentication via cookie
If cookie authentication fails and we get back a 401 see if we
tried a SPNEGO auth by checking if we had a GSSAPI context. If not
it means our session cookie was invalid or expired or some other
error happened on the server that requires us to try a full SPNEGO
handshake, so go ahead and try it.

Fixes https://pagure.io/freeipa/issue/6775

Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
2017-03-28 13:36:30 +02:00
Christian Heimes
b2bdd2e1a9
Add debug logging for keep-alive
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
2017-03-20 19:24:28 +01:00
Christian Heimes
7beb6d1cad
Use connection keep-alive
Do not forcefully close the connection after every request. This enables
HTTP connection keep-alive, also known as persistent TCP and TLS/SSL
connection. Keep-alive speed up consecutive HTTP requests by 15% (for
local, low-latency network connections to a fast server) to multiple
times (high latency connections or remote peers).

https://pagure.io/freeipa/issue/6641

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
2017-03-20 19:24:28 +01:00
Stanislav Laznicka
0bb858ea77
Add debug log in case cookie retrieval went wrong
https://pagure.io/freeipa/issue/6774

Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-03-20 19:16:30 +01:00
Jan Cholasta
8295848bfe rpc: fix crash in verbose mode
Fix a crash caused by feeding incorrect data to `json.dumps()` in
`JSONServerProxy.__request()` introduced by commit
8159c2883b.

https://pagure.io/freeipa/issue/6734

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2017-03-13 15:44:44 +01:00
Simo Sorce
7cab959555 Store session cookie in a ccache option
Instead of using the kernel keyring, store the session cookie within the
ccache. This way kdestroy will really wipe away all credentials.

Ticket: https://pagure.io/freeipa/issue/6661

Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
2017-03-10 12:40:08 +01:00
Stanislav Laznicka
24eeb4d6a3 Fix cookie with Max-Age processing
When cookie has Max-Age set it tries to get expiration by adding
to a timestamp. Without this patch the timestamp would be set to
None and thus the addition of timestamp + max_age fails

https://pagure.io/freeipa/issue/6718

Reviewed-By: Simo Sorce <ssorce@redhat.com>
2017-03-06 10:48:32 +00:00
Stanislav Laznicka
dfd560a190 Remove NSSConnection from the Python RPC module
NSSConnection was causing a lot of trouble in the past and there is
a lot of logic around it just to make it not fail. What's more,
when using NSS to create an SSL connection in FIPS mode, NSS
always requires database password which makes the `ipa` command
totally unusable.

NSSConnection is therefore replaced with Python's
httplib.HTTPSConnection which is OpenSSL based.

The HTTPSConnection is set up to handle authentication with client
certificate for connections to Dogtag server as RA agent. It allows
to handle client cert/private key in separate files and also
encrypted private key files.

https://fedorahosted.org/freeipa/ticket/5695

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2017-03-01 09:43:41 +00:00
Christian Heimes
1d7fcfe15d Explain more performance tricks in doc string
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2017-02-15 17:27:56 +01:00
Christian Heimes
3cac0378e9 Pretty print JSON in debug mode (debug level >= 2)
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2017-02-15 17:27:56 +01:00
Christian Heimes
b12b1e4c0b Convert list to tuples
Some tests assume that JSON deserializier returns tuples instead of
lists. I don't think it is necessary but let's pass the tests for now.

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2017-02-15 17:27:56 +01:00
Christian Heimes
8159c2883b Faster JSON encoder/decoder
Improve performance of FreeIPA's JSON serializer and deserializer.

* Don't indent and sort keys. Both options trigger a slow path in
  Python's json package. Without indention and sorting, encoding
  mostly happens in optimized C code.
* Replace O(n) type checks with O(1) type lookup and eliminate
  the use of isinstance().
* Check each client capability only once for every conversion.
* Use decoder's obj_hook feature to traverse the object tree once and
  to eliminate calls to isinstance().

Closes: https://fedorahosted.org/freeipa/ticket/6655
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2017-02-15 17:27:56 +01:00
Simo Sorce
e4d462ad53 Insure removal of session on identity change
If we are changing identiy (different principal) insure we remove the
session cookie stored on the rpc context so that we do not mistakenly
connect with the previous identity credentials.

https://fedorahosted.org/freeipa/ticket/6543

Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2017-02-15 07:13:37 +01:00
Simo Sorce
41c1efc44a Allow rpc callers to pass ccache and service names
This allows code to use multiple ccaches without having to muck with the
process global environment variables (KRB5CCNAME).

https://fedorahosted.org/freeipa/ticket/6543

Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2017-02-15 07:13:37 +01:00
Martin Basti
4c84341b8b py3: send Decimal number as string instead of base64 encoded value
for Decimal only from client to server direction uses __base64__
notation. Server replies with pure string for Decimal data, and also
server is able to parse string and create decimal values where needed.

without this we need ugly py3 code:
-        return {'__base64__': base64.b64encode(str(val))}
+        return {'__base64__': base64.b64encode(
+            str(val).encode('ascii')).decode('ascii')}

https://fedorahosted.org/freeipa/ticket/4985

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2017-02-08 08:32:44 +01:00
Fraser Tweedale
924794f62b Fix reference before assignment
In 'store_session_cookie', if the server does not set the session
cookie for some reason, the 'session_cookie' variable does not get
assigned, resulting in UnboundLocalError.  Set an initial value of
'None'.

Fixes: https://fedorahosted.org/freeipa/ticket/6636
Reviewed-By: Martin Basti <mbasti@redhat.com>
2017-02-06 17:28:39 +01:00
Martin Basti
18337bf7f7 py3: decode bytes for json.loads()
In py 3.5 json.loads requires to have string as input, all bytes must be
decoded.

Note: python 3.6 supports bytes for json.loads()

https://fedorahosted.org/freeipa/ticket/4985

Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2017-01-24 13:25:47 +01:00
Simo Sorce
397f2be9df Add compatibility code to retrieve headers
Python3 removed the getheaders() function and replaced it with a
get_all() one. Add compat code.

https://fedorahosted.org/freeipa/ticket/6558

Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-12-14 17:54:11 +01:00
Simo Sorce
f167869371 Properly handle multiple cookies in rpc lib.
Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-12-08 12:03:12 +01:00
Simo Sorce
560ab9e317 Properly handle multiple cookies in rpcclient
Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-12-08 12:03:12 +01:00
Christian Heimes
38e8719f72 Python3 pylint fixes
Sprinkle 'pylint disable' comments over the code base to silence a bunch
of pylint warnings on Python 3. All silenced warnings are harmless and
not bugs.

https://fedorahosted.org/freeipa/ticket/4985

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-11-25 16:18:22 +01:00
Simo Sorce
2f567f0e8e Fix error message encoding
- Use the correct unicode string for an error message, otherwise an
exception will generate another exception about incorrect type,
masking the original error.

Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
2016-10-25 14:01:06 +02:00
Jan Cholasta
0d370a959b pylint: enable the import-error check
Check for import errors with pylint to make sure new python package
dependencies are not overlooked.

https://fedorahosted.org/freeipa/ticket/6418

Reviewed-By: Petr Spacek <pspacek@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2016-10-24 14:11:08 +02:00
Martin Basti
4628522c53 Pylint: fix the rest of unused local variables
Reviewed-By: Pavel Vomacka <pvomacka@redhat.com>
2016-10-11 16:50:32 +02:00
Martin Basti
45e3aee352 Pylint: enable check for unused-variables
Unused variables may:
* make code less readable
* create dead code
* potentialy hide issues/errors

Enabled check should prevent to leave unused variable in code

Check is locally disabled for modules that fix is not clear or easy or have too many occurences of
unused variables

Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
2016-09-27 13:35:58 +02:00
Martin Babinsky
974eb7b5ef ipalib: introduce Principal parameter
This patch introduces a separate Principal parameter that allows the framework
to syntactically validate incoming/outcoming principals by using a single
shared codebase.

https://fedorahosted.org/freeipa/ticket/3864

Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2016-07-01 09:37:25 +02:00
Jan Cholasta
5a4a29be1c rpc: do not validate command name in RPCClient.forward
The validation is already done on the server.

This allows manually forwarding commands unknown to the client but known
to the server.

https://fedorahosted.org/freeipa/ticket/4739

Reviewed-By: David Kupka <dkupka@redhat.com>
2016-06-03 09:00:34 +02:00
Jan Cholasta
11de39651f rpc: optimize JSON-RPC response handling
Speed up JSON-RPC response handling by putting received response data
fragments in a list and joining them at once instead of concatenating
each fragment one by one.

https://fedorahosted.org/freeipa/ticket/4739

Reviewed-By: David Kupka <dkupka@redhat.com>
2016-06-03 09:00:34 +02:00
Jan Cholasta
e056b8dea2 rpc: allow overriding NSS DB directory in API config
Add new `nss_dir` API config option to allow rpcclient to use a non-default
NSS DB for the connection.

https://fedorahosted.org/freeipa/ticket/4739

Reviewed-By: David Kupka <dkupka@redhat.com>
2016-06-03 09:00:34 +02:00
Jan Cholasta
43dc424041 rpc: respect API config in RPCClient.create_connection
When connecting rpcclient, get the default values of the `verbose`,
`fallback` and `delegate` options from API config rather than hard-code
them.

https://fedorahosted.org/freeipa/ticket/4739

Reviewed-By: David Kupka <dkupka@redhat.com>
2016-06-03 09:00:34 +02:00
Jan Cholasta
6cfb9d73d9 dns: move code shared by client and server to separate module
Move the shared code to a new ipalib.dns module.

https://fedorahosted.org/freeipa/ticket/4739

Reviewed-By: David Kupka <dkupka@redhat.com>
2016-06-03 09:00:34 +02:00
Jan Cholasta
e0275abe6f rpc: include structured error information in responses
Include keyword arguments of exceptions in RPC responses. This is limited
to JSON-RPC, as XML-RPC does not support additional data in error
responses.

Include keyword arguments of messages in RPC responses.

Include keyword arguments of exceptions in batch command result.

https://fedorahosted.org/freeipa/ticket/4739

Reviewed-By: David Kupka <dkupka@redhat.com>
2016-05-25 16:06:26 +02:00
Jan Cholasta
c37aba291f rpc: do not crash when unable to parse JSON
When unable to parse JSON response from the server, properly raise
JSONError not to cause a crash.

https://fedorahosted.org/freeipa/ticket/4739

Reviewed-By: David Kupka <dkupka@redhat.com>
2016-05-25 16:06:26 +02:00
Petr Viktorin
baaa041b8a ipalib.rpc: Send base64-encoded data as string under Python 3
Python 3's JSON library cannot deal with bytes, so decode
base64-encoded data to string.

Part of the work for https://fedorahosted.org/freeipa/ticket/4985

Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Petr Spacek <pspacek@redhat.com>
2016-05-05 16:42:46 +02:00
Michael Simacek
831856ea55 Fix bytes/string handling in rpc
https://fedorahosted.org/freeipa/ticket/5638

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2016-02-17 10:41:29 +01:00