If the server responds 401 and the response body is empty, the
client raises ResponseNotReady. This occurs because:
1. For a non-200 response, the response read only if the
Content-Length header occurs.
2. The response must be read before another request (e.g. the
follow-up request with WWW-Authenticate header set), and this
condition was not met. For details see
https://github.com/python/cpython/blob/v3.6.7/Lib/http/client.py#L1305-L1321.
This situation should not arise in regular use, because the client
either has a session cookie, or, knowing the details of the server
it is contacting, it establishes the GSS-API context and includes
the WWW-Authenticate header in the initial request.
Nevertheless, this problem has been observed in the wild. I do not
know its ordinary cause(s), but one can force the issue by removing
an authenticated user's session cache from /run/ipa/ccaches, then
performing a request.
Resolve the issue by always reading the response. It is safe to
call response.read() regardless of whether the Content-Length header
appears, or whether the body is empty.
Fixes: https://pagure.io/freeipa/issue/7752
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Replace six.moves and six.StringIO/BytesIO imports with cannonical
Python 3 packages.
Note: six.moves.input behaves differently than builtin input function.
Therefore I left six.moves.input for now.
See: https://pagure.io/freeipa/issue/7715
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
In Python 3, six.integer_types is (int,). In most places, the alias can
be simply replaced with int. In other places, it was possible to
simplify the code by unpacking the tuple.
See: https://pagure.io/freeipa/issue/7715
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Python 2 had old style and new style classes. Python 3 has only new
style classes. There is no point to subclass from object any more.
See: https://pagure.io/freeipa/issue/7715
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Fix the following violations aiming to support Pylint 2.0
- `unneeded-not` (C0113):
Consider changing "not item in items" to "item not in items" used
when a boolean expression contains an unneeded negation.
- `useless-import-alias` (C0414):
Import alias does not rename original package Used when an import
alias is same as original package.e.g using import numpy as numpy
instead of import numpy as np
- `raising-format-tuple` (W0715):
Exception arguments suggest string formatting might be intended Used
when passing multiple arguments to an exception constructor, the
first of them a string literal containing what appears to be
placeholders intended for formatting
- `bad-continuation` (C0330):
This was already included on the disable list, although with current
version of pylint (2.0.0.dev2) violations at the end of the files
are not being ignored.
See: https://github.com/PyCQA/pylint/issues/2278
- `try-except-raise` (E0705):
The except handler raises immediately Used when an except handler
uses raise as its first or only operator. This is useless because it
raises back the exception immediately. Remove the raise operator or
the entire try-except-raise block!
- `consider-using-set-comprehension` (R1718):
Consider using a set comprehension Although there is nothing
syntactically wrong with this code, it is hard to read and can be
simplified to a set comprehension.Also it is faster since you don't
need to create another transient list
- `dict-keys-not-iterating` (W1655):
dict.keys referenced when not iterating Used when dict.keys is
referenced in a non-iterating context (returns an iterator in
Python 3)
- `comprehension-escape` (W1662):
Using a variable that was bound inside a comprehension Emitted when
using a variable, that was bound in a comprehension handler, outside
of the comprehension itself. On Python 3 these variables will be
deleted outside of the comprehension.
Issue: https://pagure.io/freeipa/issue/7614
Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
The server installer had no console logger set so print
statements were used for communication. Now that a logger
is enabled the extra prints need to be dropped.
A number of logger.info statements have been upgraded
to debug since they do not need to appear on the console
by default.
https://pagure.io/freeipa/issue/6760
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
On multiple occasions, SRV query answers were not properly sorted by
priority. Records with same priority weren't randomized and shuffled.
This caused FreeIPA to contact the same remote peer instead of
distributing the load across all available servers.
Two new helper functions now take care of SRV queries. sort_prio_weight()
sorts SRV and URI records. query_srv() combines SRV lookup with
sort_prio_weight().
Fixes: https://pagure.io/freeipa/issue/7475
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
ipa rpc server did set the LANG environment variable on each
request and it was not thread safe which led to unpredictable
mixed languages output. Also, there were mistakes regarding
setting the Accept-Language HTTP header.
Now on each request we're setting the "languages" property
in the context thread local variable and client is setting
the Accept-Language HTTP header correctly.
Also, as the server is caching the schema and the schema can
be generated for several languages it's good to store different
schema fingerprint for each language separately.
pagure: https://pagure.io/freeipa/issue/7238
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
Silence false positive by using isinstance(value, dict).
Also clean up and optimize most common cases.
https://pagure.io/freeipa/issue/7344
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Add consistent return to all functions and methods that are covered by
tox -e pylint[23]. I haven't checked if return None is always a good
idea or if we should rather raise an error.
See: https://pagure.io/freeipa/issue/7326
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The value can be passed in the create_connection() call but
wasn't used outside that call. It already defaults to
api.env.tls_ca_cert so the context.ca_certfile should be used
instead so the caller can override the cert chain on a
per-connection basis. This may be handy in the future when
there is IPA-to-IPA trust, or for IPA-to-IPA migration.
https://pagure.io/freeipa/issue/7145
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Previously, CSRs were handled as a Str parameter which brought
trouble to Python 3 because of its more strict type requirements.
We introduce a CertificateSigningRequest parameter which allows to
use python-cryptography x509.CertificateSigningRequest to represent
CSRs in the framework.
https://pagure.io/freeipa/issue/7131
bytes.encode() appeared in rpc.py by a mistake, should have been
bytes.decode()
https://pagure.io/freeipa/issue/4985
Reviewed-By: Felipe Volpone <fbarreto@redhat.com>
Up until now, Bytes parameter was used for certificate parameters
throughout the framework. However, the Bytes parameter does nothing
special for certificates, like validation, so this had to be done
for each of the parameters which were supposed to represent a
certificate.
This commit introduces a special Certificate parameter which takes
care of certificate validation so this does not have to be done
separately. It also makes sure that the certificates represented by
this parameter are always converted to DER format so that we can work
with them in a unified manner throughout the framework.
This commit also makes it possible to pass bytes directly during
instantiation of the Certificate parameter and they are still
represented correctly after their conversion in the _convert_scalar()
method.
https://pagure.io/freeipa/issue/4985
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Remove all object-specific loggers, with the exception of `Plugin.log`,
which is now deprecated. Replace affected logger calls with module-level
logger calls.
Deprecate object-specific loggers in `ipa_log_manager.get_logger`.
Reviewed-By: Martin Basti <mbasti@redhat.com>
In session_storage.py, store_data() stores data as the bytes data
type but get_data() is returning a string. Have get_data() return
bytes as well.
https://pagure.io/freeipa/issue/4985
Reviewed-By: Martin Basti <mbasti@redhat.com>
There was a recursion in RPCClient.create_connection() which under rare
circumstances would not have an ending condition. This commit removes
it and cleans up the code a bit as well.
https://pagure.io/freeipa/issue/6796
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Made several improvements to coding style:
- same use of KerberosError throughout the module
- removed some unused variables
- moved code from try-except blocks if it didn't have to be there
- preparations for putting most of RPCClient.create_connection()
to loop
https://pagure.io/freeipa/issue/6796
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
This commit removes recursion in RPCClient.forward() which may lack
end condition.
https://pagure.io/freeipa/issue/6796
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
We slice down the received cookie so that just the content that matter
is preserved. Thi is ok because servers can't trust anything else anyway
and will accept a cookie with the ancillary data missing.
By removing variable parts like the expiry component added by
mod_session or the Expiration or Max-Age metadata we keep only the part
of the cookie that changes only when a new session is generated.
This way when storing the cookie we actually add a new entry in the
ccache only when the session actually changes, and this prevents churn
on FILE based ccaches.
Related https://pagure.io/freeipa/issue/6775
Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
If cookie authentication fails and we get back a 401 see if we
tried a SPNEGO auth by checking if we had a GSSAPI context. If not
it means our session cookie was invalid or expired or some other
error happened on the server that requires us to try a full SPNEGO
handshake, so go ahead and try it.
Fixes https://pagure.io/freeipa/issue/6775
Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Do not forcefully close the connection after every request. This enables
HTTP connection keep-alive, also known as persistent TCP and TLS/SSL
connection. Keep-alive speed up consecutive HTTP requests by 15% (for
local, low-latency network connections to a fast server) to multiple
times (high latency connections or remote peers).
https://pagure.io/freeipa/issue/6641
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Tomas Krizek <tkrizek@redhat.com>
Fix a crash caused by feeding incorrect data to `json.dumps()` in
`JSONServerProxy.__request()` introduced by commit
8159c2883b.
https://pagure.io/freeipa/issue/6734
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Instead of using the kernel keyring, store the session cookie within the
ccache. This way kdestroy will really wipe away all credentials.
Ticket: https://pagure.io/freeipa/issue/6661
Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
When cookie has Max-Age set it tries to get expiration by adding
to a timestamp. Without this patch the timestamp would be set to
None and thus the addition of timestamp + max_age fails
https://pagure.io/freeipa/issue/6718
Reviewed-By: Simo Sorce <ssorce@redhat.com>
NSSConnection was causing a lot of trouble in the past and there is
a lot of logic around it just to make it not fail. What's more,
when using NSS to create an SSL connection in FIPS mode, NSS
always requires database password which makes the `ipa` command
totally unusable.
NSSConnection is therefore replaced with Python's
httplib.HTTPSConnection which is OpenSSL based.
The HTTPSConnection is set up to handle authentication with client
certificate for connections to Dogtag server as RA agent. It allows
to handle client cert/private key in separate files and also
encrypted private key files.
https://fedorahosted.org/freeipa/ticket/5695
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Some tests assume that JSON deserializier returns tuples instead of
lists. I don't think it is necessary but let's pass the tests for now.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Improve performance of FreeIPA's JSON serializer and deserializer.
* Don't indent and sort keys. Both options trigger a slow path in
Python's json package. Without indention and sorting, encoding
mostly happens in optimized C code.
* Replace O(n) type checks with O(1) type lookup and eliminate
the use of isinstance().
* Check each client capability only once for every conversion.
* Use decoder's obj_hook feature to traverse the object tree once and
to eliminate calls to isinstance().
Closes: https://fedorahosted.org/freeipa/ticket/6655
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
If we are changing identiy (different principal) insure we remove the
session cookie stored on the rpc context so that we do not mistakenly
connect with the previous identity credentials.
https://fedorahosted.org/freeipa/ticket/6543
Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
This allows code to use multiple ccaches without having to muck with the
process global environment variables (KRB5CCNAME).
https://fedorahosted.org/freeipa/ticket/6543
Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
for Decimal only from client to server direction uses __base64__
notation. Server replies with pure string for Decimal data, and also
server is able to parse string and create decimal values where needed.
without this we need ugly py3 code:
- return {'__base64__': base64.b64encode(str(val))}
+ return {'__base64__': base64.b64encode(
+ str(val).encode('ascii')).decode('ascii')}
https://fedorahosted.org/freeipa/ticket/4985
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
In 'store_session_cookie', if the server does not set the session
cookie for some reason, the 'session_cookie' variable does not get
assigned, resulting in UnboundLocalError. Set an initial value of
'None'.
Fixes: https://fedorahosted.org/freeipa/ticket/6636
Reviewed-By: Martin Basti <mbasti@redhat.com>
In py 3.5 json.loads requires to have string as input, all bytes must be
decoded.
Note: python 3.6 supports bytes for json.loads()
https://fedorahosted.org/freeipa/ticket/4985
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Python3 removed the getheaders() function and replaced it with a
get_all() one. Add compat code.
https://fedorahosted.org/freeipa/ticket/6558
Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Sprinkle 'pylint disable' comments over the code base to silence a bunch
of pylint warnings on Python 3. All silenced warnings are harmless and
not bugs.
https://fedorahosted.org/freeipa/ticket/4985
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
- Use the correct unicode string for an error message, otherwise an
exception will generate another exception about incorrect type,
masking the original error.
Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Check for import errors with pylint to make sure new python package
dependencies are not overlooked.
https://fedorahosted.org/freeipa/ticket/6418
Reviewed-By: Petr Spacek <pspacek@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Unused variables may:
* make code less readable
* create dead code
* potentialy hide issues/errors
Enabled check should prevent to leave unused variable in code
Check is locally disabled for modules that fix is not clear or easy or have too many occurences of
unused variables
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
This patch introduces a separate Principal parameter that allows the framework
to syntactically validate incoming/outcoming principals by using a single
shared codebase.
https://fedorahosted.org/freeipa/ticket/3864
Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
The validation is already done on the server.
This allows manually forwarding commands unknown to the client but known
to the server.
https://fedorahosted.org/freeipa/ticket/4739
Reviewed-By: David Kupka <dkupka@redhat.com>
Speed up JSON-RPC response handling by putting received response data
fragments in a list and joining them at once instead of concatenating
each fragment one by one.
https://fedorahosted.org/freeipa/ticket/4739
Reviewed-By: David Kupka <dkupka@redhat.com>
