Commit Graph

144 Commits

Author SHA1 Message Date
Jan Cholasta
e056b8dea2 rpc: allow overriding NSS DB directory in API config
Add new `nss_dir` API config option to allow rpcclient to use a non-default
NSS DB for the connection.

https://fedorahosted.org/freeipa/ticket/4739

Reviewed-By: David Kupka <dkupka@redhat.com>
2016-06-03 09:00:34 +02:00
Jan Cholasta
43dc424041 rpc: respect API config in RPCClient.create_connection
When connecting rpcclient, get the default values of the `verbose`,
`fallback` and `delegate` options from API config rather than hard-code
them.

https://fedorahosted.org/freeipa/ticket/4739

Reviewed-By: David Kupka <dkupka@redhat.com>
2016-06-03 09:00:34 +02:00
Jan Cholasta
6cfb9d73d9 dns: move code shared by client and server to separate module
Move the shared code to a new ipalib.dns module.

https://fedorahosted.org/freeipa/ticket/4739

Reviewed-By: David Kupka <dkupka@redhat.com>
2016-06-03 09:00:34 +02:00
Jan Cholasta
e0275abe6f rpc: include structured error information in responses
Include keyword arguments of exceptions in RPC responses. This is limited
to JSON-RPC, as XML-RPC does not support additional data in error
responses.

Include keyword arguments of messages in RPC responses.

Include keyword arguments of exceptions in batch command result.

https://fedorahosted.org/freeipa/ticket/4739

Reviewed-By: David Kupka <dkupka@redhat.com>
2016-05-25 16:06:26 +02:00
Jan Cholasta
c37aba291f rpc: do not crash when unable to parse JSON
When unable to parse JSON response from the server, properly raise
JSONError not to cause a crash.

https://fedorahosted.org/freeipa/ticket/4739

Reviewed-By: David Kupka <dkupka@redhat.com>
2016-05-25 16:06:26 +02:00
Petr Viktorin
baaa041b8a ipalib.rpc: Send base64-encoded data as string under Python 3
Python 3's JSON library cannot deal with bytes, so decode
base64-encoded data to string.

Part of the work for https://fedorahosted.org/freeipa/ticket/4985

Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Petr Spacek <pspacek@redhat.com>
2016-05-05 16:42:46 +02:00
Michael Simacek
831856ea55 Fix bytes/string handling in rpc
https://fedorahosted.org/freeipa/ticket/5638

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2016-02-17 10:41:29 +01:00
Martin Basti
e4075b1fe2 Remove unused imports
This patch removes unused imports, alse pylint has been configured to
check unused imports.

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2015-12-23 07:59:22 +01:00
Martin Babinsky
82fd4250b9 silence pylint in Python 3-specific portion of ipalib/rpc.py
Reviewed-By: Tomas Babej <tbabej@redhat.com>
2015-10-27 18:27:19 +01:00
Petr Viktorin
c38516eab7 ipalib.rpc: Update for Python 3
The client XML-RPC implementation is tied to rpclib internals,
so with a change in Python it needs to be updated. And rpclib
changed in Python 3.

Reviewed-By: Tomas Babej <tbabej@redhat.com>
2015-10-27 17:23:25 +01:00
Petr Viktorin
6811b4be6a ipapython.nsslib: Remove NSSHTTPS
This workaround is unused in Python 2.7+.

Reviewed-By: Tomas Babej <tbabej@redhat.com>
2015-10-27 17:23:25 +01:00
Petr Viktorin
4ddd1821b6 ipapython.nsslib, ipalib.rpc: Remove code for Python 2.6 and below
IPA hasn't supported these pythons for a while now.

Reviewed-By: Tomas Babej <tbabej@redhat.com>
2015-10-27 17:23:25 +01:00
Petr Viktorin
8a2b65a357 Fix more bytes/unicode issues
Reviewed-By: Tomas Babej <tbabej@redhat.com>
2015-10-22 18:34:46 +02:00
Petr Viktorin
5f7c206e74 rpc: Name argument to KerberosError
Reviewed-By: Tomas Babej <tbabej@redhat.com>
2015-10-13 14:16:32 +02:00
Petr Viktorin
e3c05fcb73 Remove uses of the types module
In Python 3, the types module no longer provide alternate names for
built-in types, e.g. `types.StringType` can just be spelled `str`.

NoneType is also removed; it needs to be replaced with type(None)

Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2015-10-07 10:27:20 +02:00
Petr Viktorin
ad2bc94725 Use six.moves.xmlrpc.client instead of xmlrpclib
The module is renamed to xmlrpc.client in Python 3.

Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2015-10-07 10:27:20 +02:00
Petr Viktorin
70b37a956c Use six.moves.urllib instead of urllib/urllib2/urlparse
In Python 3, these modules are reorganized.

Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2015-10-07 10:27:20 +02:00
Petr Viktorin
c9ca8de7a2 rpc: Don't use undocumented urllib functions
The "splittype" and "splithost" functions in urllib.parse
are undocumented and reserved for internal use,
see http://bugs.python.org/issue11009

Use urlsplit instead.

Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
2015-10-07 10:27:20 +02:00
Michael Simacek
bdccebbcdb Rewrap errors in get_principal to CCacheError
Causes nicer error message when kerberos credentials are not available.

https://fedorahosted.org/freeipa/ticket/5272

Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Tomas Babej <tbabej@redhat.com>
2015-09-22 13:29:55 +02:00
Jan Cholasta
ba5201979d Use bytes instead of str where appropriate
Under Python 2, "str" and "bytes" are synonyms.

Reviewed-By: Petr Viktorin <pviktori@redhat.com>
2015-09-17 11:08:43 +02:00
Jan Cholasta
23507e6124 Alias "unicode" to "str" under Python 3
The six way of doing this is to replace all occurences of "unicode"
with "six.text_type". However, "unicode" is non-ambiguous and
(arguably) easier to read. Also, using it makes the patches smaller,
which should help with backporting.

Reviewed-By: Petr Viktorin <pviktori@redhat.com>
2015-09-17 11:08:43 +02:00
Petr Viktorin
fbacc26a6a Use six.integer_types instead of (long, int)
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2015-09-01 11:42:01 +02:00
Petr Viktorin
3bf91eab25 Use Python3-compatible dict method names
Python 2 has keys()/values()/items(), which return lists,
iterkeys()/itervalues()/iteritems(), which return iterators,
and viewkeys()/viewvalues()/viewitems() which return views.

Python 3 has only keys()/values()/items(), which return views.
To get iterators, one can use iter() or a for loop/comprehension;
for lists there's the list() constructor.

When iterating through the entire dict, without modifying the dict,
 the difference between Python 2's items() and iteritems() is
negligible, especially on small dicts (the main overhead is
extra memory, not CPU time). In the interest of simpler code,
this patch changes many instances of iteritems() to items(),
iterkeys() to keys() etc.

In other cases, helpers like six.itervalues are used.

Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2015-09-01 11:42:01 +02:00
Petr Viktorin
dd16cc98b0 Use six.string_types instead of "basestring"
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2015-09-01 11:42:01 +02:00
Michael Simacek
aad73fad60 Port from python-krbV to python-gssapi
python-krbV library is deprecated and doesn't work with python 3. Replacing all
it's usages with python-gssapi.

- Removed Backend.krb and KRB5_CCache classes
  They were wrappers around krbV classes that cannot really work without them
- Added few utility functions for querying GSSAPI credentials
  in krb_utils module. They provide replacements for KRB5_CCache.
- Merged two kinit_keytab functions
- Changed ldap plugin connection defaults to match ipaldap
- Unified getting default realm
  Using api.env.realm instead of krbV call

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Robbie Harwood <rharwood@redhat.com>
Reviewed-By: Simo Sorce <ssorce@redhat.com>
2015-08-26 09:41:36 +02:00
Petr Viktorin
27dabb4528 Modernize 'except' clauses
The 'as' syntax works from Python 2 on, and Python 3 will
drop the "comma" syntax.

Reviewed-By: Tomas Babej <tbabej@redhat.com>
2015-08-12 18:17:23 +02:00
Michael Simacek
f0b4c4487e Port from python-kerberos to python-gssapi
kerberos library doesn't support Python 3 and probably never will.
python-gssapi library is Python 3 compatible.

https://fedorahosted.org/freeipa/ticket/5147

Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Robbie Harwood <rharwood@redhat.com>
Reviewed-By: Simo Sorce <ssorce@redhat.com>
2015-08-05 08:08:00 +02:00
Rob Crittenden
57429c1cfa Don't rely on positional arguments for python-kerberos calls
Upstream PyKerberos uses a different argument ordering than
from the patch that Fedora/RHEL was carrying for
authGSSClientInit().

Using named arguments provides forwards and backwards
compatibility.

https://fedorahosted.org/freeipa/ticket/5085

Reviewed-By: Tomas Babej <tbabej@redhat.com>
2015-07-01 12:20:07 +02:00
Petr Vobornik
a18ef90284 rpcclient: use json_encode_binary for verbose output
`json.dumps` is not able to process some IPA's object types and therefore requires to preprocess it with `json_encode_binary` call. This step was not used in  rpcclient's verbose output.

https://fedorahosted.org/freeipa/ticket/4773

Reviewed-By: Martin Basti <mbasti@redhat.com>
2015-01-13 10:18:33 +01:00
Rob Crittenden
5c0ad221e8 Use NSS protocol range API to set available TLS protocols
Protocols are configured as an inclusive range from SSLv3 through
TLSv1.2. The allowed values in the range are ssl3, tls1.0,
tls1.1 and tls1.2.

This is overridable per client by setting tls_version_min and/or
tls_version_max.

https://fedorahosted.org/freeipa/ticket/4653

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2014-11-24 13:09:44 +00:00
Endi S. Dewata
80a8df3f19 Modififed NSSConnection not to shutdown existing database.
The NSSConnection class has been modified not to shutdown the
existing NSS database if the database is already opened to
establish an SSL connection, or is already opened by another
code that uses an NSS database without establishing an SSL
connection such as vault CLIs.

https://fedorahosted.org/freeipa/ticket/4638

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2014-11-11 09:09:19 +01:00
Jan Cholasta
231f57cedb Introduce NSS database /etc/ipa/nssdb
This is the new default NSS database for IPA.

/etc/pki/nssdb is still maintained for backward compatibility.

https://fedorahosted.org/freeipa/ticket/3259

Reviewed-By: Rob Crittenden <rcritten@redhat.com>
2014-09-30 10:01:38 +02:00
Petr Viktorin
b7a6d7937e JSON client: Log pretty-printed request and response with -vv or above
The whole HTTP request is now printed with -vvv or above.

Changes `verbose` in the connection to be the level from api.env,
rather than a boolean value.

For XML-RPC, the whole request will be shown already with -v.

https://fedorahosted.org/freeipa/ticket/4233

Reviewed-By: David Kupka <dkupka@redhat.com>
2014-09-24 13:57:56 +02:00
Jan Cholasta
eaebefe5f6 Allow overriding NSS database path in RPCClient.
Part of https://fedorahosted.org/freeipa/ticket/3259
Part of https://fedorahosted.org/freeipa/ticket/3520

Reviewed-By: Rob Crittenden <rcritten@redhat.com>
2014-07-30 16:04:21 +02:00
Tomas Babej
4d2ef43f28 ipaplatform: Move all filesystem paths to ipaplatform.paths module
https://fedorahosted.org/freeipa/ticket/4052

Reviewed-By: Petr Viktorin <pviktori@redhat.com>
2014-06-16 19:48:20 +02:00
Martin Basti
7625c02844 dns_name_values capability added
Added capability to transfer DNSName type between server and client

Part of ticket:
IPA should allow internationalized domain names
https://fedorahosted.org/freeipa/ticket/3169i

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2014-06-03 15:55:32 +02:00
Tomas Babej
1df696f543 ipalib: Add DateTime parameter
Adds a parameter that represents a DateTime format using datetime.datetime
object from python's native datetime library.

In the CLI, accepts one of the following formats:
    Accepts LDAP Generalized time without in the following format:
       '%Y%m%d%H%M%SZ'

    Accepts subset of values defined by ISO 8601:
        '%Y-%m-%dT%H:%M:%SZ'
        '%Y-%m-%dT%H:%MZ'
        '%Y-%m-%dZ'

    Also accepts above formats using ' ' (space) as a separator instead of 'T'.

As a simplification, it does not deal with timezone info and ISO 8601
values with timezone info (+-hhmm) are rejected. Values are expected
to be in the UTC timezone.

Values are saved to LDAP as LDAP Generalized time values in the format
'%Y%m%d%H%SZ' (no time fractions and UTC timezone is assumed). To avoid
confusion, in addition to subset of ISO 8601 values, the LDAP generalized
time in the format '%Y%m%d%H%M%SZ' is also accepted as an input (as this is the
format user will see on the output).

Part of: https://fedorahosted.org/freeipa/ticket/3306

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
2014-05-05 18:57:29 +03:00
Jan Cholasta
8b6dc819d5 Support API version-specific RPC marshalling.
Reviewed-By: Tomas Babej <tbabej@redhat.com>
2014-04-18 14:59:20 +02:00
Petr Viktorin
894b70a164 ipa tool: Print the name of the server we are connecting to with -v
The logging level for these messages was decreaed so that they
do not show up in ipa-advise output.
Reset the log level to INFO and configure ipa-advise to not display
INFO messages from xmlclient by default.

Partially reverts commit efe5a96725

https://fedorahosted.org/freeipa/ticket/4135

Reviewed-By: Tomáš Babej <tbabej@redhat.com>
2014-02-05 15:35:36 +01:00
Petr Viktorin
1e836d2d0c Switch client to JSON-RPC
Modify ipalib.rpc to support JSON-RPC in addition to XML-RPC.
This is done by subclassing and extending xmlrpclib, because
our existing code relies on xmlrpclib internals.

The URI to use is given in the new jsonrpc_uri env variable. When
it is not given, it is generated from xmlrpc_uri by replacing
/xml with /json.

The rpc_json_uri env variable existed before, but was unused,
undocumented and not set the install scripts.
This patch removes it in favor of jsonrpc_uri (for consistency
with xmlrpc_uri).

Add the rpc_protocol env variable to control the protocol
IPA uses. rpc_protocol defaults to 'jsonrpc', but may be changed
to 'xmlrpc'.
Make backend.Executioner and tests use the backend specified by
rpc_protocol.

For compatibility with unwrap_xml, decoding JSON now gives tuples
instead of lists.

Design: http://freeipa.org/page/V3/JSON-RPC
Ticket: https://fedorahosted.org/freeipa/ticket/3299
2013-11-26 16:59:59 +01:00
Petr Viktorin
15618beab6 Fix invalid assumption NSS initialization check in SSLTransport
There code assumes that the `conn` in any Connection in the context is
a ServerProxy. This might not always be the case: ldap2 uses a
python-ldap connection here.
2013-10-30 11:50:05 +01:00
Ana Krivokapic
efe5a96725 Enable running API commands in ipa-advise plugins
https://fedorahosted.org/freeipa/ticket/3671
https://fedorahosted.org/freeipa/ticket/3672
2013-08-07 09:18:43 +02:00
Martin Kosek
adc57707c0 Remove redundant u'' character
One Python's unicode marking character was being printed by RPC plugin
which then appeared in ipa-client-install output. This patch removes
it.
2013-06-06 08:36:04 +02:00
Simo Sorce
e8cb869046 Log info on failure to connect
When multiple servers are avilable we were simply suppressing information on
why a connection failed.
Log it as 'info' so that it is possible to diagnose issues more easily.
2012-12-19 09:50:43 +01:00
John Dennis
9269e5d6dd Compliant client side session cookie behavior
In summary this patch does:

* Follow the defined rules for cookies when:

  - receiving a cookie (process the attributes)

  - storing a cookie (store cookie + attributes)

  - sending a cookie

    + validate the cookie domain against the request URL

    + validate the cookie path against the request URL

    + validate the cookie expiration

    + if valid then send only the cookie, no attribtues

* Modifies how a request URL is stored during a XMLRPC
  request/response sequence.

* Refactors a bit of the request/response logic to allow for making
  the decision whether to send a session cookie instead of full
  Kerberous auth easier.

* The server now includes expiration information in the session cookie
  it sends to the client. The server always had the information
  available to prevent using an expired session cookie. Now that
  expiration timestamp is returned to the client as well and now the
  client will not send an expired session cookie back to the server.

* Adds a new module and unit test for cookies (see below)

Formerly we were always returning the session cookie no matter what
the domain or path was in the URL. We were also sending the cookie
attributes which are for the client only (used to determine if to
return a cookie). The attributes are not meant to be sent to the
server and the previous behavior was a protocol violation. We also
were not checking the cookie expiration.

Cookie library issues:

We need a library to create, parse, manipulate and format cookies both
in a client context and a server context. Core Python has two cookie
libraries, Cookie.py and cookielib.py. Why did we add a new cookie
module instead of using either of these two core Python libaries?

Cookie.py is designed for server side generation but can be used to
parse cookies on the client. It's the library we were using in the
server. However when I tried to use it in the client I discovered it
has some serious bugs. There are 7 defined cookie elements, it fails
to correctly parse 3 of the 7 elements which makes it unusable because
we depend on those elements. Since Cookie.py was designed for server
side cookie processing it's not hard to understand how fails to
correctly parse a cookie because that's a client side need. (Cookie.py
also has an awkward baroque API and is missing some useful
functionality we would have to build on top of it).

cookielib.py is designed for client side. It's fully featured and obeys
all the RFC's. It would be great to use however it's tightly coupled
with another core library, urllib2.py. The http request and response
objects must be urllib2 objects. But we don't use urllib2, rather we use
httplib because xmlrpclib uses httplib. I don't see a reason why a
cookie library should be so tightly coupled to a protocol library, but
it is and that means we can't use it (I tried to just pick some isolated
entrypoints for our use but I kept hitting interaction/dependency problems).

I decided to solve the cookie library problems by writing a minimal
cookie library that does what we need and no more than that. It is a
new module in ipapython shared by both client and server and comes
with a new unit test. The module has plenty of documentation, no need
to repeat it here.

Request URL issues:

We also had problems in rpc.py whereby information from the request
which is needed when we process the response is not available. Most
important was the requesting URL. It turns out that the way the class
and object relationships are structured it's impossible to get this
information. Someone else must have run into the same issue because
there was a routine called reconstruct_url() which attempted to
recreate the request URL from other available
information. Unfortunately reconstruct_url() was not callable from
inside the response handler. So I decided to store the information in
the thread context and when the request is received extract it from
the thread context. It's perhaps not an ideal solution but we do
similar things elsewhere so at least it's consistent. I removed the
reconstruct_url() function because the exact information is now in the
context and trying to apply heuristics to recreate the url is probably
not robust.

Ticket https://fedorahosted.org/freeipa/ticket/3022
2012-12-10 12:45:09 -05:00
Rob Crittenden
fb7575d6b7 Close connection after each request, avoid NSS shutdown problem.
The unit tests were failing when executed against an Apache server
in F-18 due to dangling references causing NSS shutdown to fail.

https://fedorahosted.org/freeipa/ticket/3180
2012-10-24 15:07:53 -04:00
Rob Crittenden
5bf1cee702 Clear kernel keyring in client installer, save dbdir on new connections
This patch addresses two issues:

1. If a client is previously enrolled in an IPA server and the server
   gets re-installed then the client machine may still have a keyring
   entry for the old server. This can cause a redirect from the
   session URI to the negotiate one. As a rule, always clear the keyring
   when enrolling a new client.

2. We save the NSS dbdir in the connection so that when creating a new
   session we can determine if we need to re-initialize NSS or not. Most
   of the time we do not. The dbdir was not always being preserved between
   connections which could cause an NSS_Shutdown() to happen which would
   fail because of existing usage. This preserves the dbdir information when
   a new connection is created as part of the session mechanism.

https://fedorahosted.org/freeipa/ticket/3108
2012-10-03 19:22:00 +02:00
Martin Kosek
cfbea2a99e Transfer long numbers over XMLRPC
Numeric parameters in ipalib were limited by XMLRPC boundaries for
integer (2^31-1) which is too low for some LDAP attributes like DNS
SOA serial field.

Transfer numbers which are not in XMLRPC boundary as a string and not
as a number to workaround this limitation. Int parameter had to be
updated to also accept Python's long type as valid int type.

https://fedorahosted.org/freeipa/ticket/2568
2012-09-06 14:57:07 +02:00
Petr Viktorin
a95eaeac8e Internationalization for public errors
Currently, we throw many public exceptions without proper i18n.
Wrap natural-language error messages in _() so they can be translated.

In the service plugin, raise NotFound errors using handle_not_found helper
so the error message contains the offending service.

Use ScriptError instead of NotFoundError in bindinstance install.

https://fedorahosted.org/freeipa/ticket/1953
2012-09-03 18:16:12 +02:00
John Dennis
94d457e83c Use DN objects instead of strings
* Convert every string specifying a DN into a DN object

* Every place a dn was manipulated in some fashion it was replaced by
  the use of DN operators

* Add new DNParam parameter type for parameters which are DN's

* DN objects are used 100% of the time throughout the entire data
  pipeline whenever something is logically a dn.

* Many classes now enforce DN usage for their attributes which are
  dn's. This is implmented via ipautil.dn_attribute_property(). The
  only permitted types for a class attribute specified to be a DN are
  either None or a DN object.

* Require that every place a dn is used it must be a DN object.
  This translates into lot of::

    assert isinstance(dn, DN)

  sprinkled through out the code. Maintaining these asserts is
  valuable to preserve DN type enforcement. The asserts can be
  disabled in production.

  The goal of 100% DN usage 100% of the time has been realized, these
  asserts are meant to preserve that.

  The asserts also proved valuable in detecting functions which did
  not obey their function signatures, such as the baseldap pre and
  post callbacks.

* Moved ipalib.dn to ipapython.dn because DN class is shared with all
  components, not just the server which uses ipalib.

* All API's now accept DN's natively, no need to convert to str (or
  unicode).

* Removed ipalib.encoder and encode/decode decorators. Type conversion
  is now explicitly performed in each IPASimpleLDAPObject method which
  emulates a ldap.SimpleLDAPObject method.

* Entity & Entry classes now utilize DN's

* Removed __getattr__ in Entity & Entity clases. There were two
  problems with it. It presented synthetic Python object attributes
  based on the current LDAP data it contained. There is no way to
  validate synthetic attributes using code checkers, you can't search
  the code to find LDAP attribute accesses (because synthetic
  attriutes look like Python attributes instead of LDAP data) and
  error handling is circumscribed. Secondly __getattr__ was hiding
  Python internal methods which broke class semantics.

* Replace use of methods inherited from ldap.SimpleLDAPObject via
  IPAdmin class with IPAdmin methods. Directly using inherited methods
  was causing us to bypass IPA logic. Mostly this meant replacing the
  use of search_s() with getEntry() or getList(). Similarly direct
  access of the LDAP data in classes using IPAdmin were replaced with
  calls to getValue() or getValues().

* Objects returned by ldap2.find_entries() are now compatible with
  either the python-ldap access methodology or the Entity/Entry access
  methodology.

* All ldap operations now funnel through the common
  IPASimpleLDAPObject giving us a single location where we interface
  to python-ldap and perform conversions.

* The above 4 modifications means we've greatly reduced the
  proliferation of multiple inconsistent ways to perform LDAP
  operations. We are well on the way to having a single API in IPA for
  doing LDAP (a long range goal).

* All certificate subject bases are now DN's

* DN objects were enhanced thusly:
  - find, rfind, index, rindex, replace and insert methods were added
  - AVA, RDN and DN classes were refactored in immutable and mutable
    variants, the mutable variants are EditableAVA, EditableRDN and
    EditableDN. By default we use the immutable variants preserving
    important semantics. To edit a DN cast it to an EditableDN and
    cast it back to DN when done editing. These issues are fully
    described in other documentation.
  - first_key_match was removed
  - DN equalty comparison permits comparison to a basestring

* Fixed ldapupdate to work with DN's. This work included:
  - Enhance test_updates.py to do more checking after applying
    update. Add test for update_from_dict(). Convert code to use
    unittest classes.
  - Consolidated duplicate code.
  - Moved code which should have been in the class into the class.
  - Fix the handling of the 'deleteentry' update action. It's no longer
    necessary to supply fake attributes to make it work. Detect case
    where subsequent update applies a change to entry previously marked
    for deletetion. General clean-up and simplification of the
    'deleteentry' logic.
  - Rewrote a couple of functions to be clearer and more Pythonic.
  - Added documentation on the data structure being used.
  - Simplfy the use of update_from_dict()

* Removed all usage of get_schema() which was being called prior to
  accessing the .schema attribute of an object. If a class is using
  internal lazy loading as an optimization it's not right to require
  users of the interface to be aware of internal
  optimization's. schema is now a property and when the schema
  property is accessed it calls a private internal method to perform
  the lazy loading.

* Added SchemaCache class to cache the schema's from individual
  servers. This was done because of the observation we talk to
  different LDAP servers, each of which may have it's own
  schema. Previously we globally cached the schema from the first
  server we connected to and returned that schema in all contexts. The
  cache includes controls to invalidate it thus forcing a schema
  refresh.

* Schema caching is now senstive to the run time context. During
  install and upgrade the schema can change leading to errors due to
  out-of-date cached schema. The schema cache is refreshed in these
  contexts.

* We are aware of the LDAP syntax of all LDAP attributes. Every
  attribute returned from an LDAP operation is passed through a
  central table look-up based on it's LDAP syntax. The table key is
  the LDAP syntax it's value is a Python callable that returns a
  Python object matching the LDAP syntax. There are a handful of LDAP
  attributes whose syntax is historically incorrect
  (e.g. DistguishedNames that are defined as DirectoryStrings). The
  table driven conversion mechanism is augmented with a table of
  hard coded exceptions.

  Currently only the following conversions occur via the table:

  - dn's are converted to DN objects

  - binary objects are converted to Python str objects (IPA
    convention).

  - everything else is converted to unicode using UTF-8 decoding (IPA
    convention).

  However, now that the table driven conversion mechanism is in place
  it would be trivial to do things such as converting attributes
  which have LDAP integer syntax into a Python integer, etc.

* Expected values in the unit tests which are a DN no longer need to
  use lambda expressions to promote the returned value to a DN for
  equality comparison. The return value is automatically promoted to
  a DN. The lambda expressions have been removed making the code much
  simpler and easier to read.

* Add class level logging to a number of classes which did not support
  logging, less need for use of root_logger.

* Remove ipaserver/conn.py, it was unused.

* Consolidated duplicate code wherever it was found.

* Fixed many places that used string concatenation to form a new
  string rather than string formatting operators. This is necessary
  because string formatting converts it's arguments to a string prior
  to building the result string. You can't concatenate a string and a
  non-string.

* Simplify logic in rename_managed plugin. Use DN operators to edit
  dn's.

* The live version of ipa-ldap-updater did not generate a log file.
  The offline version did, now both do.

https://fedorahosted.org/freeipa/ticket/1670
https://fedorahosted.org/freeipa/ticket/1671
https://fedorahosted.org/freeipa/ticket/1672
https://fedorahosted.org/freeipa/ticket/1673
https://fedorahosted.org/freeipa/ticket/1674
https://fedorahosted.org/freeipa/ticket/1392
https://fedorahosted.org/freeipa/ticket/2872
2012-08-12 16:23:24 -04:00
Rob Crittenden
429edcfb72 Support per-principal sessions and handle session update failures
User had a system that refused to store keys into the kernel keyring.
Any operation at all on the keyring would return "Key has been revoked".

Wrap the operations in a try/except so we can ignore keyring failures.

This also adds per-principal sessions. The principal name is stored
in the session key so switching principals in the ccache doesn't
require clearing the keyring.

https://fedorahosted.org/freeipa/ticket/2880
2012-07-18 16:05:08 -04:00
Rob Crittenden
54135ecd9a Store session cookie in ccache for cli users
Try to use the URI /ipa/session/xml if there is a key in the kernel
keyring. If there is no cookie or it turns out to be invalid (expired,
whatever) then use the standard URI /ipa/xml. This in turn will create
a session that the user can then use later.

https://fedorahosted.org/freeipa/ticket/2331
2012-06-14 14:02:26 +02:00
Martin Kosek
f1ed123cad Replace DNS client based on acutil with python-dns
IPA client and server tool set used authconfig acutil module to
for client DNS operations. This is not optimal DNS interface for
several reasons:
- does not provide native Python object oriented interface
  but but rather C-like interface based on functions and
  structures which is not easy to use and extend
- acutil is not meant to be used by third parties besides
  authconfig and thus can break without notice

Replace the acutil with python-dns package which has a feature rich
interface for dealing with all different aspects of DNS including
DNSSEC. The main target of this patch is to replace all uses of
acutil DNS library with a use python-dns. In most cases, even
though the larger parts of the code are changed, the actual
functionality is changed only in the following cases:
- redundant DNS checks were removed from verify_fqdn function
  in installutils to make the whole DNS check simpler and
  less error-prone. Logging was improves for the remaining
  checks
- improved logging for ipa-client-install DNS discovery

https://fedorahosted.org/freeipa/ticket/2730
https://fedorahosted.org/freeipa/ticket/1837
2012-05-24 13:55:56 +02:00
Rob Crittenden
17ba58aa4b Don't set dbdir in the connection until after the connection is created.
We were comparing the current connection with itself so were never
going to call nss_shutdown(). dbdir needs to be set after the connection
has been made.

This worked on single server installs because we don't do a ping so
NSS would never be pre-initialized. If multiple servers are available we
call ping() to find one that is up before submitting the request, this is
what would have pre-initialized NSS.

This was tripping up request-cert because it will intialize NSS with no DB
if it hasn't been initialized. We need to initialize it to validate the
CSR.

A non-working client was doing this when calling cert-request:
 - call load_certificate_request()
 - nss.nss_nodb_init()
 - load the CSR
 - create a connection, dbdir=/etc/pki/nssdb
 - the dbdir matches within the same connection, don't call nss_shutdown()
 - connect to remote server
 - fail, untrusted CA because we are still using db from nss_nodb_init.

Instead if we set dbdir afterward then this will properly be shutdown
and NSS re-initialized with correct dbdir.

https://fedorahosted.org/freeipa/ticket/2498
2012-03-13 22:42:12 -04:00
Rob Crittenden
55f89dc689 Do kinit in client before connecting to backend
The client installer was failing because a backend connection could be
created before a kinit was done.

Allow multiple simultaneous connections. This could fail with an NSS
shutdown error when the second connection was created (objects still
in use). If all connections currently use the same database then there
is no need to initialize, let it be skipped.

Add additional logging to client installer.

https://fedorahosted.org/freeipa/ticket/2478
2012-03-04 17:23:01 -05:00
Rob Crittenden
2da6d6e746 Don't set delegation flag in client, we're using S4U2Proxy now
A forwardable ticket is still required but we no longer need to send
the TGT to the IPA server. A new flag, --delegate, is available if
the old behavior is required.

Set the minimum n-v-r for mod_auth_kerb and krb5-server to pick up
needed patches for S4U2Proxy to work.

https://fedorahosted.org/freeipa/ticket/1098
https://fedorahosted.org/freeipa/ticket/2246
2012-02-15 17:08:33 +01:00
John Dennis
bba4ccb3a0 add session manager and cache krb auth
This patch adds a session manager and support for caching
authentication in the session. Major elements of the patch are:

* Add a session manager to support cookie based sessions which
  stores session data in a memcached entry.

* Add ipalib/krb_utils.py which contains functions to parse ccache
  names, format principals, format KRB timestamps, and a KRB_CCache
  class which reads ccache entry and allows one to extract information
  such as the principal, credentials, credential timestamps, etc.

* Move krb constants defined in ipalib/rpc.py to ipa_krb_utils.py so
  that all kerberos items are co-located.

* Modify javascript in ipa.js so that the IPA.command() RPC call
  checks for authentication needed error response and if it receives
  it sends a GET request to /ipa/login URL to refresh credentials.

* Add session_auth_duration config item to constants.py, used to
  configure how long a session remains valid.

* Add parse_time_duration utility to ipalib/util.py. Used to parse the
  session_auth_duration config item.

* Update the default.conf.5 man page to document session_auth_duration
  config item (also added documentation for log_manager config items
  which had been inadvertantly omitted from a previous commit).

* Add SessionError object to ipalib/errors.py

* Move Kerberos protection in Apache config from /ipa to /ipa/xml and
  /ipa/login

* Add SessionCCache class to session.py to manage temporary Kerberos
  ccache file in effect for the duration of an RPC command.

* Adds a krblogin plugin used to implement the /ipa/login
  handler. login handler sets the session expiration time, currently
  60 minutes or the expiration of the TGT, whichever is shorter. It
  also copies the ccache provied by mod_auth_kerb into the session
  data.  The json handler will later extract and validate the ccache
  belonging to the session.

* Refactored the WSGI handlers so that json and xlmrpc could have
  independent behavior, this also moves where create and destroy
  context occurs, now done in the individual handler rather than the
  parent class.

* The json handler now looks up the session data, validates the ccache
  bound to the session, if it's expired replies with authenicated
  needed error.

* Add documentation to session.py. Fully documents the entire process,
  got questions, read the doc.

* Add exclusions to make-lint as needed.
2012-02-09 13:20:45 -06:00
Martin Kosek
092dd8db12 Replace float with Decimal
Having float type as a base type for floating point parameters in
ipalib introduces several issues, e.g. problem with representation
or value comparison. Python language provides a Decimal type which
help overcome these issues.

This patch replaces a float type and Float parameter with a
decimal.Decimal type in Decimal parameter. A precision attribute
was added to Decimal parameter that can be used to limit a number
of decimal places in parameter representation. This approach fixes
a problem with API.txt validation where comparison of float values
may fail on different architectures due to float representation error.

In order to safely transfer the parameter value over RPC it is
being converted to string which is then converted back to
decimal.Decimal number on a server side.

https://fedorahosted.org/freeipa/ticket/2260
2012-01-20 08:13:44 +01:00
Rob Crittenden
2d6eeb205e Require an HTTP Referer header in the server. Send one in ipa tools.
This is to prevent a Cross-Site Request Forgery (CSRF) attack where
a rogue server tricks a user who was logged into the FreeIPA
management interface into visiting a specially-crafted URL where
the attacker could perform FreeIPA oonfiguration changes with the
privileges of the logged-in user.

https://bugzilla.redhat.com/show_bug.cgi?id=747710
2011-12-12 17:36:45 -05:00
Jan Cholasta
12bfed37d4 Add a function for formatting network locations of the form host:port for use in URLs.
If the host part is a literal IPv6 address, it must be enclosed in square
brackets (RFC 2732).

ticket 1869
2011-10-05 10:58:25 +02:00
Martin Kosek
ffd760c100 Fix /usr/bin/ipa dupled server list
Fix get_url_list() so that the configured master server is there
just once. This fix lets /usr/bin/ipa try connecting to all IPA
masters just once and not print confusing server list with
dupled master.

https://fedorahosted.org/freeipa/ticket/1817
2011-09-22 15:08:32 +02:00
Rob Crittenden
4027b12371 Test for forwarded Kerberos credentials cache in wsgi code.
We should more gracefully handle if the TGT has not been forwarded
than returning a 500 error.

Also catch and display KerberosErrors from ping() in the client better.

ticket 1101
2011-05-18 09:35:04 +02:00
Pavel Zuna
bbc94034b0 Fallback to default locale (en_US) if env. setting is corrupt. 2011-03-01 10:31:37 -05:00
Pavel Zuna
bfca99b420 Send Accept-Language header over XML-RPC and translate on server.
Fix #904
Fix #917
2011-03-01 10:31:37 -05:00
Rob Crittenden
886e417fd7 Set the default Int maxvalue to the maximum XML-RPC can handle.
Also handle marshalling errors thrown by xmlrpclib more gracefully.

ticket 770
2011-01-18 10:03:55 -05:00
Jakub Hrozek
7493d781df Change FreeIPA license to GPLv3+
The changes include:
 * Change license blobs in source files to mention GPLv3+ not GPLv2 only
 * Add GPLv3+ license text
 * Package COPYING not LICENSE as the license blobs (even the old ones)
   mention COPYING specifically, it is also more common, I think

 https://fedorahosted.org/freeipa/ticket/239
2010-12-20 17:19:53 -05:00
Rob Crittenden
264413bcb9 Handle regular socket errors gracefully in ipa command
ticket 382
2010-10-15 15:02:18 -04:00
Rob Crittenden
d0ea0bb638 Changes to fix compatibility with Fedora 14
Fedora 14 introduced the following incompatiblities:
- the kerberos binaries moved from /usr/kerberos/[s]/bin to /usr/[s]bin
- the xmlrpclib in Python 2.7 is not fully backwards compatible to 2.6

Also, when moving the installed host service principals:
- don't assume that krbticketflags is set
- allow multiple values for krbextradata

ticket 155
2010-08-31 16:59:27 -04:00
Rob Crittenden
1df10a88cd Add support for client failover to the ipa command-line.
This adds a new global option to the ipa command, -f/--no-fallback. If this
is included then just the server configured in /etc/ipa/default.conf is used.
Otherwise that is tried first then all servers in DNS with the ldap SRV record
are tried.

Create a new Local() Command class for local-only commands. The help
command is one of these. It shouldn't need a remote connection to execute.

ticket #15
2010-08-16 10:35:27 -04:00
John Dennis
31027c6183 use NSS for SSL operations 2010-06-15 15:03:36 -04:00
Rob Crittenden
1dd7b11b0b Connect the -v cli argument to the verbose flag in xmlrpclib
If you pass two -v to the ipa command you'll get the XML-RPC data in
the output. This can be handy so you know exactly what went out over
the wire.
2010-06-03 17:08:22 -04:00
Jason Gerard DeRose
918721c1d0 XML-RPC signature change 2010-03-30 15:10:58 -04:00
Rob Crittenden
bb09db2228 Explicitly set verbosity off in the XML-RPC client
This is so I don't have to hunt for where to set this to True when doing
low-level client debugging.
2009-09-14 09:46:42 -04:00
Rob Crittenden
904e555404 Catch and handle HTTP exceptions (like 401, 404, etc) 2009-07-15 08:59:58 -04:00
rcrit
9352d2fc10 Add a local implementation of httplib.SSLFile and httplib.FakeSocket
Python 2.6 changed its internal implementation which makes it difficult
to override in a way that is backwards compatible.

508953
2009-07-01 08:51:23 -04:00
Pavel Zuna
7d0bd4b895 Rename errors2.py to errors.py. Modify all affected files. 2009-04-23 10:29:14 -04:00
Rob Crittenden
a9387b48e6 Handle GSSAPI exceptions more gracefully 2009-04-20 13:44:08 -04:00
Rob Crittenden
f2abe05398 Use OpenSSL for SSL instead of the built-in python version. 2009-02-20 10:40:54 -05:00
Rob Crittenden
2d7e0de5ea Get merged tree into an installalble state.
I have only tested the all, rpms and *clean targets directly.
install may work but the rpm moves a lot of things around for us.

The Apache configuration file isn't in its final state but it works
with the new mod_python configuration.
2009-02-03 15:29:20 -05:00
Jason Gerard DeRose
3274577cd6 Finished small tweaks to get new ipaserver.xmlrpc() mod_python handler working 2009-02-03 15:29:05 -05:00
Jason Gerard DeRose
77e782e2cb Fixed bug I introduced in KerbTransport; started work on lite-xmlrpc.py to add client address to context 2009-02-03 15:29:04 -05:00
Jason Gerard DeRose
a0aebd46a1 Got new ldap connection working using Connectible.connect() 2009-02-03 15:29:04 -05:00
Jason Gerard DeRose
66b6029e40 Ported xmlclient to subclass from Connectible 2009-02-03 15:29:03 -05:00
Jason Gerard DeRose
0a3ae60038 Ported xmlserver to subclass from Executioner 2009-02-03 15:29:03 -05:00
Jason Gerard DeRose
e0b00d5981 Removed depreciated import of errors in frontend.py 2009-02-03 15:29:02 -05:00
Jason Gerard DeRose
0313f03277 Added ServiceError (KerberosError) and make rpc.KerbTransport raise it if appropriate 2009-02-03 15:29:02 -05:00
Jason Gerard DeRose
833088955c More xmlrpc tweaks: xmlserver.execute() now logs non-public exceptions; xmlclient.forward() now handles socket error; fixed some Python 2.4 problems in lite-xmlrpc2.py 2009-02-03 15:29:01 -05:00
Jason Gerard DeRose
24b6cb89d4 Further migration toward new xmlrcp code; fixed problem with unicode Fault.faultString; fixed problem where ServerProxy method was not called correctly 2009-02-03 15:29:01 -05:00
Jason Gerard DeRose
9f48612a56 Sundry work getting ready to switch to new XML-RPC client/server code 2009-02-03 15:29:00 -05:00
Jason Gerard DeRose
55fba5420d Added rpc.xmlclient backend plugin for forwarding; added corresponding unit tests 2009-01-19 21:10:42 -07:00
Jason Gerard DeRose
462bac3c13 Added docstring cross-references between rpc and rpcserver modules 2009-01-16 01:56:39 -07:00
Jason Gerard DeRose
0227a12949 Added docstrings to the new rpc.xml_dumps() and rcp.xml_loads() functions 2009-01-16 01:23:55 -07:00
Jason Gerard DeRose
6be5e4a0a5 ipalib.rpc: now using allow_none=True after conversation with Rob; added xml_dumps() and xml_loads() functions; some name cleanup 2009-01-15 23:52:50 -07:00
Jason Gerard DeRose
237c16f0fd Started moving xmlrpc-functions from ipalib.util to ipalib.rpc 2008-11-24 12:51:03 -07:00