Python dns resolver append configured domain to queries which may lead
to false positive answer.
Exmaple: resolving "ipa.example.com" may return records for
"ipa.example.com.example.com" if domain is configured as "example.com"
https://fedorahosted.org/freeipa/ticket/5421
Reviewed-By: Petr Spacek <pspacek@redhat.com>
If the code within the private_ccache contextmanager does not
set/removes the KRB5CCNAME, the pop method will raise KeyError, which
will cause unnecessary termination of the code flow.
Make sure the KRB5CCNAME is popped out of os.environ only if present.
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
In Python 3, the types module no longer provide alternate names for
built-in types, e.g. `types.StringType` can just be spelled `str`.
NoneType is also removed; it needs to be replaced with type(None)
Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
The module is renamed to xmlrpc.client in Python 3.
Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
In Python 3, these modules are reorganized.
Reviewed-By: David Kupka <dkupka@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
During the migration from winsync replicated users to their
trusted identities, memberships are being preserved. However,
trusted users are external and as such cannot be added as
direct members to the IPA entities. External groups which
encapsulate the migrated users are added as members to those
entities instead.
The name of the external group is generated from the type
of the entity and its name. However, the entity's name can
contain characters which are invalid for use in the group
name.
Adds a helper function to convert a given string to a string
which would be valid for such use and leverages it in the
winsync-migrate tool.
https://fedorahosted.org/freeipa/ticket/5319
Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
This mimics Python 3's behavior, where sys.argv is automatically decoded
using file system encoding, as returned by sys.getfilesystemencoding(). This
includes reimplementation of os.fsdecode() from Python 3.
Reviewed-By: Petr Viktorin <pviktori@redhat.com>
In Python 3, `print` is no longer a statement. Call it as a function
everywhere, and include the future import to remove the statement
in Python 2 code as well.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
In Python 3, raw_input() was renamed to input().
Import the function from six.moves to get the right version.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Python 2 has keys()/values()/items(), which return lists,
iterkeys()/itervalues()/iteritems(), which return iterators,
and viewkeys()/viewvalues()/viewitems() which return views.
Python 3 has only keys()/values()/items(), which return views.
To get iterators, one can use iter() or a for loop/comprehension;
for lists there's the list() constructor.
When iterating through the entire dict, without modifying the dict,
the difference between Python 2's items() and iteritems() is
negligible, especially on small dicts (the main overhead is
extra memory, not CPU time). In the interest of simpler code,
this patch changes many instances of iteritems() to items(),
iterkeys() to keys() etc.
In other cases, helpers like six.itervalues are used.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
python-krbV library is deprecated and doesn't work with python 3. Replacing all
it's usages with python-gssapi.
- Removed Backend.krb and KRB5_CCache classes
They were wrappers around krbV classes that cannot really work without them
- Added few utility functions for querying GSSAPI credentials
in krb_utils module. They provide replacements for KRB5_CCache.
- Merged two kinit_keytab functions
- Changed ldap plugin connection defaults to match ipaldap
- Unified getting default realm
Using api.env.realm instead of krbV call
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Robbie Harwood <rharwood@redhat.com>
Reviewed-By: Simo Sorce <ssorce@redhat.com>
Added error handling for function user_input in order to
handle EOFError in ipautil.py
https://fedorahosted.org/freeipa/ticket/3406
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
The deprecated has_key method will be removed from dicts in Python 3.
For custom dict-like classes, has_key() is kept on Python 2,
but disabled for Python 3.
Reviewed-By: Tomas Babej <tbabej@redhat.com>
kerberos library doesn't support Python 3 and probably never will.
python-gssapi library is Python 3 compatible.
https://fedorahosted.org/freeipa/ticket/5147
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Robbie Harwood <rharwood@redhat.com>
Reviewed-By: Simo Sorce <ssorce@redhat.com>
Param suplementary_groups allows to specify list of group names to be
used for subprocess.
suplementary_groups param requires runas param to be specified.
Required for ticket: https://fedorahosted.org/freeipa/ticket/4657
Reviewed-By: Petr Spacek <pspacek@redhat.com>
kinit_keytab replaces kinit_hostprincipal and performs Kerberos auth using
keytab file. Function is also able to repeat authentication multiple times
before giving up and raising Krb5Error.
kinit_password wraps kinit auth using password and also supports FAST
authentication using httpd armor ccache.
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Simo Sorce <ssorce@redhat.com>
Reviewed-By: Petr Spacek <pspacek@redhat.com>
We use ntpd now to sync time before fetching a TGT during client
install. Unfortuantely, ntpd will hang forever if it is unable to
reach the NTP server.
This patch adds the ability for commands run via ipautil.run() to
have an optional timeout. This capability is used by the NTP sync
code that is run during ipa-client-install.
Ticket: https://fedorahosted.org/freeipa/ticket/4842
Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
When api.env is loaded, strings that "look like" floats got
auto-converted to floats.
This is wrong, as the conversion to float can lose precision.
Case in point: the api_version (e.g. '2.88') should never be
interpreted as float.
Do not automatically convert to float.
We have two numeric options: startup_timeout and wait_for_dns.
wait_for_dns is already converted to int when used in the code.
Convert startup_timeout to float explicitly when used, so
configuration that specified it with a decimal point continues
to work.
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Run function can now run the specified command as different user by
setting the both real and effective UID and GID for executed process.
Add both the missing run function attribute doc strings as well as
a doc string for the runas attribute.
Updating a CIDict with data like {'A': 1, 'a': 2} would lead to data
loss since only one of the items would get to the CIDict.
This can result in non-obvious bugs similar to this one in python-ldap:
https://bugzilla.redhat.com/show_bug.cgi?id=1007820
Raise an error in this case; any resolution must be done by the caller.
Make the CIDict interface match standard dict (except view* methods).
Add __contains__, __iter__, clear.
Add keyword and iterable support for __init__, update.
Also add values() and itervalues(). Previously the dict versions were
used; the new ones guarantee that the order matches keys().
Mark view* methods as not implemented.
CIDict.copy() now returns a CIDict.
Test the above additions, and fromkeys() which worked but wasn't tested.
These used ipautil.get_ipa_basedn. Convert that to use the new wrappers.
Beef up the error handling in ipaldap to accomodate the errors we catch
in the server discovery.
Add a DatabaseTimeout exception to errors.py.
These were the last uses of ipautil.convert_ldap_error, remove that.
https://fedorahosted.org/freeipa/ticket/3487https://fedorahosted.org/freeipa/ticket/3446
Add more dynamic attribute info to IPATypeChecker in make-lint. Remove
unnecessary pylint comments. Fix false positivies introduced by Pylint 0.26.
https://fedorahosted.org/freeipa/ticket/3379
Major changes ipa-client-install:
* Use GSSAPI connection to LDAP server to download CA cert (now
the default method)
* Add --ca-cert-file option to load the CA cert from a disk file.
Validate the file. If this option is used the supplied CA cert
is considered definitive.
* The insecure HTTP retrieval method is still supported but it must be
explicitly forced and a warning will be emitted.
* Remain backward compatible with unattended case (except for aberrant
condition when preexisting /etc/ipa/ca.crt differs from securely
obtained CA cert, see below)
* If /etc/ipa/ca.crt CA cert preexists the validate it matches the
securely acquired CA cert, if not:
- If --unattended and not --force abort with error
- If interactive query user to accept new CA cert, if not abort
In either case warn user.
* If interactive and LDAP retrieval fails prompt user if they want to
proceed with insecure HTTP method
* If not interactive and LDAP retrieval fails abort unless --force
* Backup preexisting /etc/ipa/ca.crt in FileStore prior to execution,
if ipa-client-install fails it will be restored.
Other changes:
* Add new exception class CertificateInvalidError
* Add utility convert_ldap_error() to ipalib.ipautil
* Replace all hardcoded instances of /etc/ipa/ca.crt in
ipa-client-install with CACERT constant (matches existing practice
elsewhere).
* ipadiscovery no longer retrieves CA cert via HTTP.
* Handle LDAP minssf failures during discovery, treat failure to check
ldap server as a warninbg in absebce of a provided CA certificate via
--ca-cert-file or though existing /etc/ipa/ca.crt file.
Signed-off-by: Simo Sorce <simo@redhat.com>
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
When executing ipa-replica-manage connect to an master that raises
NotFound error we now check if the master is at least IPA server.
If so, we inform the user that it is probably foreign or previously
deleted master. If not, we inform the user that the master is not
an IPA server at all.
https://fedorahosted.org/freeipa/ticket/3105
When the user interrupts a long-running command, this ensures that
the command is logged. Also, when watching log files (or the -d
output), it's apparent what's being done.
https://fedorahosted.org/freeipa/ticket/3174
A hotfix pushed in a scope of ticket 3088 forced conversion of DN
object (baseDN) in IPA client discovery so that ipa-client-install
does not crash when creating an IPA default.conf. Since this is not
a preferred way to handle DN objects, improve its usage:
- make sure, that baseDN retrieved by client discovery is always
a DN object
- update ipachangeconf.py code to handle strings better and instead
of concatenating objects, make sure they are converted to string
first
As a side-effect of ipachangeconf changes, default.conf config file
generated by ipa-client-install has no longer empty new line at the
end of a file.
Whole ipachangeconf.py has been modified to be compliant with PEP8.
https://fedorahosted.org/freeipa/ticket/3088
Put the changes from Ade's dogtag 10 patch into namespaced constants in
dogtag.py, which are then referenced in the code.
Make ipaserver.install.CAInstance use the service name specified in the
configuration. Uninstallation, where config is removed before CA uninstall,
also uses the (previously) configured value.
This and Ade's patch address https://fedorahosted.org/freeipa/ticket/2846
Public keys in the old format (raw RFC 4253 blob) are automatically
converted to OpenSSH-style public keys. OpenSSH-style public keys are now
stored in LDAP.
Changed sshpubkeyfp to be an output parameter, as that is what it actually
is.
Allow parameter normalizers to be used on values of any type, not just
unicode, so that public key blobs (which are str) can be normalized to
OpenSSH-style public keys.
ticket 2932, 2935
* Convert every string specifying a DN into a DN object
* Every place a dn was manipulated in some fashion it was replaced by
the use of DN operators
* Add new DNParam parameter type for parameters which are DN's
* DN objects are used 100% of the time throughout the entire data
pipeline whenever something is logically a dn.
* Many classes now enforce DN usage for their attributes which are
dn's. This is implmented via ipautil.dn_attribute_property(). The
only permitted types for a class attribute specified to be a DN are
either None or a DN object.
* Require that every place a dn is used it must be a DN object.
This translates into lot of::
assert isinstance(dn, DN)
sprinkled through out the code. Maintaining these asserts is
valuable to preserve DN type enforcement. The asserts can be
disabled in production.
The goal of 100% DN usage 100% of the time has been realized, these
asserts are meant to preserve that.
The asserts also proved valuable in detecting functions which did
not obey their function signatures, such as the baseldap pre and
post callbacks.
* Moved ipalib.dn to ipapython.dn because DN class is shared with all
components, not just the server which uses ipalib.
* All API's now accept DN's natively, no need to convert to str (or
unicode).
* Removed ipalib.encoder and encode/decode decorators. Type conversion
is now explicitly performed in each IPASimpleLDAPObject method which
emulates a ldap.SimpleLDAPObject method.
* Entity & Entry classes now utilize DN's
* Removed __getattr__ in Entity & Entity clases. There were two
problems with it. It presented synthetic Python object attributes
based on the current LDAP data it contained. There is no way to
validate synthetic attributes using code checkers, you can't search
the code to find LDAP attribute accesses (because synthetic
attriutes look like Python attributes instead of LDAP data) and
error handling is circumscribed. Secondly __getattr__ was hiding
Python internal methods which broke class semantics.
* Replace use of methods inherited from ldap.SimpleLDAPObject via
IPAdmin class with IPAdmin methods. Directly using inherited methods
was causing us to bypass IPA logic. Mostly this meant replacing the
use of search_s() with getEntry() or getList(). Similarly direct
access of the LDAP data in classes using IPAdmin were replaced with
calls to getValue() or getValues().
* Objects returned by ldap2.find_entries() are now compatible with
either the python-ldap access methodology or the Entity/Entry access
methodology.
* All ldap operations now funnel through the common
IPASimpleLDAPObject giving us a single location where we interface
to python-ldap and perform conversions.
* The above 4 modifications means we've greatly reduced the
proliferation of multiple inconsistent ways to perform LDAP
operations. We are well on the way to having a single API in IPA for
doing LDAP (a long range goal).
* All certificate subject bases are now DN's
* DN objects were enhanced thusly:
- find, rfind, index, rindex, replace and insert methods were added
- AVA, RDN and DN classes were refactored in immutable and mutable
variants, the mutable variants are EditableAVA, EditableRDN and
EditableDN. By default we use the immutable variants preserving
important semantics. To edit a DN cast it to an EditableDN and
cast it back to DN when done editing. These issues are fully
described in other documentation.
- first_key_match was removed
- DN equalty comparison permits comparison to a basestring
* Fixed ldapupdate to work with DN's. This work included:
- Enhance test_updates.py to do more checking after applying
update. Add test for update_from_dict(). Convert code to use
unittest classes.
- Consolidated duplicate code.
- Moved code which should have been in the class into the class.
- Fix the handling of the 'deleteentry' update action. It's no longer
necessary to supply fake attributes to make it work. Detect case
where subsequent update applies a change to entry previously marked
for deletetion. General clean-up and simplification of the
'deleteentry' logic.
- Rewrote a couple of functions to be clearer and more Pythonic.
- Added documentation on the data structure being used.
- Simplfy the use of update_from_dict()
* Removed all usage of get_schema() which was being called prior to
accessing the .schema attribute of an object. If a class is using
internal lazy loading as an optimization it's not right to require
users of the interface to be aware of internal
optimization's. schema is now a property and when the schema
property is accessed it calls a private internal method to perform
the lazy loading.
* Added SchemaCache class to cache the schema's from individual
servers. This was done because of the observation we talk to
different LDAP servers, each of which may have it's own
schema. Previously we globally cached the schema from the first
server we connected to and returned that schema in all contexts. The
cache includes controls to invalidate it thus forcing a schema
refresh.
* Schema caching is now senstive to the run time context. During
install and upgrade the schema can change leading to errors due to
out-of-date cached schema. The schema cache is refreshed in these
contexts.
* We are aware of the LDAP syntax of all LDAP attributes. Every
attribute returned from an LDAP operation is passed through a
central table look-up based on it's LDAP syntax. The table key is
the LDAP syntax it's value is a Python callable that returns a
Python object matching the LDAP syntax. There are a handful of LDAP
attributes whose syntax is historically incorrect
(e.g. DistguishedNames that are defined as DirectoryStrings). The
table driven conversion mechanism is augmented with a table of
hard coded exceptions.
Currently only the following conversions occur via the table:
- dn's are converted to DN objects
- binary objects are converted to Python str objects (IPA
convention).
- everything else is converted to unicode using UTF-8 decoding (IPA
convention).
However, now that the table driven conversion mechanism is in place
it would be trivial to do things such as converting attributes
which have LDAP integer syntax into a Python integer, etc.
* Expected values in the unit tests which are a DN no longer need to
use lambda expressions to promote the returned value to a DN for
equality comparison. The return value is automatically promoted to
a DN. The lambda expressions have been removed making the code much
simpler and easier to read.
* Add class level logging to a number of classes which did not support
logging, less need for use of root_logger.
* Remove ipaserver/conn.py, it was unused.
* Consolidated duplicate code wherever it was found.
* Fixed many places that used string concatenation to form a new
string rather than string formatting operators. This is necessary
because string formatting converts it's arguments to a string prior
to building the result string. You can't concatenate a string and a
non-string.
* Simplify logic in rename_managed plugin. Use DN operators to edit
dn's.
* The live version of ipa-ldap-updater did not generate a log file.
The offline version did, now both do.
https://fedorahosted.org/freeipa/ticket/1670https://fedorahosted.org/freeipa/ticket/1671https://fedorahosted.org/freeipa/ticket/1672https://fedorahosted.org/freeipa/ticket/1673https://fedorahosted.org/freeipa/ticket/1674https://fedorahosted.org/freeipa/ticket/1392https://fedorahosted.org/freeipa/ticket/2872
When the --hostname option is given to ipa-client-install, we
write HOSTNAME to /etc/sysconfig/network. When that file didn't exist,
the installer crashed.
Create the file if it doesn't exist and we need to write to it.
https://fedorahosted.org/freeipa/ticket/2840
Certificate renewal can be done only one one CA as the certificates need
to be shared amongst them. certmonger has been trained to communicate
directly with dogtag to perform the renewals. The initial CA installation
is the defacto certificate renewal master.
A copy of the certificate is stored in the IPA LDAP tree in
cn=ca_renewal,cn=ipa,cn=etc,$SUFFIX, the rdn being the nickname of the
certificate, when a certificate is renewed. Only the most current
certificate is stored. It is valid to have no certificates there, it means
that no renewals have taken place.
The clones are configured with a new certmonger CA type that polls this
location in the IPA tree looking for an updated certificate. If one is
not found then certmonger is put into the CA_WORKING state and will poll
every 8 hours until an updated certificate is available.
The RA agent certificate, ipaCert in /etc/httpd/alias, is a special case.
When this certificate is updated we also need to update its entry in
the dogtag tree, adding the updated certificate and telling dogtag which
certificate to use. This is the certificate that lets IPA issue
certificates.
On upgrades we check to see if the certificate tracking is already in
place. If not then we need to determine if this is the master that will
do the renewals or not. This decision is made based on whether it was
the first master installed. It is concievable that this master is no
longer available meaning that none are actually tracking renewal. We
will need to document this.
https://fedorahosted.org/freeipa/ticket/2803
Many functions use low-level socket interface for connection or
various checks. However, most of the time we don't respect
automatic address family detection but rather try to force our
values. This may cause either redundat connection tries when an
address family is disabled on system tries or even crashes
when socket exceptions are not properly caught.
Instead of forcing address families to socket, rather use
getaddrinfo interface to automatically retrieve a list of all
relevant address families and other connection settings when
connecting to remote/local machine or binding to a local port.
Now, we will also fill correctly all connection parameters like
flowinfo and scopeid for IPv6 connections which will for example
prevent issues with scoped IPv6 addresses.
bind_port_responder function was changed to at first try to bind
to IPv6 wildcard address before IPv4 as IPv6 socket is able to
accept both IPv4 and IPv6 connections (unlike IPv4 socket).
nsslib connection was refactored to use nss.io.AddrInfo class to
get all the available connections. Socket is now not created by
default in NSSConnection class initializer, but rather when the
actual connection is being made, becase we do not an address family
where connection is successful.
https://fedorahosted.org/freeipa/ticket/2913https://fedorahosted.org/freeipa/ticket/2695
All service start/restart currently go through ipapython/platform so
move the "wait for service to start" code there as well.
A dictionary of known services and ports to wait on is defined in base.py
This is referenced by the platforms by instance name to determine what
to wait for. For the case of dirsrv if we get that as a plain name
(no specific instance) it is assumed to be the main IPA service.
https://fedorahosted.org/freeipa/ticket/2375https://fedorahosted.org/freeipa/ticket/2610
