'ipa pwpolicy-find' output is now sorted by priority of the policies.
Lower position means lower priority. Global policy is then at the bottom.
The changes has also affected LDAPSearch class in baseldap.py:
LDAPSearch class sorts the search results by primary key be default
(which is usually 'cn'). Therefor a function pointer entries_sortfn
was added. If no sorting function exists, default sorting by primary key
is used.
Sorting function had to be introduced due to the fact that pwpolicy's
primary key is also it's 'cn' and global policy is not allowed to have any
priority.
https://fedorahosted.org/freeipa/ticket/2045
The default log level for server messages captured by httpd's
error_log historically was INFO. The log_manager patch had it set to
ERROR, this patch resets it back to INFO.
Although it would have been trival to set the default_level to INFO in
IPALogManager.configure_from_env() that is not logically the correct
place. It would be much better if the default_level can be reset by
simply assigning it to the log_mgr. To accomplish that
LogManager.default_level was converted to a property with a getter and
setter. The setter runs LogManager.apply_configuratin() after the
default_level is modified. LogManager.set_default_level() was also
added to allow simultaneously updating the configure_state.
While testing some minor problems were observed and also fixed:
* Removed some print statement which had been left in by mistake
* Removed the ability to set the handler level in the config file
because of chicken-and-egg issues of when handlers get created.
The Env config file format is too inflexible to support detailed
logging configuration. If the Env config format is ever made more
flexible we can come back and add this back in. The handler config
setting in Env had never been used and never worked so there is no
issue in removing it.
Remove "List" parameter type and replace all occurences of it with appropriate
multi-valued parameter ("Str" in most cases) with csv enabled.
Add new parameter type "Any", capable of holding values of any type. This is
needed by the "batch" command, as "Str" is not suitable type for the "methods"
parameter.
ticket 2007
The validator has been improved to support better both SOA format
(e-mail address in a domain name format, without '@') and standard
e-mail format. Allow '\.' character in a SOA format encoding the
standard '.' in the local-part of an e-mail. Normalization code
has been moved to one common function.
https://fedorahosted.org/freeipa/ticket/2053
We use convenience types (classes) in IPA which make working with LDAP
easier and more robust. It would be really nice if the basic python-ldap
library understood our utility types and could accept them as parameters
to the basic ldap functions and/or the basic ldap functions returned our
utility types.
Normally such a requirement would trivially be handled in an object-
oriented language (which Python is) by subclassing to extend and modify
the functionality. For some reason we didn't do this with the python-ldap
classes.
python-ldap objects are primarily used in two different places in our
code, ipaserver.ipaldap.py for the IPAdmin class and in
ipaserver/plugins/ldap2.py for the ldap2 class's .conn member.
In IPAdmin we use a IPA utility class called Entry to make it easier to
use the results returned by LDAP. The IPAdmin class is derived from
python-ldap.SimpleLDAPObject. But for some reason when we added the
support for the use of the Entry class in SimpleLDAPObject we didn't
subclass SimpleLDAPObject and extend it for use with the Entry class as
would be the normal expected methodology in an object-oriented language,
rather we used an obscure feature of the Python language to override all
methods of the SimpleLDAPObject class by wrapping those class methods in
another function call. The reason why this isn't a good approach is:
* It violates object-oriented methodology.
* Other classes cannot be derived and inherit the customization (because
the method wrapping occurs in a class instance, not within the class
type).
* It's non-obvious and obscure
* It's inefficient.
Here is a summary of what the code was doing:
It iterated over every member of the SimpleLDAPObject class and if it was
callable it wrapped the method. The wrapper function tested the name of
the method being wrapped, if it was one of a handful of methods we wanted
to customize we modified a parameter and called the original method. If
the method wasn't of interest to use we still wrapped the method.
It was inefficient because every non-customized method (the majority)
executed a function call for the wrapper, the wrapper during run-time used
logic to determine if the method was being overridden and then called the
original method. So every call to ldap was doing extra function calls and
logic processing which for the majority of cases produced nothing useful
(and was non-obvious from brief code reading some methods were being
overridden).
Object-orientated languages have support built in for calling the right
method for a given class object that do not involve extra function call
overhead to realize customized class behaviour. Also when programmers look
for customized class behaviour they look for derived classes. They might
also want to utilize the customized class as the base class for their use.
Also the wrapper logic was fragile, it did things like: if the method name
begins with "add" I'll unconditionally modify the first and second
argument. It would be some much cleaner if the "add", "add_s", etc.
methods were overridden in a subclass where the logic could be seen and
where it would apply to only the explicit functions and parameters being
overridden.
Also we would really benefit if there were classes which could be used as
a base class which had specific ldap customization.
At the moment our ldap customization needs are:
1) Support DN objects being passed to ldap operations
2) Support Entry & Entity objects being passed into and returned from
ldap operations.
We want to subclass the ldap SimpleLDAPObject class, that is the base
ldap class with all the ldap methods we're using. IPASimpleLDAPObject
class would subclass SimpleLDAPObject class which knows about DN
objects (and possilby other IPA specific types that are universally
used in IPA). Then IPAEntrySimpleLDAPObject would subclass
IPASimpleLDAPObject which knows about Entry objects.
The reason for the suggested class hierarchy is because DN objects will be
used whenever we talk to LDAP (in the future we may want to add other IPA
specific classes which will always be used). We don't add Entry support to
the the IPASimpleLDAPObject class because Entry objects are (currently)
only used in IPAdmin.
What this patch does is:
* Introduce IPASimpleLDAPObject derived from
SimpleLDAPObject. IPASimpleLDAPObject is DN object aware.
* Introduce IPAEntryLDAPObject derived from
IPASimpleLDAPObject. IPAEntryLDAPObject is Entry object aware.
* Derive IPAdmin from IPAEntryLDAPObject and remove the funky method
wrapping from IPAdmin.
* Code which called add_s() with an Entry or Entity object now calls
addEntry(). addEntry() always existed, it just wasn't always
used. add_s() had been modified to accept Entry or Entity object
(why didn't we just call addEntry()?). The add*() ldap routine in
IPAEntryLDAPObject have been subclassed to accept Entry and Entity
objects, but that should proably be removed in the future and just
use addEntry().
* Replace the call to ldap.initialize() in ldap2.create_connection()
with a class constructor for IPASimpleLDAPObject. The
ldap.initialize() is a convenience function in python-ldap, but it
always returns a SimpleLDAPObject created via the SimpleLDAPObject
constructor, thus ldap.initialize() did not allow subclassing, yet
has no particular ease-of-use advantage thus we better off using the
obvious class constructor mechanism.
* Fix the use of _handle_errors(), it's not necessary to construct an
empty dict to pass to it.
If we follow the standard class derivation pattern for ldap we can make us
of our own ldap utilities in a far easier, cleaner and more efficient
manner.
The IPAdmin class in ipaserver/ipaldap.py has methods with anonymous
undefined parameter lists.
For example:
def getList(self,*args):
In Python syntax this means you can call getList with any positional
parameter list you want.
This is bad because:
1) It's not true, *args gets passed to an ldap function with a well
defined parameter list, so you really do have to call it with a
defined parameter list. *args will let you pass anything, but once it
gets passed to the ldap function it will blow up if the parameters do
not match (what parameters are those you're wondering? see item 2).
2) The programmer does not know what the valid parameters are unless
they are defined in the formal parameter list.
3) Without a formal parameter list automatic documentation generators
cannot produce API documentation (see item 2)
4) The Python interpreter cannot validate the parameters being passed
because there is no formal parameter list. Note, Python does not
validate the type of parameters, but it does validate the correct
number of postitional parameters are passed and only defined keyword
parameters are passed. Bypassing the language support facilities leads
to programming errors.
5) Without a formal parameter list program checkers such as pylint
cannot validate the program which leads to progamming errors.
6) Without a formal parameter list which includes default keyword
parameters it's not possible to use keyword arguments nor to know what
their default values are (see item 2). One is forced to pass a keyword
argument as a positional argument, plus you must then pass every
keyword argument between the end of the positional argument list and
keyword arg of interest even of the other keyword arguments are not of
interest. This also demands you know what the default value of the
intermediate keyword arguments are (see item 2) and hope they don't
change.
Also the *args anonymous tuple get passed into the error handling code
so it can report what the called values were. But because the tuple is
anonymous the error handler cannot not describe what it was passed. In
addition the error handling code makes assumptions about the possible
contents of the anonymous tuple based on current practice instead of
actual defined values. Things like "if the number of items in the
tuple is 2 or less then the first tuple item must be a dn
(Distinguished Name)" or "if the number of items in the tuple is
greater than 2 then the 3rd item must be an ldap search filter". These
are constructs which are not robust and will fail at some point in the
future.
This patch also fixes the use of IPAdmin.addEntry(). It was sometimes
being called with (dn, modlist), sometimes a Entry object, or
sometimes a Entity object. Now it's always called with either a Entry
or Entity object and IPAdmin.addEntry() validates the type of the
parameter passed.
Add a --delattr option to round out multi-valued attribute
manipulation. The new option is available for all LDAPUpdate based
commands. --delattr is evaluated last, it can remove any value
present either in --addattr/--setattr option or in current LDAP
object.
--*attr processing was completely refactored and placed to one
independent function available for all baseldap commands. For this
purpose a missing common base class for all baseldap commands has
been implemented. The new class should serve not only for --*attr
processing but also for other common baseldap methods and
attributes.
This approach will also benefit other custom commands based neither
on LDAPCreate nor LDAPUpdate. They can easily integrate --*attr
option processing when needed.
https://fedorahosted.org/freeipa/ticket/1929
ipa-server-install may create some files in the first phase of
installation before the actual installation and configuring of
services starts. If the installation is interrupted, these files
may prevent installing the server again until IPA server is
uninstalled. This may be confusing and annoying for the user.
This patch safely recovers all known files that could be created
in the first phase of the installation. No clean up is done if
the actual installation has not started yet or the installation
returned success.
https://fedorahosted.org/freeipa/ticket/1980
automember functionality is depends on predefined data is in LDAP.
Since we add it for fresh installs only, automember cannot be used
for upgraded servers. Make sure that automember LDAP data is added
during upgrade too.
https://fedorahosted.org/freeipa/ticket/1992
There are two reasons for the plugin framework:
1. To provide a way of doing manual/complex LDAP changes without having
to keep extending ldapupdate.py (like we did with managed entries).
2. Allows for better control of restarts.
There are two types of plugins, preop and postop. A preop plugin runs
before any file-based updates are loaded. A postop plugin runs after
all file-based updates are applied.
A preop plugin may update LDAP directly or craft update entries to be
applied with the file-based updates.
Either a preop or postop plugin may attempt to restart the dirsrv instance.
The instance is only restartable if ipa-ldap-updater is being executed
as root. A warning is printed if a restart is requested for a non-root
user.
Plugins are not executed by default. This is so we can use ldapupdate
to apply simple updates in commands like ipa-nis-manage.
https://fedorahosted.org/freeipa/ticket/1789https://fedorahosted.org/freeipa/ticket/1790https://fedorahosted.org/freeipa/ticket/2032
change default_logger_level to debug in configure_standard_logging
add new ipa_log_manager module, move log_mgr there, also export
root_logger from log_mgr.
change all log_manager imports to ipa_log_manager and change
log_manager.root_logger to root_logger.
add missing import for parse_log_level()
This patch changes the way plugins are initialized. Instead of
finalizing all the plugins at once, plugins are finalized only after
they are accessed (currently applies to Command, Object and
Attribute subclasses, only in CLI by default).
This change provides significant performance boost, as only the
plugins that are actually used are finalized.
ticket 1336
"Extending FreeIPA" is a developer guide of FreeIPA core framework.
Please make sure to improve the guide every time parts of the core framework
are affected by your changes. This document ideally should correspond to
the current state of the framework.
The Guide is written using Emacs Org Mode but can be edited with any
plain text editing tool. Emacs is only required to convert it to
distribution formats like HTML and TXT.
See guide/Makefile for building the guide and README for details of
the build environment.
ipa-client-install was failing and returning traceback when
wasn't run by root. It was caused by logging initialization that
was taking part before the root privileges check. To correct it,
the check was moved before the logging initialization.
https://fedorahosted.org/freeipa/ticket/2123
When ``make client-install`` is called, create /etc/ipa and
/var/lib/ipa-client/sysrestore directories required for successful use of
ipa-client-install.
Do it only if DESTDIR is set to help packagers to notice that these
directories have to exist
https://fedorahosted.org/freeipa/ticket/1849
Make sure that PublicError does not crash when it receives
Gettext/NGettext object. Instead of throwing a type error, do the
translation to receive the required unicode text.
https://fedorahosted.org/freeipa/ticket/2096
We no longer need to enforce that no 389-ds instances exist on an IPA
server. Checking that the ports exist should be enough.
This used to be one mechanism we used to check to see if IPA was already
installed. We have a better mechanism now.
https://fedorahosted.org/freeipa/ticket/1735
Make sure that when Directory Manager password is entered,
we directly do a simple bind instead of trying binding via GSSAPI.
Also capture ldap.INVALID_CREDENTIALS exception and provide nice
error message than crash.
https://fedorahosted.org/freeipa/ticket/1927
Some Param or Output attributes do not cause API incompatibility
(e.g. doc, label or callables) and does not need to be included
in API.txt. When these attributes are modified, a lot of bogus
changes may get in API.txt - making the real API changes less
detectable.
https://fedorahosted.org/freeipa/ticket/2107
Parameters in LDAP objects missed an information if they are real
LDAP attributes or not. Real LDAP attributes are written to
entry_attrs dictionary in plugin callbacks and are being encoded.
This causes issues when plugin callbacks does not expect that
the parameters values are already encoded for submission to LDAP.
This patch introduces a new flag "noattribute" used to mark that
a parameter is not an LDAP attribute and thus should not be encoded
or added to entry_attrs. Param documentation is improved to describe
the meaning of this and other Param flags or attributes.
https://fedorahosted.org/freeipa/ticket/2097
https://fedorahosted.org/freeipa/ticket/1794
If the master does not yet support the total update list feature we still
run the memberof fixup task and not fail to replicate due to the new
attribute not being settable.
Jointly-developed-with: Simo Sorce <ssorce@redhat.com>
Jointly-developed-with: Nathank Kinder <nkinder@redhat.com>
