This allows for a consistent way to retrieve the value from
LDAP. The method is used to find an existing entry. It is not usable
to add or remove entries.
Moving it in the code allows the value to always be set in the
substitution dictionary and not rely on a specific caller.
It was moved to installutils.py to avoid circular import.
https://pagure.io/freeipa/issue/8885
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
IPA topology plugin depends on the replication plugin but
389-ds cannot handle older alias querying in the plugin
configuration with 'nsslapd-plugin-depends-on-named: ..' attribute
See https://github.com/389ds/389-ds-base/issues/4786 for details
Fixes: https://pagure.io/freeipa/issue/8799
Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
There is still some exposure to killing in a bad place. This was
reproduced by killing the process in the parser.parse() call within
__restore_config (line 230) so the values were restored from the
backup but the new dse.ldif never written or copied. But the values
had already been restored from the state file.
I'm not sure this can ever be 100% bullet-proof since it can be
externally killed but if rather than calling restore_state() on the
values in __restore_config we use get_state() which will peek at the
values in the state file without removing them. Then the last step
is to pop upgrade-in-progress and then the rest.
If the values have been restored and the new ldif written and copied
then it's only upgrade-in-progress that really matters. The rest will
be overwritten.
https://pagure.io/freeipa/issue/7534
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
When applying update files to 389 the listeners are disabled.
There is a large try/except around this so that if a failure
happens then the configuration should be automatically
restored.
We've seen multiple cases where this doesn't occur. Best guess
is that users are killing or ^C breaking out of the script.
What happens in that case is that when the next upgrade is run
the configuration is backed up again overwriting the original
values. This leaves dirsrv with no listener on 389.
Add a new state, upgrade-in-progress, so that the backup of the
config information can be skipped when the upgrader is executed
again after a failure.
The idea behind using a new state value is that if additional
attributes are ever backed up we don't need to remember to update
the list of possible saved values to check to decide if the
upgrade is in progress.
https://pagure.io/freeipa/issue/7534
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
- drop unused dm_password and ldapi arguments
- remove online feature that was never implemented
- allow passing of api object that is used to populate substitution
dictionary
- simplify substitution dictionary updates
- remove unused instances vars
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
The helper function realm_to_serverid() and realm_to_ldap_uri() are
useful outside the server installation framework. They are now in
ipapython.ipaldap along other helpers for LDAP handling in FreeIPA.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
There was an issue with ipa-server-upgrade and it was
showing an error while upgrading:
DN... does not exists or haven't been updated, caused
by not moving pointer to file begining when re-reading.
Resolves: https://pagure.io/freeipa/issue/7644
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
The schema compat plugin is disabled on upgrades but it is
possible that it is not configured at all and this will
produce a rather nasty looking error message.
Check to see if it is configured at all before trying to
disable it.
https://pagure.io/freeipa/issue/6610
Reviewed-By: Christian Heimes <cheimes@redhat.com>
If this is enabled it can cause a deadlock with SSSD trying
to look up entries and it trying to get data on AD users
from SSSD.
When reading the entry from LDIF try to get the camel-case
nsslapd-pluginEnabled and fall back to the all lower-case
nsslapd-pluginenabled if that is not found. It would be nice
if the fetch function was case sensitive but this is likely
overkill as it is, but better safe than blowing up.
Upon restoring it will always write the camel-case version.
https://pagure.io/freeipa/issue/6721
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Add absolute_import from __future__ so that pylint
does not fail and to achieve python3 behavior in
python2.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Time estimates have been updated to be more accurate. Only
tasks that are estimated to take longer than 10 seconds have
the estimate displayed.
https://pagure.io/freeipa/issue/6596
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
...and vice versa
backup requires string not bytes, but ldap provide bytes thus data must
be decoded and encoded from restore
https://fedorahosted.org/freeipa/ticket/4985
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Here is an attempt to break the import cycle of hell between ipaplatform
and ipalib. All services now pass an ipalib.api object to
services.service(). RedHatServices.__init__() still needs to do a local
import because it initializes its wellknown service dict with service
instances.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Clean up unnecessary starts/stops of DS and unnescessary attributes.
If the DS is running, establish an LDAP connection and properly close
it.
https://fedorahosted.org/freeipa/ticket/6461
Reviewed-By: Martin Basti <mbasti@redhat.com>
Instead of delegating handling of some parameters like fstore to the parent
class, the *Instance installers had the logic copy-pasted in their
constructors. Some other members were also moved to the Service class and the
parent class constructors in children were fixed to modern standards of
initializing parent class in Python.
https://fedorahosted.org/freeipa/ticket/6392
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
connect/disconnect api.Backend.ldap2 connection when directory
server is started/stopped
https://fedorahosted.org/freeipa/ticket/6461
Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Unused variables may:
* make code less readable
* create dead code
* potentialy hide issues/errors
Enabled check should prevent to leave unused variable in code
Check is locally disabled for modules that fix is not clear or easy or have too many occurences of
unused variables
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Wildcard imports should not be used.
Check for wildcard imports has been enabled in pylint.
Pylint note: options 'wildcard-import' causes too much false positive
results, so instead it I used 'unused-wildcard-import' option which has almost
the same effect.
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Python 2 has keys()/values()/items(), which return lists,
iterkeys()/itervalues()/iteritems(), which return iterators,
and viewkeys()/viewvalues()/viewitems() which return views.
Python 3 has only keys()/values()/items(), which return views.
To get iterators, one can use iter() or a for loop/comprehension;
for lists there's the list() constructor.
When iterating through the entire dict, without modifying the dict,
the difference between Python 2's items() and iteritems() is
negligible, especially on small dicts (the main overhead is
extra memory, not CPU time). In the interest of simpler code,
this patch changes many instances of iteritems() to items(),
iterkeys() to keys() etc.
In other cases, helpers like six.itervalues are used.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
To detect if DS server is running, use the slapd socket for upgrade, and the LDAP port
for installation.
Without enabled LDAPi socket checking doesnt work.
https://fedorahosted.org/freeipa/ticket/4904
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
During server upgrade we should wait until DS is ready after restart, otherwise
connection error is raised.
Instead of 389 port, the DS socket is checked.
https://fedorahosted.org/freeipa/ticket/4904
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
* Prevent to continue with upgrade if a fatal error happened
* Use exceptions to handle failures
https://fedorahosted.org/freeipa/ticket/4904
Reviewed-By: David Kupka <dkupka@redhat.com>
To avoid cyclic imports realm_to_serverid function had to be moved to
installutils from dsinstance.
Required for: https://fedorahosted.org/freeipa/ticket/4925
Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
* add 'plugin' directive
* specify plugins order in update files
* remove 'run plugins' options
* use ldapupdater API instance in plugins
* add update files representing former PreUpdate and PostUpdate order of plugins
https://fedorahosted.org/freeipa/ticket/4904
Reviewed-By: David Kupka <dkupka@redhat.com>
As --test option is not used for developing, and it is not recommended
to test if upgrade will pass, this path removes it copmletely.
https://fedorahosted.org/freeipa/ticket/3448
Reviewed-By: David Kupka <dkupka@redhat.com>
Allow running some installation after failure,
and use this for the upgradeinstance cleanup steps.
https://fedorahosted.org/freeipa/ticket/4499
Reviewed-By: Martin Kosek <mkosek@redhat.com>
Class PreSchemaUpdate is executed before ldap schema update
This is required by ticket: https://fedorahosted.org/freeipa/ticket/3210
Reviewed-By: Martin Kosek <mkosek@redhat.com>
The new updater is run as part of `ipa-ldap-updater --upgrade`
and `ipa-ldap-updater --schema` (--schema is a new option).
The --schema-file option to ipa-ldap-updater may be used (multiple
times) to select a non-default set of schema files to update against.
The updater adds an X-ORIGIN tag with the current IPA version to
all elements it adds or modifies.
https://fedorahosted.org/freeipa/ticket/3454
In order to have control over the order that updates are applied
a numbering system was created for the update files. These values
were not actually used.
The updates were sorted by DN length and in most cases this was
adequate for proper function. The exception was with roles where
in some cases a role was added as a member of a permission before
the role itself was added so the memberOf value was never created.
Now updates are computed and applied in blocks of 10.
https://fedorahosted.org/freeipa/ticket/3377
Forces more consistency into ipa-server-install output. All
descriptions of services that are not instances of
SimpleServiceInstance are now in the following format:
<Description> (<Service Name>)
Furthermore, start_creation method has been modified to support
custom start and end messages. See documentation for more info.
https://fedorahosted.org/freeipa/ticket/3059
We've been stopping both DS instances (main and PKI) when upgrading.
This can happen while the CA is running. In some cases stopping the PKI
DS also killed the CA.
Only stop the specific instance for upgrades.
Also, wait for open ports after the upgrade is complete. The wait was
skipped previously. This can prevent bugs if scripts that need a DS are
run after the upgrade.
https://fedorahosted.org/freeipa/ticket/3083
All service start/restart currently go through ipapython/platform so
move the "wait for service to start" code there as well.
A dictionary of known services and ports to wait on is defined in base.py
This is referenced by the platforms by instance name to determine what
to wait for. For the case of dirsrv if we get that as a plain name
(no specific instance) it is assumed to be the main IPA service.
https://fedorahosted.org/freeipa/ticket/2375https://fedorahosted.org/freeipa/ticket/2610
There are two reasons for the plugin framework:
1. To provide a way of doing manual/complex LDAP changes without having
to keep extending ldapupdate.py (like we did with managed entries).
2. Allows for better control of restarts.
There are two types of plugins, preop and postop. A preop plugin runs
before any file-based updates are loaded. A postop plugin runs after
all file-based updates are applied.
A preop plugin may update LDAP directly or craft update entries to be
applied with the file-based updates.
Either a preop or postop plugin may attempt to restart the dirsrv instance.
The instance is only restartable if ipa-ldap-updater is being executed
as root. A warning is printed if a restart is requested for a non-root
user.
Plugins are not executed by default. This is so we can use ldapupdate
to apply simple updates in commands like ipa-nis-manage.
https://fedorahosted.org/freeipa/ticket/1789https://fedorahosted.org/freeipa/ticket/1790https://fedorahosted.org/freeipa/ticket/2032
change default_logger_level to debug in configure_standard_logging
add new ipa_log_manager module, move log_mgr there, also export
root_logger from log_mgr.
change all log_manager imports to ipa_log_manager and change
log_manager.root_logger to root_logger.
add missing import for parse_log_level()
