IntenseWebs/freeipa
3
0
Fork 0
mirror of https://salsa.debian.org/freeipa-team/freeipa.git synced 2024-12-29 02:11:09 -06:00
Code Issues Packages Projects Releases Wiki Activity
a663e83cb2
freeipa/ipalib/plugins/migration.py

814 lines
32 KiB
Python
Raw Normal View History

Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
# Authors:
# Pavel Zuna <pzuna@redhat.com>
#
# Copyright (C) 2009 Red Hat
# see file 'COPYING' for use and warranty information
#
Change FreeIPA license to GPLv3+ The changes include: * Change license blobs in source files to mention GPLv3+ not GPLv2 only * Add GPLv3+ license text * Package COPYING not LICENSE as the license blobs (even the old ones) mention COPYING specifically, it is also more common, I think https://fedorahosted.org/freeipa/ticket/239
2010-12-09 06:59:11 -06:00
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
Change FreeIPA license to GPLv3+ The changes include: * Change license blobs in source files to mention GPLv3+ not GPLv2 only * Add GPLv3+ license text * Package COPYING not LICENSE as the license blobs (even the old ones) mention COPYING specifically, it is also more common, I think https://fedorahosted.org/freeipa/ticket/239
2010-12-09 06:59:11 -06:00
# along with this program. If not, see <http://www.gnu.org/licenses/>.
ticket 1669 - improve i18n docstring extraction This patch reverts the use of pygettext for i18n string extraction. It was originally introduced because the help documentation for commands are in the class docstring and module docstring. Docstrings are a Python construct whereby any string which immediately follows a class declaration, function/method declaration or appears first in a module is taken to be the documentation for that object. Python automatically assigns that string to the __doc__ variable associated with the object. Explicitly assigning to the __doc__ variable is equivalent and permitted. We mark strings in the source for i18n translation by embedding them in _() or ngettext(). Specialized extraction tools (e.g. xgettext) scan the source code looking for strings with those markers and extracts the string for inclusion in a translation catalog. It was mistakingly assumed one could not mark for translation Python docstrings. Since some docstrings are vital for our command help system some method had to be devised to extract docstrings for the translation catalog. pygettext has the ability to locate and extract docstrings and it was introduced to acquire the documentation for our commands located in module and class docstrings. However pygettext was too large a hammer for this task, it lacked any fined grained ability to extract only the docstrings we were interested in. In practice it extracted EVERY docstring in each file it was presented with. This caused a large number strings to be extracted for translation which had no reason to be translated, the string might have been internal code documentation never meant to be seen by users. Often the superfluous docstrings were long, complex and likely difficult to translate. This placed an unnecessary burden on our volunteer translators. Instead what is needed is some method to extract only those strings intended for translation. We already have such a mechanism and it is already widely used, namely wrapping strings intended for translation in calls to _() or _negettext(), i.e. marking a string for i18n translation. Thus the solution to the docstring translation problem is to mark the docstrings exactly as we have been doing, it only requires that instead of a bare Python docstring we instead assign the marked string to the __doc__ variable. Using the hypothetical class foo as an example. class foo(Command): ''' The foo command takes out the garbage. ''' Would become: class foo(Command): __doc__ = _('The foo command takes out the garbage.') But which docstrings need to be marked for translation? The makeapi tool knows how to iterate over every command in our public API. It was extended to validate every command's documentation and report if any documentation is missing or not marked for translation. That information was then used to identify each docstring in the code which needed to be transformed. In summary what this patch does is: * Remove the use of pygettext (modification to install/po/Makefile.in) * Replace every docstring with an explicit assignment to __doc__ where the rhs of the assignment is an i18n marking function. * Single line docstrings appearing in multi-line string literals (e.g. ''' or """) were replaced with single line string literals because the multi-line literals were introducing unnecessary whitespace and newlines in the string extracted for translation. For example: ''' The foo command takes out the garbage. ''' Would appear in the translation catalog as: "\n The foo command takes out the garbage.\n " The superfluous whitespace and newlines are confusing to translators and requires us to strip leading and trailing whitespace from the translation at run time. * Import statements were moved from below the docstring to above it. This was necessary because the i18n markers are imported functions and must be available before the the doc is parsed. Technically only the import of the i18n markers had to appear before the doc but stylistically it's better to keep all the imports together. * It was observed during the docstring editing process that the command documentation was inconsistent with respect to the use of periods to terminate a sentence. Some doc had a trailing period, others didn't. Consistency was enforced by adding a period to end of every docstring if one was missing.
2011-08-24 21:48:30 -05:00
import re
import ldap as _ldap
from ipalib import api, errors, output
Parse comma-separated lists of values in all parameter types. This can be enabled for a specific parameter by setting the "csv" option to True. Remove "List" parameter type and replace all occurences of it with appropriate multi-valued parameter ("Str" in most cases) with csv enabled. Add new parameter type "Any", capable of holding values of any type. This is needed by the "batch" command, as "Str" is not suitable type for the "methods" parameter. ticket 2007
2011-11-21 09:50:27 -06:00
from ipalib import Command, Password, Str, Flag, StrEnum
ticket 1669 - improve i18n docstring extraction This patch reverts the use of pygettext for i18n string extraction. It was originally introduced because the help documentation for commands are in the class docstring and module docstring. Docstrings are a Python construct whereby any string which immediately follows a class declaration, function/method declaration or appears first in a module is taken to be the documentation for that object. Python automatically assigns that string to the __doc__ variable associated with the object. Explicitly assigning to the __doc__ variable is equivalent and permitted. We mark strings in the source for i18n translation by embedding them in _() or ngettext(). Specialized extraction tools (e.g. xgettext) scan the source code looking for strings with those markers and extracts the string for inclusion in a translation catalog. It was mistakingly assumed one could not mark for translation Python docstrings. Since some docstrings are vital for our command help system some method had to be devised to extract docstrings for the translation catalog. pygettext has the ability to locate and extract docstrings and it was introduced to acquire the documentation for our commands located in module and class docstrings. However pygettext was too large a hammer for this task, it lacked any fined grained ability to extract only the docstrings we were interested in. In practice it extracted EVERY docstring in each file it was presented with. This caused a large number strings to be extracted for translation which had no reason to be translated, the string might have been internal code documentation never meant to be seen by users. Often the superfluous docstrings were long, complex and likely difficult to translate. This placed an unnecessary burden on our volunteer translators. Instead what is needed is some method to extract only those strings intended for translation. We already have such a mechanism and it is already widely used, namely wrapping strings intended for translation in calls to _() or _negettext(), i.e. marking a string for i18n translation. Thus the solution to the docstring translation problem is to mark the docstrings exactly as we have been doing, it only requires that instead of a bare Python docstring we instead assign the marked string to the __doc__ variable. Using the hypothetical class foo as an example. class foo(Command): ''' The foo command takes out the garbage. ''' Would become: class foo(Command): __doc__ = _('The foo command takes out the garbage.') But which docstrings need to be marked for translation? The makeapi tool knows how to iterate over every command in our public API. It was extended to validate every command's documentation and report if any documentation is missing or not marked for translation. That information was then used to identify each docstring in the code which needed to be transformed. In summary what this patch does is: * Remove the use of pygettext (modification to install/po/Makefile.in) * Replace every docstring with an explicit assignment to __doc__ where the rhs of the assignment is an i18n marking function. * Single line docstrings appearing in multi-line string literals (e.g. ''' or """) were replaced with single line string literals because the multi-line literals were introducing unnecessary whitespace and newlines in the string extracted for translation. For example: ''' The foo command takes out the garbage. ''' Would appear in the translation catalog as: "\n The foo command takes out the garbage.\n " The superfluous whitespace and newlines are confusing to translators and requires us to strip leading and trailing whitespace from the translation at run time. * Import statements were moved from below the docstring to above it. This was necessary because the i18n markers are imported functions and must be available before the the doc is parsed. Technically only the import of the i18n markers had to appear before the doc but stylistically it's better to keep all the imports together. * It was observed during the docstring editing process that the command documentation was inconsistent with respect to the use of periods to terminate a sentence. Some doc had a trailing period, others didn't. Consistency was enforced by adding a period to end of every docstring if one was missing.
2011-08-24 21:48:30 -05:00
from ipalib.cli import to_cli
Validate DN & RDN parameters for migrate command Ticket #2555 We were generating a traceback (server error) if a malformed RDN was passed as a parameter to the migrate command. * add parameter validation functions validate_dn_param() and validate_rdn_param() to ipalib.util. Those functions simply invoke the DN or RDN constructor from our dn module passing it the string representation. If the constructor does not throw an error it's valid. * Add the parameter validation function pointers to the Param objects in the migrate command. * Make the usercontainer and groupcontainer parameters required. passing --usercontainer= on the command line will produce ipa: ERROR: 'user_container' is required * Fix _get_search_bases() so if a container dn is empty it it just uses the base dn alone instead of faulting (currently bullet-proofing because now the containers are required). * Update the doc for usercontainer and groupcontainer to reflect the fact they are DN's not RDN's. A RDN can only be one level and it should be possible to have a container more than one RDN removed from the base.
2012-04-16 01:33:26 -05:00
from ipalib.util import validate_dn_param
migrate process cannot handle multivalued pkey attribute When group/user is migrated, the attribute used for RDN may be multivalued. Make sure that we pick the value used in the RDN which should be the unique one and not just the first one. https://fedorahosted.org/freeipa/ticket/1892
2011-09-29 04:55:13 -05:00
from ipalib.dn import *
Don't create private groups for migrated users, check for valid gidnumber Migrated users don't get a private group, there is no safe way to verify that the namespace is correct without redoing the uidnumber as well. Verify that the GID at least points to a valid group on the remote server and warn if it doesn't (this doesn't guarantee that the group gets migrated but at least we try). If the remote entry has no gidNumber then don't migrate that user. We don't know why that user is non-POSIX, it could be a special user used for auth, for example. Add a loginshell if the remote user doesn't have one. https://fedorahosted.org/freeipa/ticket/2562
2012-03-22 12:40:54 -05:00
from ipalib.plugins.user import NO_UPG_MAGIC
ticket 1669 - improve i18n docstring extraction This patch reverts the use of pygettext for i18n string extraction. It was originally introduced because the help documentation for commands are in the class docstring and module docstring. Docstrings are a Python construct whereby any string which immediately follows a class declaration, function/method declaration or appears first in a module is taken to be the documentation for that object. Python automatically assigns that string to the __doc__ variable associated with the object. Explicitly assigning to the __doc__ variable is equivalent and permitted. We mark strings in the source for i18n translation by embedding them in _() or ngettext(). Specialized extraction tools (e.g. xgettext) scan the source code looking for strings with those markers and extracts the string for inclusion in a translation catalog. It was mistakingly assumed one could not mark for translation Python docstrings. Since some docstrings are vital for our command help system some method had to be devised to extract docstrings for the translation catalog. pygettext has the ability to locate and extract docstrings and it was introduced to acquire the documentation for our commands located in module and class docstrings. However pygettext was too large a hammer for this task, it lacked any fined grained ability to extract only the docstrings we were interested in. In practice it extracted EVERY docstring in each file it was presented with. This caused a large number strings to be extracted for translation which had no reason to be translated, the string might have been internal code documentation never meant to be seen by users. Often the superfluous docstrings were long, complex and likely difficult to translate. This placed an unnecessary burden on our volunteer translators. Instead what is needed is some method to extract only those strings intended for translation. We already have such a mechanism and it is already widely used, namely wrapping strings intended for translation in calls to _() or _negettext(), i.e. marking a string for i18n translation. Thus the solution to the docstring translation problem is to mark the docstrings exactly as we have been doing, it only requires that instead of a bare Python docstring we instead assign the marked string to the __doc__ variable. Using the hypothetical class foo as an example. class foo(Command): ''' The foo command takes out the garbage. ''' Would become: class foo(Command): __doc__ = _('The foo command takes out the garbage.') But which docstrings need to be marked for translation? The makeapi tool knows how to iterate over every command in our public API. It was extended to validate every command's documentation and report if any documentation is missing or not marked for translation. That information was then used to identify each docstring in the code which needed to be transformed. In summary what this patch does is: * Remove the use of pygettext (modification to install/po/Makefile.in) * Replace every docstring with an explicit assignment to __doc__ where the rhs of the assignment is an i18n marking function. * Single line docstrings appearing in multi-line string literals (e.g. ''' or """) were replaced with single line string literals because the multi-line literals were introducing unnecessary whitespace and newlines in the string extracted for translation. For example: ''' The foo command takes out the garbage. ''' Would appear in the translation catalog as: "\n The foo command takes out the garbage.\n " The superfluous whitespace and newlines are confusing to translators and requires us to strip leading and trailing whitespace from the translation at run time. * Import statements were moved from below the docstring to above it. This was necessary because the i18n markers are imported functions and must be available before the the doc is parsed. Technically only the import of the i18n markers had to appear before the doc but stylistically it's better to keep all the imports together. * It was observed during the docstring editing process that the command documentation was inconsistent with respect to the use of periods to terminate a sentence. Some doc had a trailing period, others didn't. Consistency was enforced by adding a period to end of every docstring if one was missing.
2011-08-24 21:48:30 -05:00
if api.env.in_server and api.env.context in ['lite', 'server']:
try:
from ipaserver.plugins.ldap2 import ldap2
except StandardError, e:
raise e
from ipalib import _
__doc__ = _("""
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
Migration to IPA
Add documentation to the migrate-ds command. ticket 539
2010-12-07 10:47:21 -06:00
Migrate users and groups from an LDAP server to IPA.
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
Add documentation to the migrate-ds command. ticket 539
2010-12-07 10:47:21 -06:00
This performs an LDAP query against the remote server searching for
users and groups in a container. In order to migrate passwords you need
to bind as a user that can read the userPassword attribute on the remote
server. This is generally restricted to high-level admins such as
cn=Directory Manager in 389-ds (this is the default bind user).
The default user container is ou=People.
The default group container is ou=Groups.
Users and groups that already exist on the IPA server are skipped.
Two LDAP schemas define how group members are stored: RFC2307 and
RFC2307bis. RFC2307bis uses member and uniquemember to specify group
members, RFC2307 uses memberUid. The default schema is RFC2307bis.
Migration warning when compat enabled Added check into migration plugin to warn user when compat is enabled. If compat is enabled, the migration fails and user is warned that he must turn the compat off or run the script with (the newly introduced) option '--with-compat'. '--with-compat' is new flag. If it is set, the compat status is ignored. https://fedorahosted.org/freeipa/ticket/2274
2012-03-01 04:41:53 -06:00
The schema compat feature allows IPA to reformat data for systems that
do not support RFC2307bis. It is recommended that this feature is disabled
during migration to reduce system overhead. It can be re-enabled after
migration. To migrate with it enabled use the "--with-compat" option.
Add documentation to the migrate-ds command. ticket 539
2010-12-07 10:47:21 -06:00
Migrated users do not have Kerberos credentials, they have only their
Fix style and grammatical issues in built-in command help. There is a rather large API.txt change but it is only due to changes in the doc string in parameters. ticket 729
2011-03-04 10:08:54 -06:00
LDAP password. To complete the migration process, users need to go
Add documentation to the migrate-ds command. ticket 539
2010-12-07 10:47:21 -06:00
to http://ipa.example.com/ipa/migration and authenticate using their
LDAP password in order to generate their Kerberos credentials.
Fix style and grammatical issues in built-in command help. There is a rather large API.txt change but it is only due to changes in the doc string in parameters. ticket 729
2011-03-04 10:08:54 -06:00
Migration is disabled by default. Use the command ipa config-mod to
enable it:
Add documentation to the migrate-ds command. ticket 539
2010-12-07 10:47:21 -06:00
ipa config-mod --enable-migration=TRUE
Add support defaultNamingContext and add --basedn to migrate-ds There are two sides to this, the server and client side. On the server side we attempt to add a defaultNamingContext on already installed servers. This will fail on older 389-ds instances but the failure is not fatal. New installations on versions of 389-ds that support this attribute will have it already defined. On the client side we need to look for both defaultNamingContext and namingContexts. We still need to check that the defaultNamingContext is an IPA server (info=IPAV2). The migration change also takes advantage of this and adds a new option which allows one to provide a basedn to use instead of trying to detect it. https://fedorahosted.org/freeipa/ticket/1919 https://fedorahosted.org/freeipa/ticket/2314
2012-01-30 15:29:32 -06:00
If a base DN is not provided with --basedn then IPA will use either
the value of defaultNamingContext if it is set or the first value
in namingContexts set in the root of the remote LDAP server.
Add documentation to the migrate-ds command. ticket 539
2010-12-07 10:47:21 -06:00
EXAMPLES:
Fix typo in migration documentation
2010-12-13 08:48:16 -06:00
The simplest migration, accepting all defaults:
Add documentation to the migrate-ds command. ticket 539
2010-12-07 10:47:21 -06:00
ipa migrate-ds ldap://ds.example.com:389
Improve migration help Improve migration help topic so that it easier understandable: - Add missing list of Topic commands - Add one more example to demonstrate migration abilities - Add breaks to too long lines to improve readibility https://fedorahosted.org/freeipa/ticket/2174
2012-02-03 02:38:16 -06:00
Specify the user and group container. This can be used to migrate user
and group data from an IPA v1 server:
ipa migrate-ds --user-container='cn=users,cn=accounts' \\
--group-container='cn=groups,cn=accounts' \\
ldap://ds.example.com:389
Improve handling of GIDs when migrating groups Since IPA v2 server already contain predefined groups that may collide with groups in migrated (IPA v1) server (for example admins, ipausers), users having colliding group as their primary group may happen to belong to an unknown group on new IPA v2 server. Implement --group-overwrite-gid option to overwrite GID of already existing groups to prevent this issue. https://fedorahosted.org/freeipa/ticket/1866
2011-10-03 09:01:01 -05:00
Since IPA v2 server already contain predefined groups that may collide with
Improve migration help Improve migration help topic so that it easier understandable: - Add missing list of Topic commands - Add one more example to demonstrate migration abilities - Add breaks to too long lines to improve readibility https://fedorahosted.org/freeipa/ticket/2174
2012-02-03 02:38:16 -06:00
groups in migrated (IPA v1) server (for example admins, ipausers), users
having colliding group as their primary group may happen to belong to
an unknown group on new IPA v2 server.
Improve handling of GIDs when migrating groups Since IPA v2 server already contain predefined groups that may collide with groups in migrated (IPA v1) server (for example admins, ipausers), users having colliding group as their primary group may happen to belong to an unknown group on new IPA v2 server. Implement --group-overwrite-gid option to overwrite GID of already existing groups to prevent this issue. https://fedorahosted.org/freeipa/ticket/1866
2011-10-03 09:01:01 -05:00
Use --group-overwrite-gid option to overwrite GID of already existing groups
to prevent this issue:
Improve migration help Improve migration help topic so that it easier understandable: - Add missing list of Topic commands - Add one more example to demonstrate migration abilities - Add breaks to too long lines to improve readibility https://fedorahosted.org/freeipa/ticket/2174
2012-02-03 02:38:16 -06:00
ipa migrate-ds --group-overwrite-gid \\
--user-container='cn=users,cn=accounts' \\
--group-container='cn=groups,cn=accounts' \\
ldap://ds.example.com:389
Migrated users or groups may have object class and accompanied attributes
unknown to the IPA v2 server. These object classes and attributes may be
left out of the migration process:
ipa migrate-ds --user-container='cn=users,cn=accounts' \\
--group-container='cn=groups,cn=accounts' \\
--user-ignore-objectclass=radiusprofile \\
--user-ignore-attribute=radiusgroupname \\
ldap://ds.example.com:389
ticket 1669 - improve i18n docstring extraction This patch reverts the use of pygettext for i18n string extraction. It was originally introduced because the help documentation for commands are in the class docstring and module docstring. Docstrings are a Python construct whereby any string which immediately follows a class declaration, function/method declaration or appears first in a module is taken to be the documentation for that object. Python automatically assigns that string to the __doc__ variable associated with the object. Explicitly assigning to the __doc__ variable is equivalent and permitted. We mark strings in the source for i18n translation by embedding them in _() or ngettext(). Specialized extraction tools (e.g. xgettext) scan the source code looking for strings with those markers and extracts the string for inclusion in a translation catalog. It was mistakingly assumed one could not mark for translation Python docstrings. Since some docstrings are vital for our command help system some method had to be devised to extract docstrings for the translation catalog. pygettext has the ability to locate and extract docstrings and it was introduced to acquire the documentation for our commands located in module and class docstrings. However pygettext was too large a hammer for this task, it lacked any fined grained ability to extract only the docstrings we were interested in. In practice it extracted EVERY docstring in each file it was presented with. This caused a large number strings to be extracted for translation which had no reason to be translated, the string might have been internal code documentation never meant to be seen by users. Often the superfluous docstrings were long, complex and likely difficult to translate. This placed an unnecessary burden on our volunteer translators. Instead what is needed is some method to extract only those strings intended for translation. We already have such a mechanism and it is already widely used, namely wrapping strings intended for translation in calls to _() or _negettext(), i.e. marking a string for i18n translation. Thus the solution to the docstring translation problem is to mark the docstrings exactly as we have been doing, it only requires that instead of a bare Python docstring we instead assign the marked string to the __doc__ variable. Using the hypothetical class foo as an example. class foo(Command): ''' The foo command takes out the garbage. ''' Would become: class foo(Command): __doc__ = _('The foo command takes out the garbage.') But which docstrings need to be marked for translation? The makeapi tool knows how to iterate over every command in our public API. It was extended to validate every command's documentation and report if any documentation is missing or not marked for translation. That information was then used to identify each docstring in the code which needed to be transformed. In summary what this patch does is: * Remove the use of pygettext (modification to install/po/Makefile.in) * Replace every docstring with an explicit assignment to __doc__ where the rhs of the assignment is an i18n marking function. * Single line docstrings appearing in multi-line string literals (e.g. ''' or """) were replaced with single line string literals because the multi-line literals were introducing unnecessary whitespace and newlines in the string extracted for translation. For example: ''' The foo command takes out the garbage. ''' Would appear in the translation catalog as: "\n The foo command takes out the garbage.\n " The superfluous whitespace and newlines are confusing to translators and requires us to strip leading and trailing whitespace from the translation at run time. * Import statements were moved from below the docstring to above it. This was necessary because the i18n markers are imported functions and must be available before the the doc is parsed. Technically only the import of the i18n markers had to appear before the doc but stylistically it's better to keep all the imports together. * It was observed during the docstring editing process that the command documentation was inconsistent with respect to the use of periods to terminate a sentence. Some doc had a trailing period, others didn't. Consistency was enforced by adding a period to end of every docstring if one was missing.
2011-08-24 21:48:30 -05:00
""")
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
# USER MIGRATION CALLBACKS AND VARS
localize doc strings A number of doc strings were not localized, wrap them in _(). Some messages were not localized, wrap them in _() Fix a couple of failing tests: The method name in RPC should not be unicode. The doc attribute must use the .msg attribute for comparison. Also clean up imports of _() The import should come from ipalib or ipalib.text, not ugettext from request.
2010-03-05 15:11:21 -06:00
_krb_err_msg = _('Kerberos principal %s already exists. Use \'ipa user-mod\' to set it manually.')
_grp_err_msg = _('Failed to add user to the default group. Use \'ipa group-add-member\' to add manually.')
Handle LDAP search references LDAP search operation may return a search reference pointing to an LDAP resource. As the framework does not handle search references, skip these results to prevent result processing failures. Migrate operation crashed when the migrated DS contained search references. Now, it correctly skips these records and prints the failed references to user. https://fedorahosted.org/freeipa/ticket/1209
2011-06-01 11:04:24 -05:00
_ref_err_msg = _('Migration of LDAP search reference is not supported.')
migrate process cannot handle multivalued pkey attribute When group/user is migrated, the attribute used for RDN may be multivalued. Make sure that we pick the value used in the RDN which should be the unique one and not just the first one. https://fedorahosted.org/freeipa/ticket/1892
2011-09-29 04:55:13 -05:00
_dn_err_msg = _('Malformed DN')
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
_supported_schemas = (u'RFC2307bis', u'RFC2307')
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
Migration warning when compat enabled Added check into migration plugin to warn user when compat is enabled. If compat is enabled, the migration fails and user is warned that he must turn the compat off or run the script with (the newly introduced) option '--with-compat'. '--with-compat' is new flag. If it is set, the compat status is ignored. https://fedorahosted.org/freeipa/ticket/2274
2012-03-01 04:41:53 -06:00
_compat_dn = "cn=Schema Compatibility,cn=plugins,cn=config"
Fix attributes that contain DNs when migrating. Some attributes, like secretary and manager, may point to other LDAP entries. We need to fix these during migration. https://fedorahosted.org/freeipa/ticket/2562
2012-03-20 21:37:27 -05:00
def is_DN_syntax(ldap, attr):
"""
Check the schema to see if the attribute uses DN syntax.
Returns True/False
"""
obj = ldap.schema.get_obj(_ldap.schema.AttributeType, attr)
return obj is not None and obj.syntax == '1.3.6.1.4.1.1466.115.121.1.12'
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
def _pre_migrate_user(ldap, pkey, dn, entry_attrs, failed, config, ctx, **kwargs):
Fix migration to work between v2 servers and remove search/size limits. Migration from a v2 server would fail because of our fake memberofindirect attribute. This isn't in any objectclass so would cause entries to fail to migrate. We can safely just remove it. Also remove any limits on time/size when searching for entries on the remote server. Otherwise only the number of entries configured in the local IPA server can be migrated. ticket 1124
2011-04-08 09:19:42 -05:00
attr_blacklist = ['krbprincipalkey','memberofindirect','memberindirect']
Add ignore lists to migrate-ds command When user migrates users/groups from an old DS instance, the migration may fail on unsupported object classes and/or relevant LDAP object attributes. This patch implements a support for object class and attribute ignore lists that can be used to suppress these migration issues. Additionally, a redundant "dev/null" file is removed from git repo (originally added in 26b0e8fc9809a4cd9f2f9a2281f0894e2e0f8db2). https://fedorahosted.org/freeipa/ticket/1266
2011-06-03 07:21:43 -05:00
attr_blacklist.extend(kwargs.get('attr_blacklist', []))
Don't create private groups for migrated users, check for valid gidnumber Migrated users don't get a private group, there is no safe way to verify that the namespace is correct without redoing the uidnumber as well. Verify that the GID at least points to a valid group on the remote server and warn if it doesn't (this doesn't guarantee that the group gets migrated but at least we try). If the remote entry has no gidNumber then don't migrate that user. We don't know why that user is non-POSIX, it could be a special user used for auth, for example. Add a loginshell if the remote user doesn't have one. https://fedorahosted.org/freeipa/ticket/2562
2012-03-22 12:40:54 -05:00
ds_ldap = ctx['ds_ldap']
has_upg = ctx['has_upg']
search_bases = kwargs.get('search_bases', None)
valid_gids = kwargs['valid_gids']
Do not migrate krbPrincipalKey https://fedorahosted.org/freeipa/ticket/455
2010-11-26 08:37:12 -06:00
Don't create private groups for migrated users, check for valid gidnumber Migrated users don't get a private group, there is no safe way to verify that the namespace is correct without redoing the uidnumber as well. Verify that the GID at least points to a valid group on the remote server and warn if it doesn't (this doesn't guarantee that the group gets migrated but at least we try). If the remote entry has no gidNumber then don't migrate that user. We don't know why that user is non-POSIX, it could be a special user used for auth, for example. Add a loginshell if the remote user doesn't have one. https://fedorahosted.org/freeipa/ticket/2562
2012-03-22 12:40:54 -05:00
if 'gidnumber' not in entry_attrs:
raise errors.NotFound(reason=_('%(user)s is not a POSIX user') % dict(user=pkey))
else:
# See if the gidNumber at least points to a valid group on the remote
# server.
if entry_attrs['gidnumber'][0] not in valid_gids:
try:
(remote_dn, remote_entry) = ds_ldap.find_entry_by_attr(
'gidnumber', entry_attrs['gidnumber'][0], 'posixgroup',
[''], search_bases['group']
)
valid_gids.append(entry_attrs['gidnumber'][0])
except errors.NotFound:
api.log.warn('Migrated user\'s GID number %s does not point to a known group.' % entry_attrs['gidnumber'][0])
# We don't want to create a UPG so set the magic value in description
# to let the DS plugin know.
entry_attrs.setdefault('description', [])
entry_attrs['description'].append(NO_UPG_MAGIC)
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
# fill in required attributes by IPA
UUIDs: remove uuid python plugin and let DS always autogenerate merge in remove uuid
2010-10-26 09:26:06 -05:00
entry_attrs['ipauniqueid'] = 'autogenerate'
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
if 'homedirectory' not in entry_attrs:
homes_root = config.get('ipahomesrootdir', ('/home', ))[0]
home_dir = '%s/%s' % (homes_root, pkey)
home_dir = home_dir.replace('//', '/').rstrip('/')
entry_attrs['homedirectory'] = home_dir
Don't create private groups for migrated users, check for valid gidnumber Migrated users don't get a private group, there is no safe way to verify that the namespace is correct without redoing the uidnumber as well. Verify that the GID at least points to a valid group on the remote server and warn if it doesn't (this doesn't guarantee that the group gets migrated but at least we try). If the remote entry has no gidNumber then don't migrate that user. We don't know why that user is non-POSIX, it could be a special user used for auth, for example. Add a loginshell if the remote user doesn't have one. https://fedorahosted.org/freeipa/ticket/2562
2012-03-22 12:40:54 -05:00
if 'loginshell' not in entry_attrs:
default_shell = config.get('ipadefaultloginshell', ['/bin/sh'])[0]
entry_attrs.setdefault('loginshell', default_shell)
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
Fix migration to work between v2 servers and remove search/size limits. Migration from a v2 server would fail because of our fake memberofindirect attribute. This isn't in any objectclass so would cause entries to fail to migrate. We can safely just remove it. Also remove any limits on time/size when searching for entries on the remote server. Otherwise only the number of entries configured in the local IPA server can be migrated. ticket 1124
2011-04-08 09:19:42 -05:00
# do not migrate all attributes
Do not migrate krbPrincipalKey https://fedorahosted.org/freeipa/ticket/455
2010-11-26 08:37:12 -06:00
for attr in entry_attrs.keys():
if attr in attr_blacklist:
del entry_attrs[attr]
Add ignore lists to migrate-ds command When user migrates users/groups from an old DS instance, the migration may fail on unsupported object classes and/or relevant LDAP object attributes. This patch implements a support for object class and attribute ignore lists that can be used to suppress these migration issues. Additionally, a redundant "dev/null" file is removed from git repo (originally added in 26b0e8fc9809a4cd9f2f9a2281f0894e2e0f8db2). https://fedorahosted.org/freeipa/ticket/1266
2011-06-03 07:21:43 -05:00
# do not migrate all object classes
if 'objectclass' in entry_attrs:
for object_class in kwargs.get('oc_blacklist', []):
try:
entry_attrs['objectclass'].remove(object_class)
except ValueError: # object class not present
pass
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
# generate a principal name and check if it isn't already taken
Make error message in migration plugin unicode.
2010-02-17 11:32:58 -06:00
principal = u'%s@%s' % (pkey, api.env.realm)
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
try:
ldap.find_entry_by_attr(
'krbprincipalname', principal, 'krbprincipalaux', ['']
)
except errors.NotFound:
entry_attrs['krbprincipalname'] = principal
else:
Handle LDAP search references LDAP search operation may return a search reference pointing to an LDAP resource. As the framework does not handle search references, skip these results to prevent result processing failures. Migrate operation crashed when the migrated DS contained search references. Now, it correctly skips these records and prints the failed references to user. https://fedorahosted.org/freeipa/ticket/1209
2011-06-01 11:04:24 -05:00
failed[pkey] = unicode(_krb_err_msg % principal)
Translatable Param.label, Param.doc
2010-02-19 10:08:16 -06:00
Fix attributes that contain DNs when migrating. Some attributes, like secretary and manager, may point to other LDAP entries. We need to fix these during migration. https://fedorahosted.org/freeipa/ticket/2562
2012-03-20 21:37:27 -05:00
# Fix any attributes with DN syntax that point to entries in the old
# tree
for attr in entry_attrs.keys():
if is_DN_syntax(ldap, attr):
for ind, value in enumerate(entry_attrs[attr]):
try:
(remote_dn, remote_entry) = ds_ldap.get_entry(value, [api.Object.user.primary_key.name, api.Object.group.primary_key.name])
except errors.NotFound:
Don't create private groups for migrated users, check for valid gidnumber Migrated users don't get a private group, there is no safe way to verify that the namespace is correct without redoing the uidnumber as well. Verify that the GID at least points to a valid group on the remote server and warn if it doesn't (this doesn't guarantee that the group gets migrated but at least we try). If the remote entry has no gidNumber then don't migrate that user. We don't know why that user is non-POSIX, it could be a special user used for auth, for example. Add a loginshell if the remote user doesn't have one. https://fedorahosted.org/freeipa/ticket/2562
2012-03-22 12:40:54 -05:00
api.log.warn('%s: attribute %s refers to non-existent entry %s' % (pkey, attr, value))
Fix attributes that contain DNs when migrating. Some attributes, like secretary and manager, may point to other LDAP entries. We need to fix these during migration. https://fedorahosted.org/freeipa/ticket/2562
2012-03-20 21:37:27 -05:00
continue
if value.lower().endswith(search_bases['user']):
primary_key = api.Object.user.primary_key.name
container = api.env.container_user
elif value.lower().endswith(search_bases['group']):
primary_key = api.Object.group.primary_key.name
container = api.env.container_group
else:
Don't create private groups for migrated users, check for valid gidnumber Migrated users don't get a private group, there is no safe way to verify that the namespace is correct without redoing the uidnumber as well. Verify that the GID at least points to a valid group on the remote server and warn if it doesn't (this doesn't guarantee that the group gets migrated but at least we try). If the remote entry has no gidNumber then don't migrate that user. We don't know why that user is non-POSIX, it could be a special user used for auth, for example. Add a loginshell if the remote user doesn't have one. https://fedorahosted.org/freeipa/ticket/2562
2012-03-22 12:40:54 -05:00
api.log.warn('%s: value %s in attribute %s does not belong into any known container' % (pkey, value, attr))
Fix attributes that contain DNs when migrating. Some attributes, like secretary and manager, may point to other LDAP entries. We need to fix these during migration. https://fedorahosted.org/freeipa/ticket/2562
2012-03-20 21:37:27 -05:00
continue
if not remote_entry.get(primary_key):
Don't create private groups for migrated users, check for valid gidnumber Migrated users don't get a private group, there is no safe way to verify that the namespace is correct without redoing the uidnumber as well. Verify that the GID at least points to a valid group on the remote server and warn if it doesn't (this doesn't guarantee that the group gets migrated but at least we try). If the remote entry has no gidNumber then don't migrate that user. We don't know why that user is non-POSIX, it could be a special user used for auth, for example. Add a loginshell if the remote user doesn't have one. https://fedorahosted.org/freeipa/ticket/2562
2012-03-22 12:40:54 -05:00
api.log.warn('%s: there is no primary key %s to migrate for %s' % (pkey, primary_key, attr))
Fix attributes that contain DNs when migrating. Some attributes, like secretary and manager, may point to other LDAP entries. We need to fix these during migration. https://fedorahosted.org/freeipa/ticket/2562
2012-03-20 21:37:27 -05:00
continue
Don't create private groups for migrated users, check for valid gidnumber Migrated users don't get a private group, there is no safe way to verify that the namespace is correct without redoing the uidnumber as well. Verify that the GID at least points to a valid group on the remote server and warn if it doesn't (this doesn't guarantee that the group gets migrated but at least we try). If the remote entry has no gidNumber then don't migrate that user. We don't know why that user is non-POSIX, it could be a special user used for auth, for example. Add a loginshell if the remote user doesn't have one. https://fedorahosted.org/freeipa/ticket/2562
2012-03-22 12:40:54 -05:00
api.log.debug('converting DN value %s for %s in %s' % (value, attr, dn))
Fix attributes that contain DNs when migrating. Some attributes, like secretary and manager, may point to other LDAP entries. We need to fix these during migration. https://fedorahosted.org/freeipa/ticket/2562
2012-03-20 21:37:27 -05:00
rdnval = remote_entry[primary_key][0].lower()
entry_attrs[attr][ind] = \
str(DN((primary_key, rdnval),
container,
api.env.basedn))
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
return dn
def _post_migrate_user(ldap, pkey, dn, entry_attrs, failed, config, ctx):
# add user to the default group
try:
ldap.add_entry_to_group(dn, ctx['def_group_dn'])
except errors.ExecutionError, e:
Handle LDAP search references LDAP search operation may return a search reference pointing to an LDAP resource. As the framework does not handle search references, skip these results to prevent result processing failures. Migrate operation crashed when the migrated DS contained search references. Now, it correctly skips these records and prints the failed references to user. https://fedorahosted.org/freeipa/ticket/1209
2011-06-01 11:04:24 -05:00
failed[pkey] = unicode(_grp_err_msg)
Don't create private groups for migrated users, check for valid gidnumber Migrated users don't get a private group, there is no safe way to verify that the namespace is correct without redoing the uidnumber as well. Verify that the GID at least points to a valid group on the remote server and warn if it doesn't (this doesn't guarantee that the group gets migrated but at least we try). If the remote entry has no gidNumber then don't migrate that user. We don't know why that user is non-POSIX, it could be a special user used for auth, for example. Add a loginshell if the remote user doesn't have one. https://fedorahosted.org/freeipa/ticket/2562
2012-03-22 12:40:54 -05:00
if 'description' in entry_attrs and NO_UPG_MAGIC in entry_attrs['description']:
entry_attrs['description'].remove(NO_UPG_MAGIC)
kw = {'setattr': unicode('description=%s' % ','.join(entry_attrs['description']))}
try:
api.Command['user_mod'](pkey, **kw)
except (errors.EmptyModlist, errors.NotFound):
pass
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
# GROUP MIGRATION CALLBACKS AND VARS
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
def _pre_migrate_group(ldap, pkey, dn, entry_attrs, failed, config, ctx, **kwargs):
def convert_members_rfc2307bis(member_attr, search_bases, overwrite=False):
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
"""
Convert DNs in member attributes to work in IPA.
"""
new_members = []
entry_attrs.setdefault(member_attr, [])
Translatable Param.label, Param.doc
2010-02-19 10:08:16 -06:00
for m in entry_attrs[member_attr]:
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
try:
# what str2dn returns looks like [[('cn', 'foo', 4)], [('dc', 'example', 1)], [('dc', 'com', 1)]]
rdn = _ldap.dn.str2dn(m ,flags=_ldap.DN_FORMAT_LDAPV3)[0]
rdnval = rdn[0][1]
except IndexError:
api.log.error('Malformed DN %s has no RDN?' % m)
continue
if m.lower().endswith(search_bases['user']):
api.log.info('migrating user %s' % m)
m = '%s=%s,%s' % (api.Object.user.primary_key.name,
rdnval,
api.env.container_user)
elif m.lower().endswith(search_bases['group']):
api.log.info('migrating group %s' % m)
m = '%s=%s,%s' % (api.Object.group.primary_key.name,
rdnval,
api.env.container_group)
else:
api.log.error('entry %s does not belong into any known container' % m)
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
continue
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
m = ldap.normalize_dn(m)
new_members.append(m)
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
del entry_attrs[member_attr]
if overwrite:
entry_attrs['member'] = []
entry_attrs['member'] += new_members
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
def convert_members_rfc2307(member_attr):
"""
Convert usernames in member attributes to work in IPA.
"""
new_members = []
entry_attrs.setdefault(member_attr, [])
for m in entry_attrs[member_attr]:
memberdn = '%s=%s,%s' % (api.Object.user.primary_key.name,
m,
api.env.container_user)
new_members.append(ldap.normalize_dn(memberdn))
entry_attrs['member'] = new_members
Fix migration to work between v2 servers and remove search/size limits. Migration from a v2 server would fail because of our fake memberofindirect attribute. This isn't in any objectclass so would cause entries to fail to migrate. We can safely just remove it. Also remove any limits on time/size when searching for entries on the remote server. Otherwise only the number of entries configured in the local IPA server can be migrated. ticket 1124
2011-04-08 09:19:42 -05:00
attr_blacklist = ['memberofindirect','memberindirect']
Add ignore lists to migrate-ds command When user migrates users/groups from an old DS instance, the migration may fail on unsupported object classes and/or relevant LDAP object attributes. This patch implements a support for object class and attribute ignore lists that can be used to suppress these migration issues. Additionally, a redundant "dev/null" file is removed from git repo (originally added in 26b0e8fc9809a4cd9f2f9a2281f0894e2e0f8db2). https://fedorahosted.org/freeipa/ticket/1266
2011-06-03 07:21:43 -05:00
attr_blacklist.extend(kwargs.get('attr_blacklist', []))
Fix migration to work between v2 servers and remove search/size limits. Migration from a v2 server would fail because of our fake memberofindirect attribute. This isn't in any objectclass so would cause entries to fail to migrate. We can safely just remove it. Also remove any limits on time/size when searching for entries on the remote server. Otherwise only the number of entries configured in the local IPA server can be migrated. ticket 1124
2011-04-08 09:19:42 -05:00
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
schema = kwargs.get('schema', None)
UUIDs: remove uuid python plugin and let DS always autogenerate merge in remove uuid
2010-10-26 09:26:06 -05:00
entry_attrs['ipauniqueid'] = 'autogenerate'
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
if schema == 'RFC2307bis':
search_bases = kwargs.get('search_bases', None)
if not search_bases:
raise ValueError('Search bases not specified')
convert_members_rfc2307bis('member', search_bases, overwrite=True)
convert_members_rfc2307bis('uniquemember', search_bases)
elif schema == 'RFC2307':
convert_members_rfc2307('memberuid')
else:
raise ValueError('Schema %s not supported' % schema)
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
Fix migration to work between v2 servers and remove search/size limits. Migration from a v2 server would fail because of our fake memberofindirect attribute. This isn't in any objectclass so would cause entries to fail to migrate. We can safely just remove it. Also remove any limits on time/size when searching for entries on the remote server. Otherwise only the number of entries configured in the local IPA server can be migrated. ticket 1124
2011-04-08 09:19:42 -05:00
# do not migrate all attributes
for attr in entry_attrs.keys():
if attr in attr_blacklist:
del entry_attrs[attr]
Add ignore lists to migrate-ds command When user migrates users/groups from an old DS instance, the migration may fail on unsupported object classes and/or relevant LDAP object attributes. This patch implements a support for object class and attribute ignore lists that can be used to suppress these migration issues. Additionally, a redundant "dev/null" file is removed from git repo (originally added in 26b0e8fc9809a4cd9f2f9a2281f0894e2e0f8db2). https://fedorahosted.org/freeipa/ticket/1266
2011-06-03 07:21:43 -05:00
# do not migrate all object classes
if 'objectclass' in entry_attrs:
for object_class in kwargs.get('oc_blacklist', []):
try:
entry_attrs['objectclass'].remove(object_class)
except ValueError: # object class not present
pass
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
return dn
Improve handling of GIDs when migrating groups Since IPA v2 server already contain predefined groups that may collide with groups in migrated (IPA v1) server (for example admins, ipausers), users having colliding group as their primary group may happen to belong to an unknown group on new IPA v2 server. Implement --group-overwrite-gid option to overwrite GID of already existing groups to prevent this issue. https://fedorahosted.org/freeipa/ticket/1866
2011-10-03 09:01:01 -05:00
def _group_exc_callback(ldap, dn, entry_attrs, exc, options):
if isinstance(exc, errors.DuplicateEntry):
if options.get('groupoverwritegid', False) and \
entry_attrs.get('gidnumber') is not None:
try:
new_entry_attrs = {'gidnumber':entry_attrs['gidnumber']}
ldap.update_entry(dn, new_entry_attrs)
except errors.EmptyModlist:
# no change to the GID
pass
# mark as success
return
elif not options.get('groupoverwritegid', False) and \
entry_attrs.get('gidnumber') is not None:
msg = unicode(exc)
# add information about possibility to overwrite GID
msg = msg + unicode(_('. Check GID of the existing group. ' \
'Use --group-overwrite-gid option to overwrite the GID'))
raise errors.DuplicateEntry(message=msg)
raise exc
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
# DS MIGRATION PLUGIN
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
def construct_filter(template, oc_list):
oc_subfilter = ''.join([ '(objectclass=%s)' % oc for oc in oc_list])
return template % oc_subfilter
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
def validate_ldapuri(ugettext, ldapuri):
m = re.match('^ldaps?://[-\w\.]+(:\d+)?$', ldapuri)
if not m:
Add labels for passwords, fix output of exceptions, fix passwd output. Passwords didn't have internationalizable labels. Exceptions that occured during required input weren't printed as unicode so weren't being translated properly. Don't use output_for_cli() directly in the passwd plugin, use output.Output. ticket 352
2010-12-02 15:31:42 -06:00
err_msg = _('Invalid LDAP URI.')
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
raise errors.ValidationError(name='ldap_uri', error=err_msg)
class migrate_ds(Command):
ticket 1669 - improve i18n docstring extraction This patch reverts the use of pygettext for i18n string extraction. It was originally introduced because the help documentation for commands are in the class docstring and module docstring. Docstrings are a Python construct whereby any string which immediately follows a class declaration, function/method declaration or appears first in a module is taken to be the documentation for that object. Python automatically assigns that string to the __doc__ variable associated with the object. Explicitly assigning to the __doc__ variable is equivalent and permitted. We mark strings in the source for i18n translation by embedding them in _() or ngettext(). Specialized extraction tools (e.g. xgettext) scan the source code looking for strings with those markers and extracts the string for inclusion in a translation catalog. It was mistakingly assumed one could not mark for translation Python docstrings. Since some docstrings are vital for our command help system some method had to be devised to extract docstrings for the translation catalog. pygettext has the ability to locate and extract docstrings and it was introduced to acquire the documentation for our commands located in module and class docstrings. However pygettext was too large a hammer for this task, it lacked any fined grained ability to extract only the docstrings we were interested in. In practice it extracted EVERY docstring in each file it was presented with. This caused a large number strings to be extracted for translation which had no reason to be translated, the string might have been internal code documentation never meant to be seen by users. Often the superfluous docstrings were long, complex and likely difficult to translate. This placed an unnecessary burden on our volunteer translators. Instead what is needed is some method to extract only those strings intended for translation. We already have such a mechanism and it is already widely used, namely wrapping strings intended for translation in calls to _() or _negettext(), i.e. marking a string for i18n translation. Thus the solution to the docstring translation problem is to mark the docstrings exactly as we have been doing, it only requires that instead of a bare Python docstring we instead assign the marked string to the __doc__ variable. Using the hypothetical class foo as an example. class foo(Command): ''' The foo command takes out the garbage. ''' Would become: class foo(Command): __doc__ = _('The foo command takes out the garbage.') But which docstrings need to be marked for translation? The makeapi tool knows how to iterate over every command in our public API. It was extended to validate every command's documentation and report if any documentation is missing or not marked for translation. That information was then used to identify each docstring in the code which needed to be transformed. In summary what this patch does is: * Remove the use of pygettext (modification to install/po/Makefile.in) * Replace every docstring with an explicit assignment to __doc__ where the rhs of the assignment is an i18n marking function. * Single line docstrings appearing in multi-line string literals (e.g. ''' or """) were replaced with single line string literals because the multi-line literals were introducing unnecessary whitespace and newlines in the string extracted for translation. For example: ''' The foo command takes out the garbage. ''' Would appear in the translation catalog as: "\n The foo command takes out the garbage.\n " The superfluous whitespace and newlines are confusing to translators and requires us to strip leading and trailing whitespace from the translation at run time. * Import statements were moved from below the docstring to above it. This was necessary because the i18n markers are imported functions and must be available before the the doc is parsed. Technically only the import of the i18n markers had to appear before the doc but stylistically it's better to keep all the imports together. * It was observed during the docstring editing process that the command documentation was inconsistent with respect to the use of periods to terminate a sentence. Some doc had a trailing period, others didn't. Consistency was enforced by adding a period to end of every docstring if one was missing.
2011-08-24 21:48:30 -05:00
__doc__ = _('Migrate users and groups from DS to IPA.')
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
migrate_objects = {
# OBJECT_NAME: (search_filter, pre_callback, post_callback)
#
# OBJECT_NAME - is the name of an LDAPObject subclass
# search_filter - is the filter to retrieve objects from DS
# pre_callback - is called for each object just after it was
# retrieved from DS and before being added to IPA
# post_callback - is called for each object after it was added to IPA
Improve handling of GIDs when migrating groups Since IPA v2 server already contain predefined groups that may collide with groups in migrated (IPA v1) server (for example admins, ipausers), users having colliding group as their primary group may happen to belong to an unknown group on new IPA v2 server. Implement --group-overwrite-gid option to overwrite GID of already existing groups to prevent this issue. https://fedorahosted.org/freeipa/ticket/1866
2011-10-03 09:01:01 -05:00
# exc_callback - is called when adding entry to IPA raises an exception
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
#
# {pre, post}_callback parameters:
# ldap - ldap2 instance connected to IPA
# pkey - primary key value of the object (uid for users, etc.)
# dn - dn of the object as it (will be/is) stored in IPA
# entry_attrs - attributes of the object
# failed - a list of so-far failed objects
# config - IPA config entry attributes
# ctx - object context, used to pass data between callbacks
#
# If pre_callback return value evaluates to False, migration
# of the current object is aborted.
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
'user': {
'filter_template' : '(&(|%s)(uid=*))',
'oc_option' : 'userobjectclass',
Add ignore lists to migrate-ds command When user migrates users/groups from an old DS instance, the migration may fail on unsupported object classes and/or relevant LDAP object attributes. This patch implements a support for object class and attribute ignore lists that can be used to suppress these migration issues. Additionally, a redundant "dev/null" file is removed from git repo (originally added in 26b0e8fc9809a4cd9f2f9a2281f0894e2e0f8db2). https://fedorahosted.org/freeipa/ticket/1266
2011-06-03 07:21:43 -05:00
'oc_blacklist_option' : 'userignoreobjectclass',
'attr_blacklist_option' : 'userignoreattribute',
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
'pre_callback' : _pre_migrate_user,
Improve handling of GIDs when migrating groups Since IPA v2 server already contain predefined groups that may collide with groups in migrated (IPA v1) server (for example admins, ipausers), users having colliding group as their primary group may happen to belong to an unknown group on new IPA v2 server. Implement --group-overwrite-gid option to overwrite GID of already existing groups to prevent this issue. https://fedorahosted.org/freeipa/ticket/1866
2011-10-03 09:01:01 -05:00
'post_callback' : _post_migrate_user,
'exc_callback' : None
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
},
'group': {
'filter_template' : '(&(|%s)(cn=*))',
'oc_option' : 'groupobjectclass',
Add ignore lists to migrate-ds command When user migrates users/groups from an old DS instance, the migration may fail on unsupported object classes and/or relevant LDAP object attributes. This patch implements a support for object class and attribute ignore lists that can be used to suppress these migration issues. Additionally, a redundant "dev/null" file is removed from git repo (originally added in 26b0e8fc9809a4cd9f2f9a2281f0894e2e0f8db2). https://fedorahosted.org/freeipa/ticket/1266
2011-06-03 07:21:43 -05:00
'oc_blacklist_option' : 'groupignoreobjectclass',
'attr_blacklist_option' : 'groupignoreattribute',
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
'pre_callback' : _pre_migrate_group,
Improve handling of GIDs when migrating groups Since IPA v2 server already contain predefined groups that may collide with groups in migrated (IPA v1) server (for example admins, ipausers), users having colliding group as their primary group may happen to belong to an unknown group on new IPA v2 server. Implement --group-overwrite-gid option to overwrite GID of already existing groups to prevent this issue. https://fedorahosted.org/freeipa/ticket/1866
2011-10-03 09:01:01 -05:00
'post_callback' : None,
'exc_callback' : _group_exc_callback,
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
},
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
}
migrate_order = ('user', 'group')
takes_args = (
Str('ldapuri', validate_ldapuri,
cli_name='ldap_uri',
Translatable Param.label, Param.doc
2010-02-19 10:08:16 -06:00
label=_('LDAP URI'),
doc=_('LDAP URI of DS server to migrate from'),
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
),
Password('bindpw',
cli_name='password',
Add labels for passwords, fix output of exceptions, fix passwd output. Passwords didn't have internationalizable labels. Exceptions that occured during required input weren't printed as unicode so weren't being translated properly. Don't use output_for_cli() directly in the passwd plugin, use output.Output. ticket 352
2010-12-02 15:31:42 -06:00
label=_('Password'),
Improve handling of GIDs when migrating groups Since IPA v2 server already contain predefined groups that may collide with groups in migrated (IPA v1) server (for example admins, ipausers), users having colliding group as their primary group may happen to belong to an unknown group on new IPA v2 server. Implement --group-overwrite-gid option to overwrite GID of already existing groups to prevent this issue. https://fedorahosted.org/freeipa/ticket/1866
2011-10-03 09:01:01 -05:00
confirm=False,
localize doc strings A number of doc strings were not localized, wrap them in _(). Some messages were not localized, wrap them in _() Fix a couple of failing tests: The method name in RPC should not be unicode. The doc attribute must use the .msg attribute for comparison. Also clean up imports of _() The import should come from ipalib or ipalib.text, not ugettext from request.
2010-03-05 15:11:21 -06:00
doc=_('bind password'),
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
),
)
takes_options = (
Validate DN & RDN parameters for migrate command Ticket #2555 We were generating a traceback (server error) if a malformed RDN was passed as a parameter to the migrate command. * add parameter validation functions validate_dn_param() and validate_rdn_param() to ipalib.util. Those functions simply invoke the DN or RDN constructor from our dn module passing it the string representation. If the constructor does not throw an error it's valid. * Add the parameter validation function pointers to the Param objects in the migrate command. * Make the usercontainer and groupcontainer parameters required. passing --usercontainer= on the command line will produce ipa: ERROR: 'user_container' is required * Fix _get_search_bases() so if a container dn is empty it it just uses the base dn alone instead of faulting (currently bullet-proofing because now the containers are required). * Update the doc for usercontainer and groupcontainer to reflect the fact they are DN's not RDN's. A RDN can only be one level and it should be possible to have a container more than one RDN removed from the base.
2012-04-16 01:33:26 -05:00
Str('binddn?', validate_dn_param,
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
cli_name='bind_dn',
Translatable Param.label, Param.doc
2010-02-19 10:08:16 -06:00
label=_('Bind DN'),
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
default=u'cn=directory manager',
autofill=True,
),
Validate DN & RDN parameters for migrate command Ticket #2555 We were generating a traceback (server error) if a malformed RDN was passed as a parameter to the migrate command. * add parameter validation functions validate_dn_param() and validate_rdn_param() to ipalib.util. Those functions simply invoke the DN or RDN constructor from our dn module passing it the string representation. If the constructor does not throw an error it's valid. * Add the parameter validation function pointers to the Param objects in the migrate command. * Make the usercontainer and groupcontainer parameters required. passing --usercontainer= on the command line will produce ipa: ERROR: 'user_container' is required * Fix _get_search_bases() so if a container dn is empty it it just uses the base dn alone instead of faulting (currently bullet-proofing because now the containers are required). * Update the doc for usercontainer and groupcontainer to reflect the fact they are DN's not RDN's. A RDN can only be one level and it should be possible to have a container more than one RDN removed from the base.
2012-04-16 01:33:26 -05:00
Str('usercontainer', validate_dn_param,
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
cli_name='user_container',
Translatable Param.label, Param.doc
2010-02-19 10:08:16 -06:00
label=_('User container'),
Validate DN & RDN parameters for migrate command Ticket #2555 We were generating a traceback (server error) if a malformed RDN was passed as a parameter to the migrate command. * add parameter validation functions validate_dn_param() and validate_rdn_param() to ipalib.util. Those functions simply invoke the DN or RDN constructor from our dn module passing it the string representation. If the constructor does not throw an error it's valid. * Add the parameter validation function pointers to the Param objects in the migrate command. * Make the usercontainer and groupcontainer parameters required. passing --usercontainer= on the command line will produce ipa: ERROR: 'user_container' is required * Fix _get_search_bases() so if a container dn is empty it it just uses the base dn alone instead of faulting (currently bullet-proofing because now the containers are required). * Update the doc for usercontainer and groupcontainer to reflect the fact they are DN's not RDN's. A RDN can only be one level and it should be possible to have a container more than one RDN removed from the base.
2012-04-16 01:33:26 -05:00
doc=_('DN of container for users in DS relative to base DN'),
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
default=u'ou=people',
autofill=True,
),
Validate DN & RDN parameters for migrate command Ticket #2555 We were generating a traceback (server error) if a malformed RDN was passed as a parameter to the migrate command. * add parameter validation functions validate_dn_param() and validate_rdn_param() to ipalib.util. Those functions simply invoke the DN or RDN constructor from our dn module passing it the string representation. If the constructor does not throw an error it's valid. * Add the parameter validation function pointers to the Param objects in the migrate command. * Make the usercontainer and groupcontainer parameters required. passing --usercontainer= on the command line will produce ipa: ERROR: 'user_container' is required * Fix _get_search_bases() so if a container dn is empty it it just uses the base dn alone instead of faulting (currently bullet-proofing because now the containers are required). * Update the doc for usercontainer and groupcontainer to reflect the fact they are DN's not RDN's. A RDN can only be one level and it should be possible to have a container more than one RDN removed from the base.
2012-04-16 01:33:26 -05:00
Str('groupcontainer', validate_dn_param,
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
cli_name='group_container',
Translatable Param.label, Param.doc
2010-02-19 10:08:16 -06:00
label=_('Group container'),
Validate DN & RDN parameters for migrate command Ticket #2555 We were generating a traceback (server error) if a malformed RDN was passed as a parameter to the migrate command. * add parameter validation functions validate_dn_param() and validate_rdn_param() to ipalib.util. Those functions simply invoke the DN or RDN constructor from our dn module passing it the string representation. If the constructor does not throw an error it's valid. * Add the parameter validation function pointers to the Param objects in the migrate command. * Make the usercontainer and groupcontainer parameters required. passing --usercontainer= on the command line will produce ipa: ERROR: 'user_container' is required * Fix _get_search_bases() so if a container dn is empty it it just uses the base dn alone instead of faulting (currently bullet-proofing because now the containers are required). * Update the doc for usercontainer and groupcontainer to reflect the fact they are DN's not RDN's. A RDN can only be one level and it should be possible to have a container more than one RDN removed from the base.
2012-04-16 01:33:26 -05:00
doc=_('DN of container for groups in DS relative to base DN'),
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
default=u'ou=groups',
autofill=True,
),
Parse comma-separated lists of values in all parameter types. This can be enabled for a specific parameter by setting the "csv" option to True. Remove "List" parameter type and replace all occurences of it with appropriate multi-valued parameter ("Str" in most cases) with csv enabled. Add new parameter type "Any", capable of holding values of any type. This is needed by the "batch" command, as "Str" is not suitable type for the "methods" parameter. ticket 2007
2011-11-21 09:50:27 -06:00
Str('userobjectclass*',
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
cli_name='user_objectclass',
label=_('User object class'),
doc=_('Comma-separated list of objectclasses used to search for user entries in DS'),
Parse comma-separated lists of values in all parameter types. This can be enabled for a specific parameter by setting the "csv" option to True. Remove "List" parameter type and replace all occurences of it with appropriate multi-valued parameter ("Str" in most cases) with csv enabled. Add new parameter type "Any", capable of holding values of any type. This is needed by the "batch" command, as "Str" is not suitable type for the "methods" parameter. ticket 2007
2011-11-21 09:50:27 -06:00
csv=True,
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
default=(u'person',),
autofill=True,
),
Parse comma-separated lists of values in all parameter types. This can be enabled for a specific parameter by setting the "csv" option to True. Remove "List" parameter type and replace all occurences of it with appropriate multi-valued parameter ("Str" in most cases) with csv enabled. Add new parameter type "Any", capable of holding values of any type. This is needed by the "batch" command, as "Str" is not suitable type for the "methods" parameter. ticket 2007
2011-11-21 09:50:27 -06:00
Str('groupobjectclass*',
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
cli_name='group_objectclass',
label=_('Group object class'),
doc=_('Comma-separated list of objectclasses used to search for group entries in DS'),
Parse comma-separated lists of values in all parameter types. This can be enabled for a specific parameter by setting the "csv" option to True. Remove "List" parameter type and replace all occurences of it with appropriate multi-valued parameter ("Str" in most cases) with csv enabled. Add new parameter type "Any", capable of holding values of any type. This is needed by the "batch" command, as "Str" is not suitable type for the "methods" parameter. ticket 2007
2011-11-21 09:50:27 -06:00
csv=True,
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
default=(u'groupOfUniqueNames', u'groupOfNames'),
autofill=True,
),
Parse comma-separated lists of values in all parameter types. This can be enabled for a specific parameter by setting the "csv" option to True. Remove "List" parameter type and replace all occurences of it with appropriate multi-valued parameter ("Str" in most cases) with csv enabled. Add new parameter type "Any", capable of holding values of any type. This is needed by the "batch" command, as "Str" is not suitable type for the "methods" parameter. ticket 2007
2011-11-21 09:50:27 -06:00
Str('userignoreobjectclass*',
Add ignore lists to migrate-ds command When user migrates users/groups from an old DS instance, the migration may fail on unsupported object classes and/or relevant LDAP object attributes. This patch implements a support for object class and attribute ignore lists that can be used to suppress these migration issues. Additionally, a redundant "dev/null" file is removed from git repo (originally added in 26b0e8fc9809a4cd9f2f9a2281f0894e2e0f8db2). https://fedorahosted.org/freeipa/ticket/1266
2011-06-03 07:21:43 -05:00
cli_name='user_ignore_objectclass',
label=_('Ignore user object class'),
doc=_('Comma-separated list of objectclasses to be ignored for user entries in DS'),
Parse comma-separated lists of values in all parameter types. This can be enabled for a specific parameter by setting the "csv" option to True. Remove "List" parameter type and replace all occurences of it with appropriate multi-valued parameter ("Str" in most cases) with csv enabled. Add new parameter type "Any", capable of holding values of any type. This is needed by the "batch" command, as "Str" is not suitable type for the "methods" parameter. ticket 2007
2011-11-21 09:50:27 -06:00
csv=True,
Add ignore lists to migrate-ds command When user migrates users/groups from an old DS instance, the migration may fail on unsupported object classes and/or relevant LDAP object attributes. This patch implements a support for object class and attribute ignore lists that can be used to suppress these migration issues. Additionally, a redundant "dev/null" file is removed from git repo (originally added in 26b0e8fc9809a4cd9f2f9a2281f0894e2e0f8db2). https://fedorahosted.org/freeipa/ticket/1266
2011-06-03 07:21:43 -05:00
default=tuple(),
autofill=True,
),
Parse comma-separated lists of values in all parameter types. This can be enabled for a specific parameter by setting the "csv" option to True. Remove "List" parameter type and replace all occurences of it with appropriate multi-valued parameter ("Str" in most cases) with csv enabled. Add new parameter type "Any", capable of holding values of any type. This is needed by the "batch" command, as "Str" is not suitable type for the "methods" parameter. ticket 2007
2011-11-21 09:50:27 -06:00
Str('userignoreattribute*',
Add ignore lists to migrate-ds command When user migrates users/groups from an old DS instance, the migration may fail on unsupported object classes and/or relevant LDAP object attributes. This patch implements a support for object class and attribute ignore lists that can be used to suppress these migration issues. Additionally, a redundant "dev/null" file is removed from git repo (originally added in 26b0e8fc9809a4cd9f2f9a2281f0894e2e0f8db2). https://fedorahosted.org/freeipa/ticket/1266
2011-06-03 07:21:43 -05:00
cli_name='user_ignore_attribute',
label=_('Ignore user attribute'),
doc=_('Comma-separated list of attributes to be ignored for user entries in DS'),
Parse comma-separated lists of values in all parameter types. This can be enabled for a specific parameter by setting the "csv" option to True. Remove "List" parameter type and replace all occurences of it with appropriate multi-valued parameter ("Str" in most cases) with csv enabled. Add new parameter type "Any", capable of holding values of any type. This is needed by the "batch" command, as "Str" is not suitable type for the "methods" parameter. ticket 2007
2011-11-21 09:50:27 -06:00
csv=True,
Add ignore lists to migrate-ds command When user migrates users/groups from an old DS instance, the migration may fail on unsupported object classes and/or relevant LDAP object attributes. This patch implements a support for object class and attribute ignore lists that can be used to suppress these migration issues. Additionally, a redundant "dev/null" file is removed from git repo (originally added in 26b0e8fc9809a4cd9f2f9a2281f0894e2e0f8db2). https://fedorahosted.org/freeipa/ticket/1266
2011-06-03 07:21:43 -05:00
default=tuple(),
autofill=True,
),
Parse comma-separated lists of values in all parameter types. This can be enabled for a specific parameter by setting the "csv" option to True. Remove "List" parameter type and replace all occurences of it with appropriate multi-valued parameter ("Str" in most cases) with csv enabled. Add new parameter type "Any", capable of holding values of any type. This is needed by the "batch" command, as "Str" is not suitable type for the "methods" parameter. ticket 2007
2011-11-21 09:50:27 -06:00
Str('groupignoreobjectclass*',
Add ignore lists to migrate-ds command When user migrates users/groups from an old DS instance, the migration may fail on unsupported object classes and/or relevant LDAP object attributes. This patch implements a support for object class and attribute ignore lists that can be used to suppress these migration issues. Additionally, a redundant "dev/null" file is removed from git repo (originally added in 26b0e8fc9809a4cd9f2f9a2281f0894e2e0f8db2). https://fedorahosted.org/freeipa/ticket/1266
2011-06-03 07:21:43 -05:00
cli_name='group_ignore_objectclass',
label=_('Ignore group object class'),
doc=_('Comma-separated list of objectclasses to be ignored for group entries in DS'),
Parse comma-separated lists of values in all parameter types. This can be enabled for a specific parameter by setting the "csv" option to True. Remove "List" parameter type and replace all occurences of it with appropriate multi-valued parameter ("Str" in most cases) with csv enabled. Add new parameter type "Any", capable of holding values of any type. This is needed by the "batch" command, as "Str" is not suitable type for the "methods" parameter. ticket 2007
2011-11-21 09:50:27 -06:00
csv=True,
Add ignore lists to migrate-ds command When user migrates users/groups from an old DS instance, the migration may fail on unsupported object classes and/or relevant LDAP object attributes. This patch implements a support for object class and attribute ignore lists that can be used to suppress these migration issues. Additionally, a redundant "dev/null" file is removed from git repo (originally added in 26b0e8fc9809a4cd9f2f9a2281f0894e2e0f8db2). https://fedorahosted.org/freeipa/ticket/1266
2011-06-03 07:21:43 -05:00
default=tuple(),
autofill=True,
),
Parse comma-separated lists of values in all parameter types. This can be enabled for a specific parameter by setting the "csv" option to True. Remove "List" parameter type and replace all occurences of it with appropriate multi-valued parameter ("Str" in most cases) with csv enabled. Add new parameter type "Any", capable of holding values of any type. This is needed by the "batch" command, as "Str" is not suitable type for the "methods" parameter. ticket 2007
2011-11-21 09:50:27 -06:00
Str('groupignoreattribute*',
Add ignore lists to migrate-ds command When user migrates users/groups from an old DS instance, the migration may fail on unsupported object classes and/or relevant LDAP object attributes. This patch implements a support for object class and attribute ignore lists that can be used to suppress these migration issues. Additionally, a redundant "dev/null" file is removed from git repo (originally added in 26b0e8fc9809a4cd9f2f9a2281f0894e2e0f8db2). https://fedorahosted.org/freeipa/ticket/1266
2011-06-03 07:21:43 -05:00
cli_name='group_ignore_attribute',
label=_('Ignore group attribute'),
doc=_('Comma-separated list of attributes to be ignored for group entries in DS'),
Parse comma-separated lists of values in all parameter types. This can be enabled for a specific parameter by setting the "csv" option to True. Remove "List" parameter type and replace all occurences of it with appropriate multi-valued parameter ("Str" in most cases) with csv enabled. Add new parameter type "Any", capable of holding values of any type. This is needed by the "batch" command, as "Str" is not suitable type for the "methods" parameter. ticket 2007
2011-11-21 09:50:27 -06:00
csv=True,
Add ignore lists to migrate-ds command When user migrates users/groups from an old DS instance, the migration may fail on unsupported object classes and/or relevant LDAP object attributes. This patch implements a support for object class and attribute ignore lists that can be used to suppress these migration issues. Additionally, a redundant "dev/null" file is removed from git repo (originally added in 26b0e8fc9809a4cd9f2f9a2281f0894e2e0f8db2). https://fedorahosted.org/freeipa/ticket/1266
2011-06-03 07:21:43 -05:00
default=tuple(),
autofill=True,
),
Improve handling of GIDs when migrating groups Since IPA v2 server already contain predefined groups that may collide with groups in migrated (IPA v1) server (for example admins, ipausers), users having colliding group as their primary group may happen to belong to an unknown group on new IPA v2 server. Implement --group-overwrite-gid option to overwrite GID of already existing groups to prevent this issue. https://fedorahosted.org/freeipa/ticket/1866
2011-10-03 09:01:01 -05:00
Flag('groupoverwritegid',
cli_name='group_overwrite_gid',
label=_('Overwrite GID'),
doc=_('When migrating a group already existing in IPA domain overwrite the '\
'group GID and report as success'),
),
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
StrEnum('schema?',
cli_name='schema',
label=_('LDAP schema'),
doc=_('The schema used on the LDAP server. Supported values are RFC2307 and RFC2307bis. The default is RFC2307bis'),
values=_supported_schemas,
default=_supported_schemas[0],
autofill=True,
),
Handle an empty base_dn and no cn=ipaconfig in the ldap2 backend, fix migration. We lacked good error messages if the user/group container you used doesn't exist. Add a --continue option so things can continue if you use a bad user/group container. This has the side-effect of letting you migrate just users or groups by using a bad container for the one you don't want. Fix a Gettext() error when displaying the migrated password message. ticket 289
2010-09-27 12:50:54 -05:00
Flag('continue?',
Improve handling of GIDs when migrating groups Since IPA v2 server already contain predefined groups that may collide with groups in migrated (IPA v1) server (for example admins, ipausers), users having colliding group as their primary group may happen to belong to an unknown group on new IPA v2 server. Implement --group-overwrite-gid option to overwrite GID of already existing groups to prevent this issue. https://fedorahosted.org/freeipa/ticket/1866
2011-10-03 09:01:01 -05:00
label=_('Continue'),
Typos in freeIPA messages This patch fixes several reported typos in IPA messages and in comments. Contributors file has been updated + the original author of the patch reporting the typos was added. https://fedorahosted.org/freeipa/ticket/848
2011-02-02 10:15:35 -06:00
doc=_('Continuous operation mode. Errors are reported but the process continues'),
Handle an empty base_dn and no cn=ipaconfig in the ldap2 backend, fix migration. We lacked good error messages if the user/group container you used doesn't exist. Add a --continue option so things can continue if you use a bad user/group container. This has the side-effect of letting you migrate just users or groups by using a bad container for the one you don't want. Fix a Gettext() error when displaying the migrated password message. ticket 289
2010-09-27 12:50:54 -05:00
default=False,
),
Add support defaultNamingContext and add --basedn to migrate-ds There are two sides to this, the server and client side. On the server side we attempt to add a defaultNamingContext on already installed servers. This will fail on older 389-ds instances but the failure is not fatal. New installations on versions of 389-ds that support this attribute will have it already defined. On the client side we need to look for both defaultNamingContext and namingContexts. We still need to check that the defaultNamingContext is an IPA server (info=IPAV2). The migration change also takes advantage of this and adds a new option which allows one to provide a basedn to use instead of trying to detect it. https://fedorahosted.org/freeipa/ticket/1919 https://fedorahosted.org/freeipa/ticket/2314
2012-01-30 15:29:32 -06:00
Str('basedn?',
cli_name='base_dn',
label=_('Base DN'),
doc=_('Base DN on remote LDAP server'),
),
Migration warning when compat enabled Added check into migration plugin to warn user when compat is enabled. If compat is enabled, the migration fails and user is warned that he must turn the compat off or run the script with (the newly introduced) option '--with-compat'. '--with-compat' is new flag. If it is set, the compat status is ignored. https://fedorahosted.org/freeipa/ticket/2274
2012-03-01 04:41:53 -06:00
Flag('compat?',
cli_name='with_compat',
label=_('Ignore compat plugin'),
doc=_('Allows migration despite the usage of compat plugin'),
default=False,
),
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
)
has_output = (
output.Output('result',
type=dict,
localize doc strings A number of doc strings were not localized, wrap them in _(). Some messages were not localized, wrap them in _() Fix a couple of failing tests: The method name in RPC should not be unicode. The doc attribute must use the .msg attribute for comparison. Also clean up imports of _() The import should come from ipalib or ipalib.text, not ugettext from request.
2010-03-05 15:11:21 -06:00
doc=_('Lists of objects migrated; categorized by type.'),
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
),
output.Output('failed',
type=dict,
localize doc strings A number of doc strings were not localized, wrap them in _(). Some messages were not localized, wrap them in _() Fix a couple of failing tests: The method name in RPC should not be unicode. The doc attribute must use the .msg attribute for comparison. Also clean up imports of _() The import should come from ipalib or ipalib.text, not ugettext from request.
2010-03-05 15:11:21 -06:00
doc=_('Lists of objects that could not be migrated; categorized by type.'),
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
),
output.Output('enabled',
type=bool,
localize doc strings A number of doc strings were not localized, wrap them in _(). Some messages were not localized, wrap them in _() Fix a couple of failing tests: The method name in RPC should not be unicode. The doc attribute must use the .msg attribute for comparison. Also clean up imports of _() The import should come from ipalib or ipalib.text, not ugettext from request.
2010-03-05 15:11:21 -06:00
doc=_('False if migration mode was disabled.'),
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
),
Migration warning when compat enabled Added check into migration plugin to warn user when compat is enabled. If compat is enabled, the migration fails and user is warned that he must turn the compat off or run the script with (the newly introduced) option '--with-compat'. '--with-compat' is new flag. If it is set, the compat status is ignored. https://fedorahosted.org/freeipa/ticket/2274
2012-03-01 04:41:53 -06:00
output.Output('compat',
type=bool,
doc=_('False if migration fails because the compatibility plug-in is enabled.'),
),
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
)
localize doc strings A number of doc strings were not localized, wrap them in _(). Some messages were not localized, wrap them in _() Fix a couple of failing tests: The method name in RPC should not be unicode. The doc attribute must use the .msg attribute for comparison. Also clean up imports of _() The import should come from ipalib or ipalib.text, not ugettext from request.
2010-03-05 15:11:21 -06:00
exclude_doc = _('comma-separated list of %s to exclude from migration')
truncated_err_msg = _('''\
search results for objects to be migrated
have been truncated by the server;
Typos in freeIPA messages This patch fixes several reported typos in IPA messages and in comments. Contributors file has been updated + the original author of the patch reporting the typos was added. https://fedorahosted.org/freeipa/ticket/848
2011-02-02 10:15:35 -06:00
migration process might be incomplete\n''')
localize doc strings A number of doc strings were not localized, wrap them in _(). Some messages were not localized, wrap them in _() Fix a couple of failing tests: The method name in RPC should not be unicode. The doc attribute must use the .msg attribute for comparison. Also clean up imports of _() The import should come from ipalib or ipalib.text, not ugettext from request.
2010-03-05 15:11:21 -06:00
migration_disabled_msg = _('''\
Migration mode is disabled. Use \'ipa config-mod\' to enable it.''')
pwd_migration_msg = _('''\
Passwords have been migrated in pre-hashed format.
IPA is unable to generate Kerberos keys unless provided
with clear text passwords. All migrated users need to
login at https://your.domain/ipa/migration/ before they
can use their Kerberos accounts.''')
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
def get_options(self):
"""
Call get_options of the baseclass and add "exclude" options
for each type of object being migrated.
"""
for option in super(migrate_ds, self).get_options():
yield option
for ldap_obj_name in self.migrate_objects:
ldap_obj = self.api.Object[ldap_obj_name]
name = 'exclude_%ss' % to_cli(ldap_obj_name)
Add ignore lists to migrate-ds command When user migrates users/groups from an old DS instance, the migration may fail on unsupported object classes and/or relevant LDAP object attributes. This patch implements a support for object class and attribute ignore lists that can be used to suppress these migration issues. Additionally, a redundant "dev/null" file is removed from git repo (originally added in 26b0e8fc9809a4cd9f2f9a2281f0894e2e0f8db2). https://fedorahosted.org/freeipa/ticket/1266
2011-06-03 07:21:43 -05:00
doc = self.exclude_doc % ldap_obj.object_name_plural
Parse comma-separated lists of values in all parameter types. This can be enabled for a specific parameter by setting the "csv" option to True. Remove "List" parameter type and replace all occurences of it with appropriate multi-valued parameter ("Str" in most cases) with csv enabled. Add new parameter type "Any", capable of holding values of any type. This is needed by the "batch" command, as "Str" is not suitable type for the "methods" parameter. ticket 2007
2011-11-21 09:50:27 -06:00
yield Str(
'%s*' % name, cli_name=name, doc=doc, csv=True,
default=tuple(), autofill=True
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
)
def normalize_options(self, options):
"""
Convert all "exclude" option values to lower-case.
Also, empty List parameters are converted to None, but the migration
plugin doesn't like that - convert back to empty lists.
"""
for p in self.params():
Parse comma-separated lists of values in all parameter types. This can be enabled for a specific parameter by setting the "csv" option to True. Remove "List" parameter type and replace all occurences of it with appropriate multi-valued parameter ("Str" in most cases) with csv enabled. Add new parameter type "Any", capable of holding values of any type. This is needed by the "batch" command, as "Str" is not suitable type for the "methods" parameter. ticket 2007
2011-11-21 09:50:27 -06:00
if p.csv:
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
if options[p.name]:
options[p.name] = tuple(
v.lower() for v in options[p.name]
)
else:
options[p.name] = tuple()
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
def _get_search_bases(self, options, ds_base_dn, migrate_order):
search_bases = dict()
don't append basedn to container if it is included ticket #2566 When specifying a container to ds-migrate we should not automatically append the basedn if it is provided by the end-user. This is easy to detect using DN objects because DN objects have a endswith() method which can easily and correctly ascertain if a base already exists.
2012-04-16 17:48:57 -05:00
ds_base_dn = DN(ds_base_dn)
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
for ldap_obj_name in migrate_order:
Validate DN & RDN parameters for migrate command Ticket #2555 We were generating a traceback (server error) if a malformed RDN was passed as a parameter to the migrate command. * add parameter validation functions validate_dn_param() and validate_rdn_param() to ipalib.util. Those functions simply invoke the DN or RDN constructor from our dn module passing it the string representation. If the constructor does not throw an error it's valid. * Add the parameter validation function pointers to the Param objects in the migrate command. * Make the usercontainer and groupcontainer parameters required. passing --usercontainer= on the command line will produce ipa: ERROR: 'user_container' is required * Fix _get_search_bases() so if a container dn is empty it it just uses the base dn alone instead of faulting (currently bullet-proofing because now the containers are required). * Update the doc for usercontainer and groupcontainer to reflect the fact they are DN's not RDN's. A RDN can only be one level and it should be possible to have a container more than one RDN removed from the base.
2012-04-16 01:33:26 -05:00
container = options.get('%scontainer' % to_cli(ldap_obj_name))
if container:
don't append basedn to container if it is included ticket #2566 When specifying a container to ds-migrate we should not automatically append the basedn if it is provided by the end-user. This is easy to detect using DN objects because DN objects have a endswith() method which can easily and correctly ascertain if a base already exists.
2012-04-16 17:48:57 -05:00
container = DN(container)
# Don't append base dn if user already appended it in the container dn
if container.endswith(ds_base_dn):
search_base = container
else:
search_base = DN(container, ds_base_dn)
Validate DN & RDN parameters for migrate command Ticket #2555 We were generating a traceback (server error) if a malformed RDN was passed as a parameter to the migrate command. * add parameter validation functions validate_dn_param() and validate_rdn_param() to ipalib.util. Those functions simply invoke the DN or RDN constructor from our dn module passing it the string representation. If the constructor does not throw an error it's valid. * Add the parameter validation function pointers to the Param objects in the migrate command. * Make the usercontainer and groupcontainer parameters required. passing --usercontainer= on the command line will produce ipa: ERROR: 'user_container' is required * Fix _get_search_bases() so if a container dn is empty it it just uses the base dn alone instead of faulting (currently bullet-proofing because now the containers are required). * Update the doc for usercontainer and groupcontainer to reflect the fact they are DN's not RDN's. A RDN can only be one level and it should be possible to have a container more than one RDN removed from the base.
2012-04-16 01:33:26 -05:00
else:
search_base = ds_base_dn
don't append basedn to container if it is included ticket #2566 When specifying a container to ds-migrate we should not automatically append the basedn if it is provided by the end-user. This is easy to detect using DN objects because DN objects have a endswith() method which can easily and correctly ascertain if a base already exists.
2012-04-16 17:48:57 -05:00
search_bases[ldap_obj_name] = str(search_base)
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
return search_bases
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
def migrate(self, ldap, config, ds_ldap, ds_base_dn, options):
"""
Migrate objects from DS to LDAP.
"""
migrated = {} # {'OBJ': ['PKEY1', 'PKEY2', ...], ...}
failed = {} # {'OBJ': {'PKEY1': 'Failed 'cos blabla', ...}, ...}
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
search_bases = self._get_search_bases(options, ds_base_dn, self.migrate_order)
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
for ldap_obj_name in self.migrate_order:
ldap_obj = self.api.Object[ldap_obj_name]
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
search_filter = construct_filter(self.migrate_objects[ldap_obj_name]['filter_template'],
options[to_cli(self.migrate_objects[ldap_obj_name]['oc_option'])])
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
exclude = options['exclude_%ss' % to_cli(ldap_obj_name)]
Fix attributes that contain DNs when migrating. Some attributes, like secretary and manager, may point to other LDAP entries. We need to fix these during migration. https://fedorahosted.org/freeipa/ticket/2562
2012-03-20 21:37:27 -05:00
context = dict(ds_ldap = ds_ldap)
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
migrated[ldap_obj_name] = []
failed[ldap_obj_name] = {}
Handle an empty base_dn and no cn=ipaconfig in the ldap2 backend, fix migration. We lacked good error messages if the user/group container you used doesn't exist. Add a --continue option so things can continue if you use a bad user/group container. This has the side-effect of letting you migrate just users or groups by using a bad container for the one you don't want. Fix a Gettext() error when displaying the migrated password message. ticket 289
2010-09-27 12:50:54 -05:00
try:
(entries, truncated) = ds_ldap.find_entries(
Fix migration to work between v2 servers and remove search/size limits. Migration from a v2 server would fail because of our fake memberofindirect attribute. This isn't in any objectclass so would cause entries to fail to migrate. We can safely just remove it. Also remove any limits on time/size when searching for entries on the remote server. Otherwise only the number of entries configured in the local IPA server can be migrated. ticket 1124
2011-04-08 09:19:42 -05:00
search_filter, ['*'], search_bases[ldap_obj_name],
Add support defaultNamingContext and add --basedn to migrate-ds There are two sides to this, the server and client side. On the server side we attempt to add a defaultNamingContext on already installed servers. This will fail on older 389-ds instances but the failure is not fatal. New installations on versions of 389-ds that support this attribute will have it already defined. On the client side we need to look for both defaultNamingContext and namingContexts. We still need to check that the defaultNamingContext is an IPA server (info=IPAV2). The migration change also takes advantage of this and adds a new option which allows one to provide a basedn to use instead of trying to detect it. https://fedorahosted.org/freeipa/ticket/1919 https://fedorahosted.org/freeipa/ticket/2314
2012-01-30 15:29:32 -06:00
_ldap.SCOPE_ONELEVEL,
Handle LDAP search references LDAP search operation may return a search reference pointing to an LDAP resource. As the framework does not handle search references, skip these results to prevent result processing failures. Migrate operation crashed when the migrated DS contained search references. Now, it correctly skips these records and prints the failed references to user. https://fedorahosted.org/freeipa/ticket/1209
2011-06-01 11:04:24 -05:00
time_limit=0, size_limit=-1,
search_refs=True # migrated DS may contain search references
Handle an empty base_dn and no cn=ipaconfig in the ldap2 backend, fix migration. We lacked good error messages if the user/group container you used doesn't exist. Add a --continue option so things can continue if you use a bad user/group container. This has the side-effect of letting you migrate just users or groups by using a bad container for the one you don't want. Fix a Gettext() error when displaying the migrated password message. ticket 289
2010-09-27 12:50:54 -05:00
)
except errors.NotFound:
if not options.get('continue',False):
Fix translatable strings in ipalib plugins. Needed for xgettext/pygettext processing.
2011-02-23 15:47:49 -06:00
raise errors.NotFound(
Add support defaultNamingContext and add --basedn to migrate-ds There are two sides to this, the server and client side. On the server side we attempt to add a defaultNamingContext on already installed servers. This will fail on older 389-ds instances but the failure is not fatal. New installations on versions of 389-ds that support this attribute will have it already defined. On the client side we need to look for both defaultNamingContext and namingContexts. We still need to check that the defaultNamingContext is an IPA server (info=IPAV2). The migration change also takes advantage of this and adds a new option which allows one to provide a basedn to use instead of trying to detect it. https://fedorahosted.org/freeipa/ticket/1919 https://fedorahosted.org/freeipa/ticket/2314
2012-01-30 15:29:32 -06:00
reason=_('Container for %(container)s not found at %(search_base)s') % {'container': ldap_obj_name, 'search_base': search_bases[ldap_obj_name]}
Fix translatable strings in ipalib plugins. Needed for xgettext/pygettext processing.
2011-02-23 15:47:49 -06:00
)
Handle an empty base_dn and no cn=ipaconfig in the ldap2 backend, fix migration. We lacked good error messages if the user/group container you used doesn't exist. Add a --continue option so things can continue if you use a bad user/group container. This has the side-effect of letting you migrate just users or groups by using a bad container for the one you don't want. Fix a Gettext() error when displaying the migrated password message. ticket 289
2010-09-27 12:50:54 -05:00
else:
truncated = False
entries = []
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
if truncated:
self.log.error(
'%s: %s' % (
Fixed object_name and object_name_plural internationalization The object_name, object_name_plural and messages that use these attributes have been converted to support translation. The label attribute in the Param class has been modified to accept unicode string. Ticket #1435
2011-07-12 11:01:25 -05:00
ldap_obj.name, self.truncated_err_msg
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
)
)
Add ignore lists to migrate-ds command When user migrates users/groups from an old DS instance, the migration may fail on unsupported object classes and/or relevant LDAP object attributes. This patch implements a support for object class and attribute ignore lists that can be used to suppress these migration issues. Additionally, a redundant "dev/null" file is removed from git repo (originally added in 26b0e8fc9809a4cd9f2f9a2281f0894e2e0f8db2). https://fedorahosted.org/freeipa/ticket/1266
2011-06-03 07:21:43 -05:00
blacklists = {}
for blacklist in ('oc_blacklist', 'attr_blacklist'):
blacklist_option = self.migrate_objects[ldap_obj_name][blacklist+'_option']
if blacklist_option is not None:
blacklists[blacklist] = options.get(blacklist_option, tuple())
else:
blacklists[blacklist] = tuple()
Don't create private groups for migrated users, check for valid gidnumber Migrated users don't get a private group, there is no safe way to verify that the namespace is correct without redoing the uidnumber as well. Verify that the GID at least points to a valid group on the remote server and warn if it doesn't (this doesn't guarantee that the group gets migrated but at least we try). If the remote entry has no gidNumber then don't migrate that user. We don't know why that user is non-POSIX, it could be a special user used for auth, for example. Add a loginshell if the remote user doesn't have one. https://fedorahosted.org/freeipa/ticket/2562
2012-03-22 12:40:54 -05:00
# get default primary group for new users
if 'def_group_dn' not in context:
def_group = config.get('ipadefaultprimarygroup')
context['def_group_dn'] = api.Object.group.get_dn(def_group)
try:
(g_dn, g_attrs) = ldap.get_entry(context['def_group_dn'], ['gidnumber', 'cn'])
except errors.NotFound:
error_msg = _('Default group for new users not found')
raise errors.NotFound(reason=error_msg)
if 'gidnumber' in g_attrs:
context['def_group_gid'] = g_attrs['gidnumber'][0]
context['has_upg'] = ldap.has_upg()
valid_gids = []
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
for (dn, entry_attrs) in entries:
Handle LDAP search references LDAP search operation may return a search reference pointing to an LDAP resource. As the framework does not handle search references, skip these results to prevent result processing failures. Migrate operation crashed when the migrated DS contained search references. Now, it correctly skips these records and prints the failed references to user. https://fedorahosted.org/freeipa/ticket/1209
2011-06-01 11:04:24 -05:00
if dn is None: # LDAP search reference
failed[ldap_obj_name][entry_attrs[0]] = unicode(_ref_err_msg)
continue
migrate process cannot handle multivalued pkey attribute When group/user is migrated, the attribute used for RDN may be multivalued. Make sure that we pick the value used in the RDN which should be the unique one and not just the first one. https://fedorahosted.org/freeipa/ticket/1892
2011-09-29 04:55:13 -05:00
try:
dn = DN(dn)
except ValueError:
failed[ldap_obj_name][dn] = unicode(_dn_err_msg)
continue
ava = dn[0][0]
if ava.attr == ldap_obj.primary_key.name:
# In case if pkey attribute is in the migrated object DN
# and the original LDAP is multivalued, make sure that
# we pick the correct value (the unique one stored in DN)
pkey = ava.value.lower()
else:
pkey = entry_attrs[ldap_obj.primary_key.name][0].lower()
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
if pkey in exclude:
continue
dn = ldap_obj.get_dn(pkey)
entry_attrs['objectclass'] = list(
set(
config.get(
ldap_obj.object_class_config, ldap_obj.object_class
) + [o.lower() for o in entry_attrs['objectclass']]
)
)
Normalize the primary key value to lowercase during migration. https://bugzilla.redhat.com/show_bug.cgi?id=804609
2012-03-20 21:50:17 -05:00
entry_attrs[ldap_obj.primary_key.name][0] = entry_attrs[ldap_obj.primary_key.name][0].lower()
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
Make the migration plugin more configurable This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
2010-10-26 15:10:42 -05:00
callback = self.migrate_objects[ldap_obj_name]['pre_callback']
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
if callable(callback):
Don't create private groups for migrated users, check for valid gidnumber Migrated users don't get a private group, there is no safe way to verify that the namespace is correct without redoing the uidnumber as well. Verify that the GID at least points to a valid group on the remote server and warn if it doesn't (this doesn't guarantee that the group gets migrated but at least we try). If the remote entry has no gidNumber then don't migrate that user. We don't know why that user is non-POSIX, it could be a special user used for auth, for example. Add a loginshell if the remote user doesn't have one. https://fedorahosted.org/freeipa/ticket/2562
2012-03-22 12:40:54 -05:00
try:
dn = callback(
ldap, pkey, dn, entry_attrs, failed[ldap_obj_name],
config, context, schema = options['schema'],
search_bases = search_bases,
valid_gids = valid_gids,
**blacklists
)
if not dn:
continue
except errors.NotFound, e:
failed[ldap_obj_name][pkey] = unicode(e.reason)
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
continue
try:
ldap.add_entry(dn, entry_attrs)
except errors.ExecutionError, e:
Improve handling of GIDs when migrating groups Since IPA v2 server already contain predefined groups that may collide with groups in migrated (IPA v1) server (for example admins, ipausers), users having colliding group as their primary group may happen to belong to an unknown group on new IPA v2 server. Implement --group-overwrite-gid option to overwrite GID of already existing groups to prevent this issue. https://fedorahosted.org/freeipa/ticket/1866
2011-10-03 09:01:01 -05:00
callback = self.migrate_objects[ldap_obj_name]['exc_callback']
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
if callable(callback):
Improve handling of GIDs when migrating groups Since IPA v2 server already contain predefined groups that may collide with groups in migrated (IPA v1) server (for example admins, ipausers), users having colliding group as their primary group may happen to belong to an unknown group on new IPA v2 server. Implement --group-overwrite-gid option to overwrite GID of already existing groups to prevent this issue. https://fedorahosted.org/freeipa/ticket/1866
2011-10-03 09:01:01 -05:00
try:
callback(ldap, dn, entry_attrs, e, options)
except errors.ExecutionError, e:
failed[ldap_obj_name][pkey] = unicode(e)
continue
else:
failed[ldap_obj_name][pkey] = unicode(e)
continue
migrated[ldap_obj_name].append(pkey)
callback = self.migrate_objects[ldap_obj_name]['post_callback']
if callable(callback):
callback(
ldap, pkey, dn, entry_attrs, failed[ldap_obj_name],
config, context,
)
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
return (migrated, failed)
def execute(self, ldapuri, bindpw, **options):
ldap = self.api.Backend.ldap2
self.normalize_options(options)
config = ldap.get_ipa_config()[1]
Add support defaultNamingContext and add --basedn to migrate-ds There are two sides to this, the server and client side. On the server side we attempt to add a defaultNamingContext on already installed servers. This will fail on older 389-ds instances but the failure is not fatal. New installations on versions of 389-ds that support this attribute will have it already defined. On the client side we need to look for both defaultNamingContext and namingContexts. We still need to check that the defaultNamingContext is an IPA server (info=IPAV2). The migration change also takes advantage of this and adds a new option which allows one to provide a basedn to use instead of trying to detect it. https://fedorahosted.org/freeipa/ticket/1919 https://fedorahosted.org/freeipa/ticket/2314
2012-01-30 15:29:32 -06:00
ds_base_dn = options.get('basedn')
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
# check if migration mode is enabled
if config.get('ipamigrationenabled', ('FALSE', ))[0] == 'FALSE':
Migration warning when compat enabled Added check into migration plugin to warn user when compat is enabled. If compat is enabled, the migration fails and user is warned that he must turn the compat off or run the script with (the newly introduced) option '--with-compat'. '--with-compat' is new flag. If it is set, the compat status is ignored. https://fedorahosted.org/freeipa/ticket/2274
2012-03-01 04:41:53 -06:00
return dict(result={}, failed={}, enabled=False, compat=True)
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
# connect to DS
ds_ldap = ldap2(shared_instance=False, ldap_uri=ldapuri, base_dn='')
ds_ldap.connect(bind_dn=options['binddn'], bind_pw=bindpw)
Migration warning when compat enabled Added check into migration plugin to warn user when compat is enabled. If compat is enabled, the migration fails and user is warned that he must turn the compat off or run the script with (the newly introduced) option '--with-compat'. '--with-compat' is new flag. If it is set, the compat status is ignored. https://fedorahosted.org/freeipa/ticket/2274
2012-03-01 04:41:53 -06:00
#check whether the compat plugin is enabled
if not options.get('compat'):
Fix migration plugin compat check Ticket #2274 implements a check for compat plugin and warns user if it is enabled. However, there are 2 issues connected with the plugin: 1) The check is performed against the remote (migrated) LDAP server and not the local LDAP server, which does not make much sense 2) When the compat plugin is missing in cn=plugins,cn=config, it raises an error and thus breaks the migration This patch fixes both issues. https://fedorahosted.org/freeipa/ticket/2508
2012-03-09 05:36:03 -06:00
try:
(dn,check_compat) = ldap.get_entry(_compat_dn, normalize=False)
if check_compat is not None and \
check_compat.get('nsslapd-pluginenabled', [''])[0].lower() == 'on':
Don't create private groups for migrated users, check for valid gidnumber Migrated users don't get a private group, there is no safe way to verify that the namespace is correct without redoing the uidnumber as well. Verify that the GID at least points to a valid group on the remote server and warn if it doesn't (this doesn't guarantee that the group gets migrated but at least we try). If the remote entry has no gidNumber then don't migrate that user. We don't know why that user is non-POSIX, it could be a special user used for auth, for example. Add a loginshell if the remote user doesn't have one. https://fedorahosted.org/freeipa/ticket/2562
2012-03-22 12:40:54 -05:00
return dict(result={}, failed={}, enabled=True, compat=False)
Fix migration plugin compat check Ticket #2274 implements a check for compat plugin and warns user if it is enabled. However, there are 2 issues connected with the plugin: 1) The check is performed against the remote (migrated) LDAP server and not the local LDAP server, which does not make much sense 2) When the compat plugin is missing in cn=plugins,cn=config, it raises an error and thus breaks the migration This patch fixes both issues. https://fedorahosted.org/freeipa/ticket/2508
2012-03-09 05:36:03 -06:00
except errors.NotFound:
pass
Migration warning when compat enabled Added check into migration plugin to warn user when compat is enabled. If compat is enabled, the migration fails and user is warned that he must turn the compat off or run the script with (the newly introduced) option '--with-compat'. '--with-compat' is new flag. If it is set, the compat status is ignored. https://fedorahosted.org/freeipa/ticket/2274
2012-03-01 04:41:53 -06:00
Add support defaultNamingContext and add --basedn to migrate-ds There are two sides to this, the server and client side. On the server side we attempt to add a defaultNamingContext on already installed servers. This will fail on older 389-ds instances but the failure is not fatal. New installations on versions of 389-ds that support this attribute will have it already defined. On the client side we need to look for both defaultNamingContext and namingContexts. We still need to check that the defaultNamingContext is an IPA server (info=IPAV2). The migration change also takes advantage of this and adds a new option which allows one to provide a basedn to use instead of trying to detect it. https://fedorahosted.org/freeipa/ticket/1919 https://fedorahosted.org/freeipa/ticket/2314
2012-01-30 15:29:32 -06:00
if not ds_base_dn:
# retrieve base DN from remote LDAP server
(entries, truncated) = ds_ldap.find_entries(
'', ['namingcontexts', 'defaultnamingcontext'], '',
_ldap.SCOPE_BASE, size_limit=-1, time_limit=0,
)
if 'defaultnamingcontext' in entries[0][1]:
ds_base_dn = entries[0][1]['defaultnamingcontext'][0]
else:
try:
ds_base_dn = entries[0][1]['namingcontexts'][0]
except (IndexError, KeyError), e:
raise StandardError(str(e))
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
# migrate!
(migrated, failed) = self.migrate(
ldap, config, ds_ldap, ds_base_dn, options
)
Migration warning when compat enabled Added check into migration plugin to warn user when compat is enabled. If compat is enabled, the migration fails and user is warned that he must turn the compat off or run the script with (the newly introduced) option '--with-compat'. '--with-compat' is new flag. If it is set, the compat status is ignored. https://fedorahosted.org/freeipa/ticket/2274
2012-03-01 04:41:53 -06:00
return dict(result=migrated, failed=failed, enabled=True, compat=True)
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
def output_for_cli(self, textui, result, ldapuri, bindpw, **options):
textui.print_name(self.name)
if not result['enabled']:
textui.print_plain(self.migration_disabled_msg)
return 1
Migration warning when compat enabled Added check into migration plugin to warn user when compat is enabled. If compat is enabled, the migration fails and user is warned that he must turn the compat off or run the script with (the newly introduced) option '--with-compat'. '--with-compat' is new flag. If it is set, the compat status is ignored. https://fedorahosted.org/freeipa/ticket/2274
2012-03-01 04:41:53 -06:00
if not result['compat']:
textui.print_plain("The compat plug-in is enabled. This can increase the memory requirements during migration. Disable the compat plug-in with \'ipa-compat-manage disable\' or re-run this script with \'--with-compat\' option.")
return 1
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
textui.print_plain('Migrated:')
textui.print_entry1(
result['result'], attr_order=self.migrate_order,
one_value_per_line=False
)
for ldap_obj_name in self.migrate_order:
textui.print_plain('Failed %s:' % ldap_obj_name)
textui.print_entry1(
result['failed'][ldap_obj_name], attr_order=self.migrate_order,
one_value_per_line=True,
)
textui.print_plain('-' * len(self.name))
Handle an empty base_dn and no cn=ipaconfig in the ldap2 backend, fix migration. We lacked good error messages if the user/group container you used doesn't exist. Add a --continue option so things can continue if you use a bad user/group container. This has the side-effect of letting you migrate just users or groups by using a bad container for the one you don't want. Fix a Gettext() error when displaying the migrated password message. ticket 289
2010-09-27 12:50:54 -05:00
textui.print_plain(unicode(self.pwd_migration_msg))
Add DS migration plugin and password migration page.
2010-01-12 09:40:09 -06:00
api.register(migrate_ds)
Reference in New Issue Copy Permalink