These methods currently take (dn, entry_attrs, normalize=True)
(or (dn, normalize=True) for delete).
Change them to also accept just an LDAPEntry.
For add and update, document the old style as deprecated.
Part of the work for: https://fedorahosted.org/freeipa/ticket/2660
The get_ipa_config method relies on IPA being installed.
Leave the implementation in ldap2, and add stub get_ipa_config that
simply returns an empty dictionary to LDAPConnection.
The get_members method contains an optimization that also relies on
the api object. The optimization is disabled in base LDAPConfig.
Part of the work for: https://fedorahosted.org/freeipa/ticket/2660
ldap2 has "DN normalization" functionality, which silently adds the base
DN to DNs that don't already end with it.
This functionality is left in the ldap2 class only.
Part of the work for: https://fedorahosted.org/freeipa/ticket/2660
Instead of inheritance, the LDAPObject is kept in a `conn` attribute,
as the ldap2 class does it.
However, all IPASimpleLDAPObject methods are still available as before
(using __getattr__-based proxying).
Also, remove the IPAEntryLDAPObject class. The methods it overrode
were never called in ways that would make them behave differently than
the superclass.
Part of the work for: https://fedorahosted.org/freeipa/ticket/2660
The ldap2 plugin should only contain the actual CRUDBackend plugin.
Common LDAP classes are moved to ipaldap.
Part of the work for: https://fedorahosted.org/freeipa/ticket/2660
This decision used the api object, which might not be available
in installer code. Move the decision to callers.
Part of the work for: https://fedorahosted.org/freeipa/ticket/2660
The connection code depends on the api object. If we want to use
the cache in installer code, where the api object is not always
available, the dependency must be removed.
Luckily, SchemaCache.get_schema is currently always given a connection.
Part of the work for: https://fedorahosted.org/freeipa/ticket/2660
Legacy Entry methods such as setValue are added to LDAPEntry directly,
so that we can use connection classes that return LDAPEntry with
code that expects Entries.
The Entry and its unique __init__ are still kept for compatibility.
Part of the work for: https://fedorahosted.org/freeipa/ticket/2660
This change makes it easier to see what is going on, in addition
to getting rid of pylint exceptions.
Also, make logging imports use explicit names instead of `import *`.
Part of the work for: https://fedorahosted.org/freeipa/ticket/2660
Some commands require a connection for interactive prompting.
Prompt after the connection is created.
Option parsing is still done before connecting so that help
can be printed out without a Kerberos ticket.
https://fedorahosted.org/freeipa/ticket/3453
The code split the permission string on commas, essentially doing
poor man's CSV parsing. So if a permission contained a
comma-separated list of valid permissions, validation would pass
but we'd get errors later.
https://fedorahosted.org/freeipa/ticket/3420
This is to prevent a fatal name clash wih the new common "messages" Output.
Since i18n_messages is an internal plugin, the change does not affect
our public API.
The API version the client sends can now be used to check what the client
expects or is capable of.
All version tests IPA does will be be named and listed in one module,
ipalib.capabilities, which includes a function to test a specific capability
against an API version.
Similarly to Python's __future__ module, capabilities.py also serves as
documentation of backwards-incompatible changes to the API.
The first capability to be defined is "messages". Recent enough clients can
accept a list of warnings or other info under the "messages" key in the
result dict.
If a JSON client does not send the API version, it is assumed this is a testing
client (e.g. curl from the command line). Such a client "has" all capabilities,
but it will always receive a warning mentioning that forward compatibility
is not guaranteed.
If a XML client does not send the API version, it is assumed it uses the API
version before capabilities were introduced. (This is to keep backwards
compatibility with clients containing bug https://fedorahosted.org/freeipa/ticket/3294)
Whenever a capability is added, the API version must be incremented.
To ensure that, capabilities are written to API.txt and checked by
`makeapi --validate`.
Design page: http://freeipa.org/page/V3/Messages
Ticket: https://fedorahosted.org/freeipa/ticket/2732
The messages module contains message classes that can be added
to a RPC response to provide additional information or warnings.
This patch adds only the module with a single public message,
VersionMissing, and unit tests.
Since message classes are very similar to public errors, some
functionality and unit tests were shared.
Design page: http://freeipa.org/page/V3/Messages
Ticket: https://fedorahosted.org/freeipa/ticket/2732
Several Commands were missing the 'version' option. Add it to those
that were missing it.
Do not remove the version option before calling commands. This means
methods such as execute(), forward(), run() receive it.
Several of these needed `**options` added to their signatures.
Commands in the Cert plugin passed any unknown options to the underlying
functions, these are changed to pass what's needed explicitly.
Some commands in DNS and Batch plugins now pass version to commands
they call.
When the option is not given, fill it in automatically. (In a subsequent
commit, a warning will be added in this case).
Note that the public API did not change: all RPC calls already accepted
a version option. There's no need for an API version bump (even though
API.txt changes substantially).
Design page: http://freeipa.org/page/V3/Messages
Tickets:
https://fedorahosted.org/freeipa/ticket/2732https://fedorahosted.org/freeipa/ticket/3294
Sudo commands created in the past have the sudocmd in their RDN, while
the new case-sensitive ones have ipaUniqueID. In order for permissions
to apply to both of these, use a targetfilter for objectclass=ipasudocmd
instead of sudocmd=* in the target.
Since it is not really possible to separate SSH errors from
errors of the called program, add a SSH check before
calling replica-conncheck on the master.
The check also adds the master to a temporary known_hosts file,
so suppressing SSH's warning about unknown host is no longer
necessary. If the "real" connection fails despite the check,
any SSH errors will be included in the output.
https://fedorahosted.org/freeipa/ticket/3402
Move the parser setup from bootstrap_with_global_options to bootstrap,
so all API objects have access to it.
Add some CLI tests for the help system.
Part of the effort for https://fedorahosted.org/freeipa/ticket/3060
This avoids the problem with ambiguous command/topic names.
No functionality is changed; `ipa help <COMMAND>` still works as before
if there's no topic with the same name.
https://fedorahosted.org/freeipa/ticket/3247
Make `ipa -h` and `ipa help` output the same message.
Since `ipa -h` output is generated by the OptionParser, we need to make
the parser available. Store it in `api.parser`.
Part of the effort for https://fedorahosted.org/freeipa/ticket/3060
Whenever a command is used incorrectly, it should output
an error message (and possibly additional help) to stderr.
This patch adds a parameter to a bunch of places to allow
selecting either stdout or stderr for help output, and makes
badly called commands output to stderr only.
Part of the effort for https://fedorahosted.org/freeipa/ticket/3060
Fix the usage string to match actual usage.
Add command description.
Put information about `ipa help topics` etc. to the epilog,
instead of using empty option groups. Use a custom formatter
to preserve newlines.
Add the -h/--help option manually to ensure consistent case
(capital S).
Part of the effort for https://fedorahosted.org/freeipa/ticket/3060
Modify update file to use default: rather than add: in
cn=replication,cn=etc,$SUFFIX.
Drop quotes around nsDS5ReplicaRoot because default: values
are not parsed as CSV.
https://fedorahosted.org/freeipa/ticket/3394
- Automatically add a "Logging and output options" group with the --quiet,
--verbose, --log-file options.
- Set up logging based on these options; details are in the setup_logging
docstring and in the design document.
- Don't bind log methods as individual methods of the class. This means one
less linter exception.
- Make the help for command line options consistent with optparse's --help and
--version options.
Design document: http://freeipa.org/page/V3/Logging_and_output
The DS is installed before the CA cert is generated. Trying to
add the cert to LDAP before it exists resulted in a nasty-looking
error message.
This moves the cert upload to after the CA cert is ready and the
certdb is created.
Move the cert upload to after thecertdb is generated.
https://fedorahosted.org/freeipa/ticket/3375
Python does not guarantee dict order. Our reliance on it in `makeapi`
is technically incorrect, even if it doesn't matter yet in any of our
developers' environments.
This patch sorts the options by name instead of relying on dict order.
As an added benefit, future API.txt diffs should be be neater.
The ipa-csreplica-manage tool often assumed that the port numbers are the
same on both sides of a replication agreement.
This assumption doesn't hold in a cluster with both old-style hosts and
ones with merged DBs.
When managing agreements, determine the port with the PKI (or merged) DS
on each master, and use it.
Also, in CSReplicationManager, always use starttls rather than ldaps://.
The ancient IPAdmin class used some heuristics to determine the protocol
to connect with (ldap, ldaps, or ldapi). In turn, some calling code used
questionable mechanisms to get the correct protocol: in
ipaserver/install/replication.py, the CA cert was either passed to the
constructor or added to the class afterwards, to get ldap:// or ldaps://.
Add an explicit protocol argument and only fall back to backwards-compatible
guessing if it is not given.
Stopping certificate tracking was done as part of the PKI DS uninstall.
Since with the merged DB, thePKI DS is not used any more, this step
was skipped.
Move certificate untracking to a separate step and call it separately.
Also, the post-uninstall check for tracked certificates used the wrong
set of Dogtag constants. Fix the issue.
This brings /etc/dirsrv/slapd-REALM/certmap.conf under IPA control.
The file is overwritten on upgrades.
This ensures that the cert for the ipaca user is recognized when
ipa-ca-install is run on older masters.
The new merged database will replicate with both the IPA and CA trees, so all
DS instances (IPA and CA on the existing master, and the merged one on the
replica) need to have the same schema.
Dogtag does all its schema modifications online. Those are replicated normally.
The basic IPA schema, however, is delivered in ldif files, which are not
replicated. The files are not present on old CA DS instances. Any schema
update that references objects in these files will fail.
The whole 99user.ldif (i.e. changes introduced dynamically over LDAP) is
replicated as a blob. If we updated the old master's CA schema dynamically
during replica install, it would conflict with updates done during the
installation: the one with the lower CSN would get lost.
Dogtag's spawn script recently grew a new flag, 'pki_clone_replicate_schema'.
Turning it off tells Dogtag to create its schema in the clone, where the IPA
modifications are taking place, so that it is not overwritten by the IPA schema
on replication.
The patch solves the problems by:
- In __spawn_instance, turning off the pki_clone_replicate_schema flag.
- Providing a script to copy the IPA schema files to the CA DS instance.
The script needs to be copied to old masters and run there.
- At replica CA install, checking if the schema is updated, and failing if not.
The --skip-schema-check option is added to ipa-{replica,ca}-install to
override the check.
All pre-3.1 CA servers in a domain will have to have the script run on them to
avoid schema replication errors.
https://fedorahosted.org/freeipa/ticket/3213
- Provide a function for determinig the CA status using Dogtag 10's new
getStatus endpoint.
This must be done over HTTPS, but since our client certificate may not be set
up yet, we need HTTPS without client authentication.
Rather than copying from the existing http_request and https_request
function, shared code is factored out to a common helper.
- Call the new function when restarting the CA service. Since our Service
can only be extended in platform-specific code, do this for Fedora only.
Also, the status is only checked with Dogtag 10+.
- When a restart call in cainstance failed, users were refered to the
installation log, but no info was actually logged. Log the exception.
https://fedorahosted.org/freeipa/ticket/3084
Previously, ipa-replica-install tried to check DNS resolution on the master
being cloned. If that master was not a DNS server, the check failed.
Change the check to query the first available configured DNS server.
Log about the check before actually running it.
Log in the case the check is skipped (no IPA DNS servers installed).
https://fedorahosted.org/freeipa/ticket/3194
When the user interrupts a long-running command, this ensures that
the command is logged. Also, when watching log files (or the -d
output), it's apparent what's being done.
https://fedorahosted.org/freeipa/ticket/3174
Dogtag opens not only the insecure port (8080 or 9180, for d10 and
d9 respectively), but also secure ports (8443 or 9443&9444).
Wait for them when starting.
Part of the fix for https://fedorahosted.org/freeipa/ticket/3084
Our installation added two final dots to the NS records,
so the records were invalid, Bind ignored the entire zone,
and name resolution didn't work.
Fix this error and add a check for empty DNS labels to the validator
Instead of `except:`, use `except Exception:`. This means that errors
like KeyboardInterrupt are not handled, letting them terminate the
script as expected.
https://fedorahosted.org/freeipa/ticket/2941
When there were no updated attrs when modifying a nonexistent DNS record,
the error was not handled and caused an internal server error later (old_entry
was used uninitialized).
https://fedorahosted.org/freeipa/ticket/3055
Forward and reverse resolution of the newly created replica is already
checked via get_host_name (which calls verify_fqdn).
Add the same check for the existing master.
Additionally, if DNS is installed on the remote host, check forward
and reverse resolution of both replicas using that DNS only
(ignoring /etc/hosts). These checks give only warnings and, in interactive
installs, a "Continue?" prompt.
https://fedorahosted.org/freeipa/ticket/2845
We've been stopping both DS instances (main and PKI) when upgrading.
This can happen while the CA is running. In some cases stopping the PKI
DS also killed the CA.
Only stop the specific instance for upgrades.
Also, wait for open ports after the upgrade is complete. The wait was
skipped previously. This can prevent bugs if scripts that need a DS are
run after the upgrade.
https://fedorahosted.org/freeipa/ticket/3083
Put the changes from Ade's dogtag 10 patch into namespaced constants in
dogtag.py, which are then referenced in the code.
Make ipaserver.install.CAInstance use the service name specified in the
configuration. Uninstallation, where config is removed before CA uninstall,
also uses the (previously) configured value.
This and Ade's patch address https://fedorahosted.org/freeipa/ticket/2846
When clients install, they use kinit to obtain a TGT, which uses DNS to find
the KDC to connect to. It might happen that the newly created principal
has not replicated to selected KDC yet, making kinit fail and aborting the
install.
The client sets a temporary krb5 config file while installing via $KRB5_CONFIG.
Modify this file so that the kerberos library only uses the specific server
we're installing under, and call kinit while it's still in place.
Clean up the configure_krb5_conf function to remove unused arguments. For
clarity, use keyword arguments when calling it.
https://fedorahosted.org/freeipa/ticket/2982
Generalize the fix_replica_memberof update plugin to allow updating more
replication attributes.
Add nsds5ReplicaStripAttrs to replication agreements on update and
replica install.
https://fedorahosted.org/freeipa/ticket/2534
Currently, we throw many public exceptions without proper i18n.
Wrap natural-language error messages in _() so they can be translated.
In the service plugin, raise NotFound errors using handle_not_found helper
so the error message contains the offending service.
Use ScriptError instead of NotFoundError in bindinstance install.
https://fedorahosted.org/freeipa/ticket/1953
When the --hostname option is given to ipa-client-install, we
write HOSTNAME to /etc/sysconfig/network. When that file didn't exist,
the installer crashed.
Create the file if it doesn't exist and we need to write to it.
https://fedorahosted.org/freeipa/ticket/2840
Update the pot file to match current source, and pull
translations from Transifex
The commands used to create this commit were:
tx pull -fa # needs an account and transifex client setup
cd install/po
make update-pot
(update LINGUAS to contain all languages with translations, and no empty ones)
make update-po
The .po files we use for translations have two shortcomings when used in Git:
- They include file locations, which change each time the source is updated.
This results in large, unreadable diffs that don't merge well.
- They include source strings for untranslated messages, wasting space
unnecessarily.
Update the Makefile so that the extraneous information is stripped when the
files are updated or pulled form Transifex, and empty translation files are
removed entirely.
Also, translations are normalized to a common style. This should help diffs
and merges.
The validator requires file location comments to identify the programming
language, and to produce good error reports.
To make this work, merge the comments in before validation.
First patch for: https://fedorahosted.org/freeipa/ticket/2435
Currently, FreeIPA's install/admin scripts are long pieces of code
that aren't very reusable, importable, or testable.
They have been extended over time with features such as logging and
error handling, but since each tool was extended individually, there
is much inconsistency and code duplication.
This patch starts a framework which the admin tools can use, and
converts ipa-ldap-updater to use the framework.
Common tasks the tools do -- option parsing, validation, logging
setup, error handling -- are represented as methods. Individual
tools can extend, override or reuse the defaults as they see fit.
The ipa-ldap-updater has two modes (normal and --upgrade) that
don't share much functionality. They are represented by separate
classes. Option parsing, and selecting which class to run, happens
before they're instantiated.
All code is moved to importable modules to aid future testing. The
only thing that remains in the ipa-ldap-updater script is a two-line
call to the library.
First part of the work for:
https://fedorahosted.org/freeipa/ticket/2652
The upgrade script set the "psearch" directive in some circumstances,
but did not remember that it was set, so later, when setting
minimum_connections, it assumed psearch is not set.
Also, the script did not set minimum_connections if the directive wasn't
already there. It should be set in that case.
Related to https://fedorahosted.org/freeipa/ticket/2554
The Batch command did not report errors correctly: it reported
the text of *all* errors, not just PublicError, used unicode(e)
instead of e.strerror (which results in incorrect i18n), and only
reported the text of error messages, not their type and code.
Fix these problems. Update tests.
https://fedorahosted.org/freeipa/ticket/2874https://fedorahosted.org/freeipa/ticket/2901
Petr Viktorin