This is to prevent a fatal name clash wih the new common "messages" Output.
Since i18n_messages is an internal plugin, the change does not affect
our public API.
The API version the client sends can now be used to check what the client
expects or is capable of.
All version tests IPA does will be be named and listed in one module,
ipalib.capabilities, which includes a function to test a specific capability
against an API version.
Similarly to Python's __future__ module, capabilities.py also serves as
documentation of backwards-incompatible changes to the API.
The first capability to be defined is "messages". Recent enough clients can
accept a list of warnings or other info under the "messages" key in the
result dict.
If a JSON client does not send the API version, it is assumed this is a testing
client (e.g. curl from the command line). Such a client "has" all capabilities,
but it will always receive a warning mentioning that forward compatibility
is not guaranteed.
If a XML client does not send the API version, it is assumed it uses the API
version before capabilities were introduced. (This is to keep backwards
compatibility with clients containing bug https://fedorahosted.org/freeipa/ticket/3294)
Whenever a capability is added, the API version must be incremented.
To ensure that, capabilities are written to API.txt and checked by
`makeapi --validate`.
Design page: http://freeipa.org/page/V3/Messages
Ticket: https://fedorahosted.org/freeipa/ticket/2732
Several Commands were missing the 'version' option. Add it to those
that were missing it.
Do not remove the version option before calling commands. This means
methods such as execute(), forward(), run() receive it.
Several of these needed `**options` added to their signatures.
Commands in the Cert plugin passed any unknown options to the underlying
functions, these are changed to pass what's needed explicitly.
Some commands in DNS and Batch plugins now pass version to commands
they call.
When the option is not given, fill it in automatically. (In a subsequent
commit, a warning will be added in this case).
Note that the public API did not change: all RPC calls already accepted
a version option. There's no need for an API version bump (even though
API.txt changes substantially).
Design page: http://freeipa.org/page/V3/Messages
Tickets:
https://fedorahosted.org/freeipa/ticket/2732https://fedorahosted.org/freeipa/ticket/3294
Use a new RESTful API provided by dogtag 10+. Construct an XML document
representing the search request. The output is limited to whatever dogtag
sends us, there is no way to request additional attributes other than
to read each certificate individually.
dogtag uses a boolean for each search term to indicate that it is used.
Presense of the search item is not enough, both need to be set.
The search operation is unauthenticated
Design page: http://freeipa.org/page/V3/Cert_findhttps://fedorahosted.org/freeipa/ticket/2528
Add new LDAP container to store the list of domains associated with IPA realm.
Add two new ipa commands (ipa realmdomains-show and ipa realmdomains-mod) to allow
manipulation of the list of realm domains.
Unit test file covering these new commands was added.
https://fedorahosted.org/freeipa/ticket/2945
When adding/modifying an ID range for a trusted domain, the newly
added option --dom-name can be used. This looks up SID of the
trusted domain in LDAP and therefore the user is not required
to write it down in CLI. If the lookup fails, error message
asking the user to specify the SID manually is shown.
https://fedorahosted.org/freeipa/ticket/3133
Update our LDAP schema and add 2 new attributes for SID blacklist
definition. These new attributes can now be set per-trust with
trustconfig command.
https://fedorahosted.org/freeipa/ticket/3289
Global trust configuration is generated ipa-adtrust-install script
is run. Add convenience commands to show auto-generated options
like SID or GUID or options chosen by user (NetBIOS). Most of these
options are not modifiable via trustconfig-mod command as it would
break current trusts.
Unit test file covering these new commands was added.
https://fedorahosted.org/freeipa/ticket/3333
Python does not guarantee dict order. Our reliance on it in `makeapi`
is technically incorrect, even if it doesn't matter yet in any of our
developers' environments.
This patch sorts the options by name instead of relying on dict order.
As an added benefit, future API.txt diffs should be be neater.
All *-find commands now enable leading/trailing whitespaces in the
search phrase. Behaviour has been implemented directly into
crud.Search class. IPA_API_VERSION_MINOR incremented to 45.
https://fedorahosted.org/freeipa/ticket/2981
bind-dyndb-ldap allows disabling global forwarder per-zone. This may
be useful in a scenario when we do not want requests to delegated
sub-zones (like sub.example.com. in zone example.com.) to be routed
through global forwarder.
Few lines to help added to explain the feature to users too.
https://fedorahosted.org/freeipa/ticket/3209
Nameserver hostname passed to dnszone_add command was always treated
as FQDN even though it was a relative DNS name to the new zone. All
relative names were being rejected as unresolvable.
Modify --name-server option processing in dnszone_add and dnszone_mod
to respect FQDN/relative DNS name and do the checks accordingly. With
this change, user can add a new zone "example.com" and let dnszone_add
to create NS record "ns" in it, when supplied with its IP address. IP
address check is more strict so that it is not entered when no forward
record is created. Places misusing the option were fixed.
Nameserver option now also accepts zone name, which means that NS and A
record is placed to DNS zone itself. Also "@" is accepted as a nameserver
name, BIND understand it also as a zone name. As a side-effect of this
change, other records with hostname part (MX, KX, NS, SRV) accept "@"
as valid hostname. BIND replaces it with respective zone name as well.
Unit tests were updated to test the new format.
https://fedorahosted.org/freeipa/ticket/3204
PAC type (ipakrbauthzdata attribute) was being filled for all new
service automatically. However, the PAC type attribute was designed
to serve only as an override to default PAC type configured in
IPA config. With PAC type set in all services, users would have
to update all services to get new PAC types configured in IPA config.
Do not set PAC type for new services. Add new NONE value meaning that
we do not want any PAC for the service (empty/missing attribute means
that the default PAC type list from IPA config is read).
https://fedorahosted.org/freeipa/ticket/2184
Public keys in the old format (raw RFC 4253 blob) are automatically
converted to OpenSSH-style public keys. OpenSSH-style public keys are now
stored in LDAP.
Changed sshpubkeyfp to be an output parameter, as that is what it actually
is.
Allow parameter normalizers to be used on values of any type, not just
unicode, so that public key blobs (which are str) can be normalized to
OpenSSH-style public keys.
ticket 2932, 2935
* Convert every string specifying a DN into a DN object
* Every place a dn was manipulated in some fashion it was replaced by
the use of DN operators
* Add new DNParam parameter type for parameters which are DN's
* DN objects are used 100% of the time throughout the entire data
pipeline whenever something is logically a dn.
* Many classes now enforce DN usage for their attributes which are
dn's. This is implmented via ipautil.dn_attribute_property(). The
only permitted types for a class attribute specified to be a DN are
either None or a DN object.
* Require that every place a dn is used it must be a DN object.
This translates into lot of::
assert isinstance(dn, DN)
sprinkled through out the code. Maintaining these asserts is
valuable to preserve DN type enforcement. The asserts can be
disabled in production.
The goal of 100% DN usage 100% of the time has been realized, these
asserts are meant to preserve that.
The asserts also proved valuable in detecting functions which did
not obey their function signatures, such as the baseldap pre and
post callbacks.
* Moved ipalib.dn to ipapython.dn because DN class is shared with all
components, not just the server which uses ipalib.
* All API's now accept DN's natively, no need to convert to str (or
unicode).
* Removed ipalib.encoder and encode/decode decorators. Type conversion
is now explicitly performed in each IPASimpleLDAPObject method which
emulates a ldap.SimpleLDAPObject method.
* Entity & Entry classes now utilize DN's
* Removed __getattr__ in Entity & Entity clases. There were two
problems with it. It presented synthetic Python object attributes
based on the current LDAP data it contained. There is no way to
validate synthetic attributes using code checkers, you can't search
the code to find LDAP attribute accesses (because synthetic
attriutes look like Python attributes instead of LDAP data) and
error handling is circumscribed. Secondly __getattr__ was hiding
Python internal methods which broke class semantics.
* Replace use of methods inherited from ldap.SimpleLDAPObject via
IPAdmin class with IPAdmin methods. Directly using inherited methods
was causing us to bypass IPA logic. Mostly this meant replacing the
use of search_s() with getEntry() or getList(). Similarly direct
access of the LDAP data in classes using IPAdmin were replaced with
calls to getValue() or getValues().
* Objects returned by ldap2.find_entries() are now compatible with
either the python-ldap access methodology or the Entity/Entry access
methodology.
* All ldap operations now funnel through the common
IPASimpleLDAPObject giving us a single location where we interface
to python-ldap and perform conversions.
* The above 4 modifications means we've greatly reduced the
proliferation of multiple inconsistent ways to perform LDAP
operations. We are well on the way to having a single API in IPA for
doing LDAP (a long range goal).
* All certificate subject bases are now DN's
* DN objects were enhanced thusly:
- find, rfind, index, rindex, replace and insert methods were added
- AVA, RDN and DN classes were refactored in immutable and mutable
variants, the mutable variants are EditableAVA, EditableRDN and
EditableDN. By default we use the immutable variants preserving
important semantics. To edit a DN cast it to an EditableDN and
cast it back to DN when done editing. These issues are fully
described in other documentation.
- first_key_match was removed
- DN equalty comparison permits comparison to a basestring
* Fixed ldapupdate to work with DN's. This work included:
- Enhance test_updates.py to do more checking after applying
update. Add test for update_from_dict(). Convert code to use
unittest classes.
- Consolidated duplicate code.
- Moved code which should have been in the class into the class.
- Fix the handling of the 'deleteentry' update action. It's no longer
necessary to supply fake attributes to make it work. Detect case
where subsequent update applies a change to entry previously marked
for deletetion. General clean-up and simplification of the
'deleteentry' logic.
- Rewrote a couple of functions to be clearer and more Pythonic.
- Added documentation on the data structure being used.
- Simplfy the use of update_from_dict()
* Removed all usage of get_schema() which was being called prior to
accessing the .schema attribute of an object. If a class is using
internal lazy loading as an optimization it's not right to require
users of the interface to be aware of internal
optimization's. schema is now a property and when the schema
property is accessed it calls a private internal method to perform
the lazy loading.
* Added SchemaCache class to cache the schema's from individual
servers. This was done because of the observation we talk to
different LDAP servers, each of which may have it's own
schema. Previously we globally cached the schema from the first
server we connected to and returned that schema in all contexts. The
cache includes controls to invalidate it thus forcing a schema
refresh.
* Schema caching is now senstive to the run time context. During
install and upgrade the schema can change leading to errors due to
out-of-date cached schema. The schema cache is refreshed in these
contexts.
* We are aware of the LDAP syntax of all LDAP attributes. Every
attribute returned from an LDAP operation is passed through a
central table look-up based on it's LDAP syntax. The table key is
the LDAP syntax it's value is a Python callable that returns a
Python object matching the LDAP syntax. There are a handful of LDAP
attributes whose syntax is historically incorrect
(e.g. DistguishedNames that are defined as DirectoryStrings). The
table driven conversion mechanism is augmented with a table of
hard coded exceptions.
Currently only the following conversions occur via the table:
- dn's are converted to DN objects
- binary objects are converted to Python str objects (IPA
convention).
- everything else is converted to unicode using UTF-8 decoding (IPA
convention).
However, now that the table driven conversion mechanism is in place
it would be trivial to do things such as converting attributes
which have LDAP integer syntax into a Python integer, etc.
* Expected values in the unit tests which are a DN no longer need to
use lambda expressions to promote the returned value to a DN for
equality comparison. The return value is automatically promoted to
a DN. The lambda expressions have been removed making the code much
simpler and easier to read.
* Add class level logging to a number of classes which did not support
logging, less need for use of root_logger.
* Remove ipaserver/conn.py, it was unused.
* Consolidated duplicate code wherever it was found.
* Fixed many places that used string concatenation to form a new
string rather than string formatting operators. This is necessary
because string formatting converts it's arguments to a string prior
to building the result string. You can't concatenate a string and a
non-string.
* Simplify logic in rename_managed plugin. Use DN operators to edit
dn's.
* The live version of ipa-ldap-updater did not generate a log file.
The offline version did, now both do.
https://fedorahosted.org/freeipa/ticket/1670https://fedorahosted.org/freeipa/ticket/1671https://fedorahosted.org/freeipa/ticket/1672https://fedorahosted.org/freeipa/ticket/1673https://fedorahosted.org/freeipa/ticket/1674https://fedorahosted.org/freeipa/ticket/1392https://fedorahosted.org/freeipa/ticket/2872
range plugin was missing range-mod command that could be used for
example to fix a size for a range generated during upgrades. The
range should be updated with a caution though, a misconfiguration
could break trusts.
iparangetype is now also handled better and filled in all commands
instead of just range-show. objectclass attribute is deleted only
when really needed now.
When using ipaExternalGroup/ipaExternalMember attributes it is
possible to add group members which don't exist in IPA database.
This is primarily is required for AD trusts support and therefore
validation is accepting only secure identifier (SID) format.
https://fedorahosted.org/freeipa/ticket/2664
IPA implements read/write permissions for DNS record or zones.
Provided set of permissions and privileges can, however, only grant
access to the whole DNS tree, which may not be appropriate.
Administrators may miss more fine-grained permissions allowing
them to delegate access per-zone.
Create a new IPA auxiliary objectclass ipaDNSZone allowing
a managedBy attribute for a DNS zone. This attribute will hold
a group DN (in this case a permission) which allows its members
to read or write in a zone. Member permissions in given zone
will only have 2 limitations:
1) Members cannot delete the zone
2) Members cannot edit managedBy attribute
Current DNS deny ACI used to enforce read access is removed so that
DNS privileges are based on allow ACIs only, which is much more
flexible approach as deny ACIs have always precedence and limit
other extensions. Per-zone access is allowed in 3 generic ACIs
placed in cn=dns,$SUFFIX so that no special ACIs has to be added
to DNS zones itselves.
2 new commands have been added which allows an administrator to
create the system permission allowing the per-zone access and
fill a zone's managedBy attribute:
* dnszone-add-permission: Add per-zone permission
* dnszone-remove-permission: Remove per-zone permission
https://fedorahosted.org/freeipa/ticket/2511
Add two global ipaConfig options to disable undesirable writes that have
performance impact.
The "KDC:Disable Last Success" will disable writing back to ldap the last
successful AS Request time (successful kinit)
The "KDC:Disable Lockout" will disable completely writing back lockout
related data. This means lockout policies will stop working.
https://fedorahosted.org/freeipa/ticket/2734
When no user/group was found, migration plugin reported an ambiguous
error about invalid container. But the root cause may be for example
in a wrong list of user/group objectclasses. Report both in the error
message to avoid user confusion.
User/group objectclass attribute is now also marked as required.
Without the list of objectclasses, an invalid LDAP search is
produced.
https://fedorahosted.org/freeipa/ticket/2206
For security reasons, dynamic updates are not enabled for new DNS
zones. In order to enable the dynamic zone securely, user needs to
allow dynamic updates and create a zone update policy.
The policy is not easy to construct for regular users, we should
rather fill it by default and let users just switch the policy
on or off.
https://fedorahosted.org/freeipa/ticket/2441
Make --{set,add,del}attr fail on parameters with the no_update/no_create
flag for the respective command.
For attributes that can be modified, but we just don't want to display
in the CLI, use the 'no_option' flag. These are "locking" attributes
(ipaenabledflag, nsaccountlock) and externalhost.
Document the 'no_option' flag. Add some tests.
https://fedorahosted.org/freeipa/ticket/2580
ipa permission-mod was prompting for all parameters because they had
specified flag 'ask_update'. The flag was removed. Additionally the
exec_callback for permission-mod was updated to unify the behaviour with
other ipa commands (raise exception when no modification was specified).
https://fedorahosted.org/freeipa/ticket/2280
Ticket #2555
We were generating a traceback (server error) if a malformed RDN was
passed as a parameter to the migrate command.
* add parameter validation functions validate_dn_param() and
validate_rdn_param() to ipalib.util. Those functions simply invoke
the DN or RDN constructor from our dn module passing it the string
representation. If the constructor does not throw an error it's
valid.
* Add the parameter validation function pointers to the Param objects
in the migrate command.
* Make the usercontainer and groupcontainer parameters required.
passing --usercontainer= on the command line will produce
ipa: ERROR: 'user_container' is required
* Fix _get_search_bases() so if a container dn is empty it it just
uses the base dn alone instead of faulting (currently
bullet-proofing because now the containers are required).
* Update the doc for usercontainer and groupcontainer to reflect the
fact they are DN's not RDN's. A RDN can only be one level and it
should be possible to have a container more than one RDN removed
from the base.
The DN and ACI code doesn't always escape special characters properly.
Rather than trying to fix it, this patch takes the easy way out and
enforces that the names are safe.
https://fedorahosted.org/freeipa/ticket/2585
This will prevent errors if an empty reason is provided and it is
set by default one doesn't have to always set it on the command-line.
https://fedorahosted.org/freeipa/ticket/2597
nisdomain validation:
Added pattern to the 'nisdomain' parameter to validate the specified
nisdomain name. According to most common use cases the same pattern as
for netgroup should fit. Unit-tests added.
https://fedorahosted.org/freeipa/ticket/2448
'add_external_pre_callback' function was created to allow validation of
all external members. Validation is based on usage of objects primary
key parameter. The 'add_external_pre_callback' fucntion has to be called
directly from in the 'pre_callback' function. This change affects
netgroup, hbacrule and sudorule commands.
For hostname, the validator allows non-fqdn and underscore characters.
validate_hostname function in ipalib.util was modified and contains
additional option that allows hostname to contain underscore characters.
This option is disabled by default.
Unit-tests added.
https://fedorahosted.org/freeipa/ticket/2447
Add a support for new global options in bind-dyndb-ldap, that is:
* idnsforwardpolicy: Default policy for conditional forwarding
* idnsallowsyncptr: Allow globaly PTR synchronization for dynamic
updates
* idnszonerefresh: Default interval between regular polls of the
name server for new DNS zones
https://fedorahosted.org/freeipa/ticket/2439
Update ipaSudoRule objectClass on upgrades to add new attributes.
Ensure uniqueness of sudoOrder in rules.
The attributes sudoNotBefore and sudoNotAfter are being added to
schema but not as Params.
https://fedorahosted.org/freeipa/ticket/1314
Added check into migration plugin to warn user when compat is enabled.
If compat is enabled, the migration fails and user is warned that he
must turn the compat off or run the script with (the newly introduced)
option '--with-compat'.
'--with-compat' is new flag. If it is set, the compat status is ignored.
https://fedorahosted.org/freeipa/ticket/2274
There may be cases, for whatever reason, that an otherwise illegal
entry gets created that doesn't match the criteria for a valid
user/host/group name. If this happens (i.e. migration) there is no way
to remove this using the IPA tools because we always applied the name
pattern. So you can't, for example, delete a user with an illegal name.
Primary keys are cloned with query=True in PKQuery which causes no
rules to be applied on mod/show/find. This reverts a change from commit
3a5e26a0 which applies class rules when query=True (for enforcing no
white space).
Replace rdnattr with rdn_is_primary_key. This was meant to tell us when
an RDN change was necessary to do a rename. There could be a disconnect
where the rdnattr wasn't the primary key and in that case we don't
need to do an RDN change, so use a boolean instead so that it is
clear that RDN == primary key.
Add a test to ensure that nowhitespace is actually enforced.
https://fedorahosted.org/freeipa/ticket/2115
Related: https://fedorahosted.org/freeipa/ticket/2089
Whitespace tickets:
https://fedorahosted.org/freeipa/ticket/1285https://fedorahosted.org/freeipa/ticket/1286https://fedorahosted.org/freeipa/ticket/1287
DNS plugin did not check DNS zone and DNS record validity and
user was thus able to create domains like "foo bar" or other
invalid DNS labels which would really confuse both user and
bind-dyndb-ldap plugin.
This patch at first consolidates hostname/domain name validators
so that they use common functions and we don't have regular
expressions and other checks defined in several places. These
new cleaned validators are then used for zone/record name
validation.
https://fedorahosted.org/freeipa/ticket/2384
