Add new users to the default users group in batches of 100. The
biggest overhead of migration is in calculating the modlist when
managing the default user's group and applying the changes. A
significant amount of time can be saved by not doing this on every
add operation.
Some other minor improvements include:
Add a negative cache for groups not found in the remote LDAP server.
Replace call to user_mod with a direct LDAP update.
Catch some occurances of LimitError and handle more gracefully.
I also added some debug logging to report on migration status and
performance.
https://fedorahosted.org/freeipa/ticket/3386
Some parts of install scripts used only ccache name as returned by
krbV.CCache.name attribute. However, when this name is used again
to initialize krbV.CCache object or when it is used in KRB5CCNAME
environmental variable, it fails for new DIR type of CCACHE.
We should always use both CCACHE type and name when referring to
them to avoid these crashes. ldap2 backend was also updated to
accept directly krbV.CCache object which contains everything we need
to authenticate with ccache.
https://fedorahosted.org/freeipa/ticket/3381
openldap server does not store its schema in cn=schema entry, but
rather in cn=subschema. Add a fallback to ldap2 plugin to read from
this entry when cn=schema is not found. ldap2 plugin uses the schema
when doing some of the automatic encoding, like an automatic
encoding of DN object.
IPA migration plugin DN attribute processing is now also more
tolerant when it finds that some DN attribute was not autoencoded.
It tries to convert it to DN on its own and report a warning and
continue with user processing when the conversion fails instead of
crashing with AssertionError and thus abandoning the whole
migration run.
https://fedorahosted.org/freeipa/ticket/3372
Add more dynamic attribute info to IPATypeChecker in make-lint. Remove
unnecessary pylint comments. Fix false positivies introduced by Pylint 0.26.
https://fedorahosted.org/freeipa/ticket/3379
Check to see if NSS is initialized before trying to do so again.
If we are temporarily creating a certificate be sure to delete it in order
to remove references to it and avoid NSS shutdown issues.
In the certificate load validator shut down NSS if we end up initializing
it. I'm not entirely sure why but this prevents a later shutdown issue
if we are passed the --ca-cert-file option.
Major changes ipa-client-install:
* Use GSSAPI connection to LDAP server to download CA cert (now
the default method)
* Add --ca-cert-file option to load the CA cert from a disk file.
Validate the file. If this option is used the supplied CA cert
is considered definitive.
* The insecure HTTP retrieval method is still supported but it must be
explicitly forced and a warning will be emitted.
* Remain backward compatible with unattended case (except for aberrant
condition when preexisting /etc/ipa/ca.crt differs from securely
obtained CA cert, see below)
* If /etc/ipa/ca.crt CA cert preexists the validate it matches the
securely acquired CA cert, if not:
- If --unattended and not --force abort with error
- If interactive query user to accept new CA cert, if not abort
In either case warn user.
* If interactive and LDAP retrieval fails prompt user if they want to
proceed with insecure HTTP method
* If not interactive and LDAP retrieval fails abort unless --force
* Backup preexisting /etc/ipa/ca.crt in FileStore prior to execution,
if ipa-client-install fails it will be restored.
Other changes:
* Add new exception class CertificateInvalidError
* Add utility convert_ldap_error() to ipalib.ipautil
* Replace all hardcoded instances of /etc/ipa/ca.crt in
ipa-client-install with CACERT constant (matches existing practice
elsewhere).
* ipadiscovery no longer retrieves CA cert via HTTP.
* Handle LDAP minssf failures during discovery, treat failure to check
ldap server as a warninbg in absebce of a provided CA certificate via
--ca-cert-file or though existing /etc/ipa/ca.crt file.
Signed-off-by: Simo Sorce <simo@redhat.com>
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Target Group parameter was not processed correctly which caused
permission-find to always crash when this search parameter was used.
Fix the crash and create a unit test case to avoid future regression.
https://fedorahosted.org/freeipa/ticket/3335
A wrong way of handling --group DN object caused Internal Error
for this command. Fix that and also provide unit tests to avoid
another regression.
https://fedorahosted.org/freeipa/ticket/3311
When multiple servers are avilable we were simply suppressing information on
why a connection failed.
Log it as 'info' so that it is possible to diagnose issues more easily.
Creating an id range with overlapping primary and secondary
rid range using idrange-add or idrange-mod command now
raises ValidationError. Unit tests have been added to
test_range_plugin.py.
https://fedorahosted.org/freeipa/ticket/3171
All *-find commands now enable leading/trailing whitespaces in the
search phrase. Behaviour has been implemented directly into
crud.Search class. IPA_API_VERSION_MINOR incremented to 45.
https://fedorahosted.org/freeipa/ticket/2981
In summary this patch does:
* Follow the defined rules for cookies when:
- receiving a cookie (process the attributes)
- storing a cookie (store cookie + attributes)
- sending a cookie
+ validate the cookie domain against the request URL
+ validate the cookie path against the request URL
+ validate the cookie expiration
+ if valid then send only the cookie, no attribtues
* Modifies how a request URL is stored during a XMLRPC
request/response sequence.
* Refactors a bit of the request/response logic to allow for making
the decision whether to send a session cookie instead of full
Kerberous auth easier.
* The server now includes expiration information in the session cookie
it sends to the client. The server always had the information
available to prevent using an expired session cookie. Now that
expiration timestamp is returned to the client as well and now the
client will not send an expired session cookie back to the server.
* Adds a new module and unit test for cookies (see below)
Formerly we were always returning the session cookie no matter what
the domain or path was in the URL. We were also sending the cookie
attributes which are for the client only (used to determine if to
return a cookie). The attributes are not meant to be sent to the
server and the previous behavior was a protocol violation. We also
were not checking the cookie expiration.
Cookie library issues:
We need a library to create, parse, manipulate and format cookies both
in a client context and a server context. Core Python has two cookie
libraries, Cookie.py and cookielib.py. Why did we add a new cookie
module instead of using either of these two core Python libaries?
Cookie.py is designed for server side generation but can be used to
parse cookies on the client. It's the library we were using in the
server. However when I tried to use it in the client I discovered it
has some serious bugs. There are 7 defined cookie elements, it fails
to correctly parse 3 of the 7 elements which makes it unusable because
we depend on those elements. Since Cookie.py was designed for server
side cookie processing it's not hard to understand how fails to
correctly parse a cookie because that's a client side need. (Cookie.py
also has an awkward baroque API and is missing some useful
functionality we would have to build on top of it).
cookielib.py is designed for client side. It's fully featured and obeys
all the RFC's. It would be great to use however it's tightly coupled
with another core library, urllib2.py. The http request and response
objects must be urllib2 objects. But we don't use urllib2, rather we use
httplib because xmlrpclib uses httplib. I don't see a reason why a
cookie library should be so tightly coupled to a protocol library, but
it is and that means we can't use it (I tried to just pick some isolated
entrypoints for our use but I kept hitting interaction/dependency problems).
I decided to solve the cookie library problems by writing a minimal
cookie library that does what we need and no more than that. It is a
new module in ipapython shared by both client and server and comes
with a new unit test. The module has plenty of documentation, no need
to repeat it here.
Request URL issues:
We also had problems in rpc.py whereby information from the request
which is needed when we process the response is not available. Most
important was the requesting URL. It turns out that the way the class
and object relationships are structured it's impossible to get this
information. Someone else must have run into the same issue because
there was a routine called reconstruct_url() which attempted to
recreate the request URL from other available
information. Unfortunately reconstruct_url() was not callable from
inside the response handler. So I decided to store the information in
the thread context and when the request is received extract it from
the thread context. It's perhaps not an ideal solution but we do
similar things elsewhere so at least it's consistent. I removed the
reconstruct_url() function because the exact information is now in the
context and trying to apply heuristics to recreate the url is probably
not robust.
Ticket https://fedorahosted.org/freeipa/ticket/3022
When user from other realm than FreeIPA's tries to use Web UI
(login via forms-based auth or with valid trusted realm ticket),
the 401 Unauthorized error with X-Ipa-Rejection-Reason=denied
is returned.
Also, the support for usernames of the form user@SERVER.REALM
or user@server.realm was added.
https://fedorahosted.org/freeipa/ticket/3252
The best way to configure DNS depends on the environment and no general
recommendations should be given by the CLI or Web UI. Especially
forwarders should not be recommended by only be option of last resort.
Fixes https://fedorahosted.org/freeipa/ticket/3261
Web UI part of 'disable global forwaring per zone' effort.
Option "Forwarding disabled" was added to 'DNS global config' and 'DNS zone' forwarding policy. It corresponds to 'none' value of idnsforwardpolicy.
https://fedorahosted.org/freeipa/ticket/3209
bind-dyndb-ldap allows disabling global forwarder per-zone. This may
be useful in a scenario when we do not want requests to delegated
sub-zones (like sub.example.com. in zone example.com.) to be routed
through global forwarder.
Few lines to help added to explain the feature to users too.
https://fedorahosted.org/freeipa/ticket/3209
Nameserver hostname passed to dnszone_add command was always treated
as FQDN even though it was a relative DNS name to the new zone. All
relative names were being rejected as unresolvable.
Modify --name-server option processing in dnszone_add and dnszone_mod
to respect FQDN/relative DNS name and do the checks accordingly. With
this change, user can add a new zone "example.com" and let dnszone_add
to create NS record "ns" in it, when supplied with its IP address. IP
address check is more strict so that it is not entered when no forward
record is created. Places misusing the option were fixed.
Nameserver option now also accepts zone name, which means that NS and A
record is placed to DNS zone itself. Also "@" is accepted as a nameserver
name, BIND understand it also as a zone name. As a side-effect of this
change, other records with hostname part (MX, KX, NS, SRV) accept "@"
as valid hostname. BIND replaces it with respective zone name as well.
Unit tests were updated to test the new format.
https://fedorahosted.org/freeipa/ticket/3204
Since trust-add re-establishes the trust every time it is run and all the other
information fetched from the remote domain controller stays the same, it
can be run multiple times. The only change would occur is update of
trust relationship credentials -- they are supposed to be updated
periodically by underlying infrastructure anyway.
A sequence is following:
1. Match external member against existing trusted domain
2. Find trusted domain's domain controller and preferred GC hosts
3. Fetch trusted domain account auth info
4. Set up ccache in /var/run/ipa_memcached/krb5cc_TD<domain> with principal ourdomain$@trusted.domain
5. Do LDAP SASL interactive bind using the ccache
6. Search for the member's SID
7. Decode SID
8. Replace external member name by SID
The unit tests were failing when executed against an Apache server
in F-18 due to dangling references causing NSS shutdown to fail.
https://fedorahosted.org/freeipa/ticket/3180
Instructions to generate certificate were simplified.
New instructions:
1) Create a certificate database or use an existing one. To create a new database:
# certutil -N -d <database path>
2) Create a CSR with subject CN=<hostname>,O=<realm>, for example:
# certutil -R -d <database path> -a -g <key size> -s 'CN=dev.example.com,O=DEV.EXAMPLE.COM'
3) Copy and paste the CSR (from -----BEGIN NEW CERTIFICATE REQUEST----- to -----END NEW CERTIFICATE REQUEST-----) into the text area below:
https://fedorahosted.org/freeipa/ticket/3056
When long additional text should follow the error message, one can
supply instructions parameter to a class derived from PublicError.
This will cause following text added to the error message:
Additional instructions:
<additional text>
`instructions' optional parameter could be a list or anything that coerces
into unicode(). List entries will be joined with '\n'.
https://fedorahosted.org/freeipa/ticket/3167
This patch contains additional minor fixes which were proposed during
review but were not pushed (accidentaly). Also amends a name of the
default SMB group in a list of protected groups in group.py.
https://fedorahosted.org/freeipa/ticket/3147
Establishing trust implies discovery of the trusted domain's domain controller via DNS.
If DNS discovery is not possible, NotFound exception is raised.
Intercept the exception and process it to help diagnose and fix actual problem:
- if IPA is managing DNS, suggest to make a forward for the domain's zone
- otherwise suggest to setup DNS forwarder at upstream DNS server
https://fedorahosted.org/freeipa/ticket/3103
browserconfig.html was changed to use new FF extension. The page is completely Firefox specific therefore the title was changed from 'Configure browser' to 'Firefox configuration'. Instruction to import CA cert in unauthorized.html are FF specific too, so they were moved to browserconfig.html. Unauthorized.html text was changed to distinguish FF config and other browsers. Now the page shows link for FF (browserconfig.html) and other browsers (ssbrowser.html). Ssbrowser.html should be enhanced by more configurations and browsers later [1].
Old configuration method was moved to ssbrowser.html.
Unauthorized dialog in Web UI now links to http://../unauthorized.html instead of https. This change is done because of FF strange handling of extension installations from https sites [2]. Firefox allows ext. installation from https sites only when the certificate is signed by some build-in CA. To allow custom CAs an option in about:config has to be changed which don't help us at all because we wants to avoid manual changes in about:config.
The design of browserconfig is inspired by Kyle Baker's design (2.1 Enhancements_v2.odt). It is not exactly the same. Highlighting of the steps wasn't used because in some cases we can switch some steps.
Ticket: https://fedorahosted.org/freeipa/ticket/3094
[1] https://fedorahosted.org/freeipa/ticket/823
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=688383
This patch addresses two issues:
1. If a client is previously enrolled in an IPA server and the server
gets re-installed then the client machine may still have a keyring
entry for the old server. This can cause a redirect from the
session URI to the negotiate one. As a rule, always clear the keyring
when enrolling a new client.
2. We save the NSS dbdir in the connection so that when creating a new
session we can determine if we need to re-initialize NSS or not. Most
of the time we do not. The dbdir was not always being preserved between
connections which could cause an NSS_Shutdown() to happen which would
fail because of existing usage. This preserves the dbdir information when
a new connection is created as part of the session mechanism.
https://fedorahosted.org/freeipa/ticket/3108
From IPA 3.0, services have by default ipakrbprincipal objectclass which
allows ipakrbprincipalalias attribute used for case-insensitive principal
searches. However, services created in previous version do not have
this objectclass (and attribute) and thus case-insensitive searches
may return inconsistent results.
Fill ipakrbprincipalalias on upgrades for all 2.x services. Also treat
Treat the ipakrbprincipal as optional to avoid missing services in
service-find command if the upgrade fails for any reason.
https://fedorahosted.org/freeipa/ticket/3106
Group-mod command no longer allows --rename and/or --external
changes made to the admins group. In such cases, ProtectedEntryError
is being raised.
https://fedorahosted.org/freeipa/ticket/3098
On adding new user, user-add tries to make it a member of default
user group. This, however, can raise AlreadyGroupMember when the
user is already member of this group due to automember rule or
default group configured. This patch makes sure AlreadyGroupMember
exception is caught in such cases.
https://fedorahosted.org/freeipa/ticket/3097
PAC type (ipakrbauthzdata attribute) was being filled for all new
service automatically. However, the PAC type attribute was designed
to serve only as an override to default PAC type configured in
IPA config. With PAC type set in all services, users would have
to update all services to get new PAC types configured in IPA config.
Do not set PAC type for new services. Add new NONE value meaning that
we do not want any PAC for the service (empty/missing attribute means
that the default PAC type list from IPA config is read).
https://fedorahosted.org/freeipa/ticket/2184
Every <plugin>-del command executes an "(objectclass=*)" search
to find out if a deleted node has any child nodes which would need
to be deleted first. This produces an unindexed search for every del
command which biases access log audits and may affect performance too.
Since most of the *-del commands delete just a single object (user,
group, RBAC objects, SUDO or HBAC objects, ...) and not a tree
(automount location, dns zone, ...) run a single entry delete first
and only revert to subtree search&delete when that fails.
Do not print list of possible values as "%r" but simply as a list
of quoted values which should make it easier to read for users.
Also add a special case when there is just one allowed value.
https://fedorahosted.org/freeipa/ticket/2869
Our installation added two final dots to the NS records,
so the records were invalid, Bind ignored the entire zone,
and name resolution didn't work.
Fix this error and add a check for empty DNS labels to the validator
config-mod is capable of changing default SELinux user map order
and a default SELinux user. Validate the new config values to
prevent bogus default SELinux users to be assigned to IPA users.
https://fedorahosted.org/freeipa/ticket/2993
