Make `ipa -h` and `ipa help` output the same message.
Since `ipa -h` output is generated by the OptionParser, we need to make
the parser available. Store it in `api.parser`.
Part of the effort for https://fedorahosted.org/freeipa/ticket/3060
Whenever a command is used incorrectly, it should output
an error message (and possibly additional help) to stderr.
This patch adds a parameter to a bunch of places to allow
selecting either stdout or stderr for help output, and makes
badly called commands output to stderr only.
Part of the effort for https://fedorahosted.org/freeipa/ticket/3060
Fix the usage string to match actual usage.
Add command description.
Put information about `ipa help topics` etc. to the epilog,
instead of using empty option groups. Use a custom formatter
to preserve newlines.
Add the -h/--help option manually to ensure consistent case
(capital S).
Part of the effort for https://fedorahosted.org/freeipa/ticket/3060
When adding/modifying an ID range for a trusted domain, the newly
added option --dom-name can be used. This looks up SID of the
trusted domain in LDAP and therefore the user is not required
to write it down in CLI. If the lookup fails, error message
asking the user to specify the SID manually is shown.
https://fedorahosted.org/freeipa/ticket/3133
Add correct labeling of matched/nonmatched output attributes. Also
make sure that "\" is not interpreted as newline escape character
but really as a "\" character.
How this works:
1. When a trusted domain user is tested, AD GC is searched
for the user entry Distinguished Name
2. The user entry is then read from AD GC and its SID and SIDs
of all its assigned groups (tokenGroups attribute) are retrieved
3. The SIDs are then used to search IPA LDAP database to find
all external groups which have any of these SIDs as external
members
4. All these groups having these groups as direct or indirect
members are added to hbactest allowing it to perform the search
LIMITATIONS:
- only Trusted Admins group members can use this function as it
uses secret for IPA-Trusted domain link
- List of group SIDs does not contain group memberships outside
of the trusted domain
https://fedorahosted.org/freeipa/ticket/2997
When group-add-member does not receive any resolved trusted domain
object SID, it raises an exception which hides any useful error
message passed by underlying resolution methods. Remove the exception
to reveal this error messages to user.
https://fedorahosted.org/freeipa/ticket/2997
Modify access methods to AD GC so that callers can specify a custom
basedn, filter, scope and attribute list, thus allowing it to perform
any LDAP search.
Error checking methodology in these functions was changed, so that it
rather raises an exception with a desription instead of simply returning
a None or False value which would made an investigation why something
does not work much more difficult. External membership method in
group-add-member command was updated to match this approach.
https://fedorahosted.org/freeipa/ticket/2997
Update our LDAP schema and add 2 new attributes for SID blacklist
definition. These new attributes can now be set per-trust with
trustconfig command.
https://fedorahosted.org/freeipa/ticket/3289
Global trust configuration is generated ipa-adtrust-install script
is run. Add convenience commands to show auto-generated options
like SID or GUID or options chosen by user (NetBIOS). Most of these
options are not modifiable via trustconfig-mod command as it would
break current trusts.
Unit test file covering these new commands was added.
https://fedorahosted.org/freeipa/ticket/3333
Add new users to the default users group in batches of 100. The
biggest overhead of migration is in calculating the modlist when
managing the default user's group and applying the changes. A
significant amount of time can be saved by not doing this on every
add operation.
Some other minor improvements include:
Add a negative cache for groups not found in the remote LDAP server.
Replace call to user_mod with a direct LDAP update.
Catch some occurances of LimitError and handle more gracefully.
I also added some debug logging to report on migration status and
performance.
https://fedorahosted.org/freeipa/ticket/3386
Some parts of install scripts used only ccache name as returned by
krbV.CCache.name attribute. However, when this name is used again
to initialize krbV.CCache object or when it is used in KRB5CCNAME
environmental variable, it fails for new DIR type of CCACHE.
We should always use both CCACHE type and name when referring to
them to avoid these crashes. ldap2 backend was also updated to
accept directly krbV.CCache object which contains everything we need
to authenticate with ccache.
https://fedorahosted.org/freeipa/ticket/3381
openldap server does not store its schema in cn=schema entry, but
rather in cn=subschema. Add a fallback to ldap2 plugin to read from
this entry when cn=schema is not found. ldap2 plugin uses the schema
when doing some of the automatic encoding, like an automatic
encoding of DN object.
IPA migration plugin DN attribute processing is now also more
tolerant when it finds that some DN attribute was not autoencoded.
It tries to convert it to DN on its own and report a warning and
continue with user processing when the conversion fails instead of
crashing with AssertionError and thus abandoning the whole
migration run.
https://fedorahosted.org/freeipa/ticket/3372
Add more dynamic attribute info to IPATypeChecker in make-lint. Remove
unnecessary pylint comments. Fix false positivies introduced by Pylint 0.26.
https://fedorahosted.org/freeipa/ticket/3379
Check to see if NSS is initialized before trying to do so again.
If we are temporarily creating a certificate be sure to delete it in order
to remove references to it and avoid NSS shutdown issues.
In the certificate load validator shut down NSS if we end up initializing
it. I'm not entirely sure why but this prevents a later shutdown issue
if we are passed the --ca-cert-file option.
Major changes ipa-client-install:
* Use GSSAPI connection to LDAP server to download CA cert (now
the default method)
* Add --ca-cert-file option to load the CA cert from a disk file.
Validate the file. If this option is used the supplied CA cert
is considered definitive.
* The insecure HTTP retrieval method is still supported but it must be
explicitly forced and a warning will be emitted.
* Remain backward compatible with unattended case (except for aberrant
condition when preexisting /etc/ipa/ca.crt differs from securely
obtained CA cert, see below)
* If /etc/ipa/ca.crt CA cert preexists the validate it matches the
securely acquired CA cert, if not:
- If --unattended and not --force abort with error
- If interactive query user to accept new CA cert, if not abort
In either case warn user.
* If interactive and LDAP retrieval fails prompt user if they want to
proceed with insecure HTTP method
* If not interactive and LDAP retrieval fails abort unless --force
* Backup preexisting /etc/ipa/ca.crt in FileStore prior to execution,
if ipa-client-install fails it will be restored.
Other changes:
* Add new exception class CertificateInvalidError
* Add utility convert_ldap_error() to ipalib.ipautil
* Replace all hardcoded instances of /etc/ipa/ca.crt in
ipa-client-install with CACERT constant (matches existing practice
elsewhere).
* ipadiscovery no longer retrieves CA cert via HTTP.
* Handle LDAP minssf failures during discovery, treat failure to check
ldap server as a warninbg in absebce of a provided CA certificate via
--ca-cert-file or though existing /etc/ipa/ca.crt file.
Signed-off-by: Simo Sorce <simo@redhat.com>
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Target Group parameter was not processed correctly which caused
permission-find to always crash when this search parameter was used.
Fix the crash and create a unit test case to avoid future regression.
https://fedorahosted.org/freeipa/ticket/3335
A wrong way of handling --group DN object caused Internal Error
for this command. Fix that and also provide unit tests to avoid
another regression.
https://fedorahosted.org/freeipa/ticket/3311
When multiple servers are avilable we were simply suppressing information on
why a connection failed.
Log it as 'info' so that it is possible to diagnose issues more easily.
Creating an id range with overlapping primary and secondary
rid range using idrange-add or idrange-mod command now
raises ValidationError. Unit tests have been added to
test_range_plugin.py.
https://fedorahosted.org/freeipa/ticket/3171
All *-find commands now enable leading/trailing whitespaces in the
search phrase. Behaviour has been implemented directly into
crud.Search class. IPA_API_VERSION_MINOR incremented to 45.
https://fedorahosted.org/freeipa/ticket/2981
In summary this patch does:
* Follow the defined rules for cookies when:
- receiving a cookie (process the attributes)
- storing a cookie (store cookie + attributes)
- sending a cookie
+ validate the cookie domain against the request URL
+ validate the cookie path against the request URL
+ validate the cookie expiration
+ if valid then send only the cookie, no attribtues
* Modifies how a request URL is stored during a XMLRPC
request/response sequence.
* Refactors a bit of the request/response logic to allow for making
the decision whether to send a session cookie instead of full
Kerberous auth easier.
* The server now includes expiration information in the session cookie
it sends to the client. The server always had the information
available to prevent using an expired session cookie. Now that
expiration timestamp is returned to the client as well and now the
client will not send an expired session cookie back to the server.
* Adds a new module and unit test for cookies (see below)
Formerly we were always returning the session cookie no matter what
the domain or path was in the URL. We were also sending the cookie
attributes which are for the client only (used to determine if to
return a cookie). The attributes are not meant to be sent to the
server and the previous behavior was a protocol violation. We also
were not checking the cookie expiration.
Cookie library issues:
We need a library to create, parse, manipulate and format cookies both
in a client context and a server context. Core Python has two cookie
libraries, Cookie.py and cookielib.py. Why did we add a new cookie
module instead of using either of these two core Python libaries?
Cookie.py is designed for server side generation but can be used to
parse cookies on the client. It's the library we were using in the
server. However when I tried to use it in the client I discovered it
has some serious bugs. There are 7 defined cookie elements, it fails
to correctly parse 3 of the 7 elements which makes it unusable because
we depend on those elements. Since Cookie.py was designed for server
side cookie processing it's not hard to understand how fails to
correctly parse a cookie because that's a client side need. (Cookie.py
also has an awkward baroque API and is missing some useful
functionality we would have to build on top of it).
cookielib.py is designed for client side. It's fully featured and obeys
all the RFC's. It would be great to use however it's tightly coupled
with another core library, urllib2.py. The http request and response
objects must be urllib2 objects. But we don't use urllib2, rather we use
httplib because xmlrpclib uses httplib. I don't see a reason why a
cookie library should be so tightly coupled to a protocol library, but
it is and that means we can't use it (I tried to just pick some isolated
entrypoints for our use but I kept hitting interaction/dependency problems).
I decided to solve the cookie library problems by writing a minimal
cookie library that does what we need and no more than that. It is a
new module in ipapython shared by both client and server and comes
with a new unit test. The module has plenty of documentation, no need
to repeat it here.
Request URL issues:
We also had problems in rpc.py whereby information from the request
which is needed when we process the response is not available. Most
important was the requesting URL. It turns out that the way the class
and object relationships are structured it's impossible to get this
information. Someone else must have run into the same issue because
there was a routine called reconstruct_url() which attempted to
recreate the request URL from other available
information. Unfortunately reconstruct_url() was not callable from
inside the response handler. So I decided to store the information in
the thread context and when the request is received extract it from
the thread context. It's perhaps not an ideal solution but we do
similar things elsewhere so at least it's consistent. I removed the
reconstruct_url() function because the exact information is now in the
context and trying to apply heuristics to recreate the url is probably
not robust.
Ticket https://fedorahosted.org/freeipa/ticket/3022
When user from other realm than FreeIPA's tries to use Web UI
(login via forms-based auth or with valid trusted realm ticket),
the 401 Unauthorized error with X-Ipa-Rejection-Reason=denied
is returned.
Also, the support for usernames of the form user@SERVER.REALM
or user@server.realm was added.
https://fedorahosted.org/freeipa/ticket/3252
The best way to configure DNS depends on the environment and no general
recommendations should be given by the CLI or Web UI. Especially
forwarders should not be recommended by only be option of last resort.
Fixes https://fedorahosted.org/freeipa/ticket/3261
Web UI part of 'disable global forwaring per zone' effort.
Option "Forwarding disabled" was added to 'DNS global config' and 'DNS zone' forwarding policy. It corresponds to 'none' value of idnsforwardpolicy.
https://fedorahosted.org/freeipa/ticket/3209
bind-dyndb-ldap allows disabling global forwarder per-zone. This may
be useful in a scenario when we do not want requests to delegated
sub-zones (like sub.example.com. in zone example.com.) to be routed
through global forwarder.
Few lines to help added to explain the feature to users too.
https://fedorahosted.org/freeipa/ticket/3209
Nameserver hostname passed to dnszone_add command was always treated
as FQDN even though it was a relative DNS name to the new zone. All
relative names were being rejected as unresolvable.
Modify --name-server option processing in dnszone_add and dnszone_mod
to respect FQDN/relative DNS name and do the checks accordingly. With
this change, user can add a new zone "example.com" and let dnszone_add
to create NS record "ns" in it, when supplied with its IP address. IP
address check is more strict so that it is not entered when no forward
record is created. Places misusing the option were fixed.
Nameserver option now also accepts zone name, which means that NS and A
record is placed to DNS zone itself. Also "@" is accepted as a nameserver
name, BIND understand it also as a zone name. As a side-effect of this
change, other records with hostname part (MX, KX, NS, SRV) accept "@"
as valid hostname. BIND replaces it with respective zone name as well.
Unit tests were updated to test the new format.
https://fedorahosted.org/freeipa/ticket/3204
Since trust-add re-establishes the trust every time it is run and all the other
information fetched from the remote domain controller stays the same, it
can be run multiple times. The only change would occur is update of
trust relationship credentials -- they are supposed to be updated
periodically by underlying infrastructure anyway.
A sequence is following:
1. Match external member against existing trusted domain
2. Find trusted domain's domain controller and preferred GC hosts
3. Fetch trusted domain account auth info
4. Set up ccache in /var/run/ipa_memcached/krb5cc_TD<domain> with principal ourdomain$@trusted.domain
5. Do LDAP SASL interactive bind using the ccache
6. Search for the member's SID
7. Decode SID
8. Replace external member name by SID
The unit tests were failing when executed against an Apache server
in F-18 due to dangling references causing NSS shutdown to fail.
https://fedorahosted.org/freeipa/ticket/3180
Instructions to generate certificate were simplified.
New instructions:
1) Create a certificate database or use an existing one. To create a new database:
# certutil -N -d <database path>
2) Create a CSR with subject CN=<hostname>,O=<realm>, for example:
# certutil -R -d <database path> -a -g <key size> -s 'CN=dev.example.com,O=DEV.EXAMPLE.COM'
3) Copy and paste the CSR (from -----BEGIN NEW CERTIFICATE REQUEST----- to -----END NEW CERTIFICATE REQUEST-----) into the text area below:
https://fedorahosted.org/freeipa/ticket/3056
When long additional text should follow the error message, one can
supply instructions parameter to a class derived from PublicError.
This will cause following text added to the error message:
Additional instructions:
<additional text>
`instructions' optional parameter could be a list or anything that coerces
into unicode(). List entries will be joined with '\n'.
https://fedorahosted.org/freeipa/ticket/3167
This patch contains additional minor fixes which were proposed during
review but were not pushed (accidentaly). Also amends a name of the
default SMB group in a list of protected groups in group.py.
https://fedorahosted.org/freeipa/ticket/3147
Establishing trust implies discovery of the trusted domain's domain controller via DNS.
If DNS discovery is not possible, NotFound exception is raised.
Intercept the exception and process it to help diagnose and fix actual problem:
- if IPA is managing DNS, suggest to make a forward for the domain's zone
- otherwise suggest to setup DNS forwarder at upstream DNS server
https://fedorahosted.org/freeipa/ticket/3103
Petr Viktorin
Tomas Babej
Martin Kosek
Rob Crittenden
Jan Cholasta
John Dennis
Ana Krivokapic
Simo Sorce
Lynn Root
Sumit Bose
Petr Vobornik
Alexander Bokovoy