ticket 1650 (https://fedorahosted.org/freeipa/ticket/1650) has
an extensive discussion of the issues, please refer to that.
This patch does the following:
* does not count fuzzy translations when computing translation
statistics via the "msg-stats" make target in install/po
* adds a new make target called "pull-po" which pulls updated po files
from Transifex (configure.ac includes some trailing whitespace fixes)
* turns off the generation of fuzzy translation suggestions during the
message merge phase.
Our LINGUAS file and the set of po files have diverged from what's on
Transifex. We should update the LINGUAS file to match the set of
translations on Transifex and add po files currently on Transifex but
not in our git repo to our git repo.
dns.py at line 976 has an invalid i18n string and cannot be processed
during message extraction causing message catalog generation to fail.
The format parameters are trapped inside the i18n string. Also it's
not necessary to promote the i18n string literal to unicode via the u
prefix because the _() function returns unicode.
We have a larger goal of replacing all DN creation via string
formatting/concatenation with DN object operations because string
operations are not a safe way to form a DN nor to compare a DN. This
work needs to be broken into smaller chunks for easier review and
testing.
Addressing the unit tests first makes sense because we don't want to
be modifying both the core code and the tests used to verify the core
code simultaneously. If we modify the unittests first with existing
core code and no regressions are found then we can move on to
modifying parts of the core code with the belief the unittests can
validate the changes in the core code. Also by doing the unittests
first we also help to validate the DN objects are working correctly
(although they do have an extensive unittest).
The fundamental changes are:
* replace string substitution & concatenation with DN object
constructor
* when comparing dn's the comparision is done after promotion
to a DN object, then two DN objects are compared
* when a list of string dn's are to be compared a new list is
formed where each string dn is replaced by a DN object
* because the unittest framework accepts a complex data structure of
expected values where dn's are represeted as strings the unittest
needs to express the expected value of a dn as a callable object
(e.g. a lambda expression) which promotes the dn string to a DN
object in order to do the comparision.
The DN unittest was lacking a test for i18n. The unittest was
updated to store "Hello" in Arabic with both utf-8 and unicode
and verify the values could be properly retrieved and converted
to dn string syntax.
During the testing a few problems were discovered and corrected.
* passing in utf-8 caused an ASCII decode error becuase of Python's
silly default encoding of ASCII. The fix was to explictly use
the utf-8 codec.
* there were a couple of places where encode/decode were not
called correctly.
* the internal attr and value members of the AVA class were renamed
to explicitly show they are stored as unicode.
Of course the unittest was updated as well.
Pull the new translations for Spanish (es) and Ukrainian (uk)
Update the LINGUAS file to add comment showing the friendly
name for the language abbreviation.
The make target msg-stats which produces a report about the state
of the translations no longer maintained it's column alignment
due to larger numbers so the formating was tweaked to maintain
column alignment.
Replace deepcopy with constructor (i.e. type call)
Can now "clone" with configuration changes by passing object
of the same type to it's constructor, e.g.
dn1 = DN(('cn', 'foo'))
dn2 = DN(dn1)
dn2 = DN(dn1, first_key_match=False)
Remove pairwise grouping for RDN's. Had previously removed it
for DN's, left it in for RDN's because it seemed to make sense
because of the way RDN's work but consistency is a higher goal.
Add keyword constructor parameters to pass configuration options.
Make first_key_match a configuration keyword.
Updated documentation.
Updated unit test.
FWIW, I noticed the unittest is now running 2x faster, not sure why,
removal of deepcopy? Anyway, hard to argue with performance doubling.
The pairwise grouping used to form RDN's and AVA's proved to be
confusing in practice, this patch removes that functionality thus
requiring programmers to explicitly pair attr,value using a tuple or
list.
In addition it was discovered additional functionality was needed to
support some DN operations in freeipa. DN objects now support
startswith(), endswith() and the "in" membership test. These functions
and operators will accept either a DN or RDN.
The unittest was modified to remove the pairwise tests and add new
explicit tests. The unittest was augmented to test the new
functionality. In addition the unittest was cleaned up a bit to use
common utilty functions for improved readabilty and robustness.
The documentation was updated.
fix test_role_plugin use of DN to avoid pairwise grouping
Introduce a comma into a privilege name to assure we can handle
commas.
Commas must be escaped for some parameters, add escape_comma() utility
and invoke it for the necessary parameters.
Utilize a DN object to properly construct a DN and most importantly to
allow equality testing beween the DN we expect and the one
returned. This is necessary because a DN can be encoded according to
different encoding syntaxes all of which are valid. DN objects always
decode from their input. DN objects can test for equality between DN's
without being affected by DN encoding.
Add a equality callback for the dn in the expected dict. When the test
framework tests for equality between the expected value and the
returned value it will call back into a function we provide which will
convert the returned dn into a DN object. An equality test is then
performed between two DN objects. This is the only way to properly
compare two dn's.
Do not call status after pkisilent, it will return non-zero.
Instead restart server after pkisilent so configuration
changes take effect, the check the status.
We want to manually make the .pot file, we shouldn't have anything
in the Makefile which will cause the .pot file to be rebuilt
because of dependencies.
A number of doc strings were not localized, wrap them in _().
Some messages were not localized, wrap them in _()
Fix a couple of failing tests:
The method name in RPC should not be unicode.
The doc attribute must use the .msg attribute for comparison.
Also clean up imports of _() The import should come from
ipalib or ipalib.text, not ugettext from request.
As a consequence of using doc=_('some message') the _()
method was returning a Gettext instance, thus when optparse
was handed the help text it received a Gettext instance instead
of a basestring. optparse tried to operate on the Gettext instance
as if it were a basestring and since it wasn't threw an exception.
The fix is to promote (e.g. cast) the option.doc to unicode.
If the option.doc was a str it becomes unicode, if it was unicode
nothing happens, if it was Gettext (or any other object implementing
the __unicode__() method) object is converted to unicode via the
objects rules.
By the way, while debugging this I discovered strings which were not
localized, sometimes option.doc would be a str and sometimes a Gettext
object. In a subsequent patch I'll fix all those unlocalized doc
strings, but I don't want to bury this fix along with a load of
string fixes.
Traverse the objects passed to JSON for encoding and decoding.
When binary data is seen during encode replace the binary
data with a dict {'__base64__' : base64_encoding_of_binary_value}.
On decode if a dict is seen whose single key is '__base64__' replace
that dict with the base64 decoded value of the key's value.
There was a typo in install/po/Makefile.in which caused (some) of
the .po files to be overwritten because the test to see if a po
file existed had a typo in it.
This patch also removes the unnecessary rebuilding of the pot which was
happening when using the "all" target (the default). The pot file now
must be manually remade, which is what we want.
Added a new target "mo-files" to manually generate the .mo files.
This is useful to run before checking in a new .po file just to
assure it "compiles" and we don't have to discover this during a
build.
Formerly running 'make update-pot' would write an insignificantly
different pot file even if the msgid's xgettext found were unmodfied.
Now the result of running xgettext is compared to the existing
pot file after adjusting for things like timestamps, and only
copies the result of xgettext to the new pot file if there
were differences.
This will help eliminate git commits on the pot file if all one
did was see if the pot file was up to date, if it was up to date
git won't see any modifications. It used to be that timestamps
would be different in the pot file just by virtue of checking
if the pot file was current.
fix exit status; replace POT with $(DOMAN).pot
Add new Russian translation.
Update the Polish translation.
Add count of how many po translations we have in msg-stats.
Current translation statistics with this patch:
ipa.pot has 133 messages. There are 5 po translation files.
bn_IN: 14/133 10.5% 106 po untranslated, 13 missing, 119 untranslated
id: 107/133 80.5% 13 po untranslated, 13 missing, 26 untranslated
kn: 4/133 3.0% 116 po untranslated, 13 missing, 129 untranslated
pl: 133/133 100.0% 0 po untranslated, 0 missing, 0 untranslated
ru: 120/133 90.2% 0 po untranslated, 13 missing, 13 untranslated
The goal is to get the statistics to line up in
columns and not exceed an 80 character line which might
cause wrapping. Removing .po suffix from the translation
name gives us 3 extra characters. Formatting problems were
observed when bn_IN.po was added.
The Makefile in install/po has a new target "msg-stats" which
prints out statistics concerning the current pot and po files.
Here is an example:
% make msg-stats
ipa.pot has 133 messages
id.po: 107/133 80.5% 13 po untranslated, 13 missing, 26 untranslated
kn.po: 4/133 3.0% 116 po untranslated, 13 missing, 129 untranslated
pl.po: 120/133 90.2% 0 po untranslated, 13 missing, 13 untranslated
Also update configure.ac to search for msgcmp, awk & sed programs.
Add 3 new translations for Polish, Indonesian, Kannada.
Remove the dummy Italian translation (it.po), it's only purpose was to
have some translation in the tarball so %{find_lang} wouldn't fail.
Add a new file contributing_translators.txt so we can track our
translators contributions and give them credit.
The %{find_lang} macro in the ipa.spec file will fail if there
are no language translations causing the build to fail.
This patch creates an Italian translation (in honor of Simo).
The Italian translation is initially fully untranslated.
msginit should have been passed the locale because the resulting
.po file is parameterized from the locale. Also, if the target
locale is not specified it defaults to the current locale.
If the target locale is Engish msgid's are copied to their msgstr's
resulting in a fully translated .po instead of a fully untranslated
.po.
Add some comments to better explain some of the cryptic sed commands.
A new directory install/po has been added which contains all
the translations for all files in IPA.
The build has been agumented to build these files. Also the
autogen.sh script was mostly replaced by autoreconf, the preferred
method. The old autogen.sh sript also had some serious bugs in the
way it compared versions which caused it to run old versions of some
of the tools, using standared autoreconf is much better.
This error message was producing a warning from xgettext
because there were multiple substations in the string.
In some languages it may be necessary to reorder the
substitutions for a proper translation, this is only
possible if the substitutions use named values.
Remove SAFE_STRING_PATTERN, safe_string_re, needs_base64(),
wrap_binary_data(), unwrap_binary_data() from both instances
of ipautil.py. This code is no longer in use and the
SAFE_STRING_PATTERN regular expression string was causing xgettext
to abort because it wasn't a valid ASCII string.
This modifies the original patch by including a unit test, handling floats
when passed as unicode, and handling large magnitude values beyond maxint.
The INT parameter class was not respecting any radix prefix (e.g. 0x) the user
may have supplied. This patch implements _convert_scalar method for the Int
class so that we can pass the special radix base of zero to the int constructor
telling it to determine the radix from the prefix (if present).
Signed-off-by: John Dennis <jdennis@redhat.com>
along with stdout,stderr also log the initial command
This implements better logging of external commands. Formerly we were
just outputting stdout and stderr without labeling which was which. We
also omitted the initial command and it's arguments. This made it
difficult when reviewing the logs to know what the command was and
what was stdout vs. stderr. This patch fixes that.
Signed-off-by: John Dennis <jdennis@redhat.com>
If plugin fails to load log the traceback
If a plugin fails to load due to some kind of error it would be nice
if the error log contained the traceback so you can examine what went
wrong rather than being left blind as to why it failed to load.
profile command line tools to select between shared
and per user profiles
modify AttributeValueCompleter so default values prefer
previously entered values in editing session
get_radius_profile_by_uid
add_radius_profile
update_radius_profile
delete_radius_profile
find_radius_profiles
Rewrite command line arg handling, now support pair entry, interactive
mode with auto completion, reading pairs from a file, better handling
of mandatory values, better help, long arg names now match attribute
name in pairs
Establish mappings for all attributes and names used in clients and
profiles
Add notion of containers to radius clients and profiles in LDAP
Move common code, variables, constants, and strings into the files
radius_client.py, radius_util.py, ipautil.py to eliminate redundant
elements which could get out of sync if modified and to provide access
to other code which might benefit from using these items in the
future.
Add utility functions:
format_list()
parse_key_value_pairs()
Add utility class:
AttributeValueCompleter
Unify attribute usage in radius ldap schema
John Dennis
jdennis@VAIO