mirror of
https://salsa.debian.org/freeipa-team/freeipa.git
synced 2025-01-16 03:11:57 -06:00
e2a3907726
We had been using shell scripts and sed to test our translations. But
trying to edit pot and po files with sed is nearly impossible because
the file format can vary significantly and the sed editing was failing
and gettext tools were complaining about our test strategy. We had
been using a Python script (test_i18n.py) to perform the actual test
after using shell, sed, and gettext tools to create the files. There
is a Python library (polib) which can read/write/edit pot/po/mo files
(used internally by Transifex, our translation portal). The strategy
now is to do everything in Python (in test_i18n.py). This is easier,
more robust and allows us to do more things.
* add python-polib to BuildRequires
* Remove the logic for creating the test lang from Makefile.in and
replace it with calls to test_i18n.py
* add argument parsing, usage, configuration parameters, etc. to
test_i18n.py to make it easier to use and configurable.
* add function to generate a test po and mo file. It also
writes the files and creates the test directory structure.
* Took the existing validate code and refactored it into validation
function. It used to just pick one string and test it, now it
iterates over all strings and all plural forms.
* Validate anonymous Python format substitutions in pot file
* added support for plural forms.
* Add pot po file validation for variable substitution
* In install/po subdir you can now do:
$ make test
$ make validate-pot
$ make validate-po
* The options for running test_i18n.py are:
$ ./test_i18n.py --help
Usage:
test_i18n.py --test-gettext
test_i18n.py --create-test
test_i18n.py --validate-pot [pot_file1, ...]
test_i18n.py --validate-po po_file1 [po_file2, ...]
Options:
-h, --help show this help message and exit
-s, --show-strings show the offending string when an error is detected
--pedantic be aggressive when validating
-v, --verbose be informative
--traceback print the traceback when an exception occurs
Operational Mode:
You must select one these modes to run in
-g, --test-gettext create the test translation file(s) and exercise them
-c, --create-test create the test translation file(s)
-P, --validate-pot validate pot file(s)
-p, --validate-po validate po file(s)
Run Time Parameters:
These may be used to modify the run time defaults
--test-lang=TEST_LANG
test po file uses this as it's basename (default=test)
--lang=LANG lang used for locale, MUST be a valid lang
(default=xh_ZA)
--domain=DOMAIN translation domain used during test (default=ipa)
--locale=LOCALE locale used during test (default=test_locale)
--pot-file=POT_FILE
default pot file, used when validating pot file or
generating test po and mo files (default=ipa.pot)
https://fedorahosted.org/freeipa/ticket/2044
|
||
|---|---|---|
| .. | ||
| as.po | ||
| bn_IN.po | ||
| contributing_translators.txt | ||
| de.po | ||
| el.po | ||
| es.po | ||
| fa.po | ||
| fr.po | ||
| gu.po | ||
| he.po | ||
| id.po | ||
| ipa.pot | ||
| it.po | ||
| ja_JP.po | ||
| ja.po | ||
| kn.po | ||
| ko.po | ||
| LINGUAS | ||
| Makefile.in | ||
| nl.po | ||
| pl.po | ||
| pt_BR.po | ||
| pt.po | ||
| pygettext.py | ||
| README | ||
| ru.po | ||
| sv.po | ||
| test_i18n.py | ||
| uk.po | ||
| zh_CN.po | ||
| zh_TW.po | ||
Q: I've added a new source file, how do I make sure it's strings get translated?
A: Edit Makefile.in and add the source file to the appropriate *_POTFILES list.
Then run "make update-po".
NOTE: Now this i only necessary for python files that lack the .py
extension. All .py, .c and .h files are automatically sourced.
Q: How do I pick up new strings to translate from the source files after the
source have been modified?
A: make update-po
This regenerates the pot template file by scanning all the source files.
Then the new strings are merged into each .po file from the new pot file.
Q: How do I just regenerate the pot template file without regenerating all the
.po files?
A: make update-pot
Q: How do I add a new language for translation?
A: Edit the LINGUAS file and add the new language. Then run "make create-po".
This will generate a new .po file for each language which doesn't have one
yet. Be sure to add the new .po file(s) to the source code repository. For
certain languages, you may have to edit the Plurals line. See:
http://www.gnu.org/software/hello/manual/gettext/Plural-forms.html
However, if this line is wrong, it is often an indicator that the locale
value is incorrect. For example, using 'jp' for Japanese in stead of 'ja'
will result in an invailid Plural's line.
Q: What files must be under source code control?
A: The files Makefile.in, LINGUAS control the build, they must be in the SCM.
The *.pot and *.po files are used by translators, they must be in SCM so the
translator can checkout out a .po files, add the translations, and then check
the .po file back in.
Be careful, .po files may be automatically updated when the source files
change (or the .pot changes, usually the .pot file changes only as a result
of rescanning the source files). This mean a .po file might be automatically
updated while a translator has the file out for editing, all the caveats
about SCM merging apply.
Q: Which are automatically generated and thus do not need to be in SCM?
A: The *.mo files are automatically generated on demand from their corresponding
.po file.
Q: What role does the .pot file play?
A: The .pot file is called a template file. It is generated by scanning all the
source files (e.g. *.py *.c *.h) in the project using xgettext. xgettext
locates every translatable string (e.g. strings marked with _()) and adds
that string along with metadata about it's location to the .pot file. Thus
the .pot file is a collection of every translatable string in the project. If
you edit a source file and add a translatable string you will have to
regenerate the .pot file in order to pick up the new string.
Q: What is the relationship between a .po file and the .pot file?
A: A .po file contains the translations for particular language. It derives from
the .pot file. When the .pot file is updated with new strings to translate
each .po will merge the new strings in. The .po file is where translators
work providing translations for their language. Thus it's important the .po
not be recreated from scratch and is kept in SCM, otherwise the translators
work will be lost.
Let's use an example for French, it's .po file will be fr.po.
1) Developer creates main.c with one translatable sting _("Begin").
2) Produce the .pot file by running xgettext on main.c
3) .pot file contains one msgid, "Begin"
4) fr.po is created from the .pot file, it also contains one msgid, "Begin"
5) Translator edits fr.po and provide the French translation of "Begin".
6) Developer adds new translatable sting _("End") to main.c
7) Generate a new .pot file by running xgettext on main.c
8) .pot file contains two msgid's, "Begin", and "End"
9) fr.po is missing the new msgid in the .pot file, so the .pot is merged
into fr.po by running msgmerge. This copies into fr.po the new "End" msgid
but preserves the existing translations in fr.po (e.g. "Begin"). The fr.po
will now have 2 msgid's one which is translated already (e.g. "Begin") and
one that untranslated (e.g. "End").
10) Sometime later the French translator comes back to see if he/she needs to
add more translations to fr.po. They see there is a missing translation,
they check fr.po out from SCM, add the missing translation, and then
check fr.po back into SCM.
This means at any given moment the set of .po files will have varying degrees
of translation completeness. Because the .po files are merged when the source
code files are updated existing translations are not lost. It also means a
.po file which was fully translated may need new translations after a .pot
update. It is permissible to have incomplete translations in a message
catalog, at run time if a translation for a particular string is available in
the message catalog the user will be presented with the string in their
language. However if the string is not yet translated in the .po file then
they just get the original string (typically in English).
Q: What are .mo files?
A: .mo files are the content of a .po file but in "machine" format for fast
run time access (mo = Machine Object, po = Portable Object). .mo files are
what gets installed along with the package. Think of a .po as a source file
which is compiled into a object file for run time use.
Q: Why don't we use gettexize and autopoint?
A: Because the framework they produce is too limited. Specifically there is no
way to pass the source language to xgettext when it scans a file. xgettext
only knows how to automatically determine the language from the source files
extension. However we have many files without extensions, thus we have to
group all Python (et. al.) files together and run xgettext on every file *we*
know to Python (because xgettext can't figure this out itself if there is no
file extension). There is another added benefit of avoiding gettextize and
autopoint, simplicity. Managing translations is complex and hard enough as it
is, gettextize and autopoint adds another whole layer of complexity which
just further obscures things.
Q: Who created the awful mess and who do I ask when things don't work as I
expect or I have further questions?
A: John Dennis <jdennis@redhat.com>