[Po4a-dev][po-debconf] Status report

[Denis Barbier]

Hi fellow developers,

po-debconf is almost finished, here are some words about it, and about
po4a in general.


  1.  Introduction
      ------------

My main concern is to have standardized tools for i18n/l10n so that
developers don't have to spend time learning how to be fair with foreign
people.

Gettext does this job for programs, but other source files (documentation,
data, ...) lack similar tools.  Intltool is used by the GNOME project and
seems to be a good candidate for becoming a standard in this area.
For this reason, I send patches to intltool authors in order to add
support for rfc822-compliant Debian system files.  Feedback was positive,
and they should be incorporated soon.  When it is done, po-debconf would
mainly become a wrapper for intltool scripts.

  2.  Debconf templates files: current i18n
      -------------------------------------

  ==================== [ Sample templates file ] ===========================
  Template: debconf/showold
  Type: boolean
  Default: false
  Description: Show all old questions again and again?
   Debconf normally only asks you any given question once. Then it remembers
   your answer and never asks you that question again. If you prefer, debconf
   can ask you questions over and over again, each time you upgrade or
   reinstall a package that asks them.
   .
   Note that no matter what you choose here, you can see old questions again
   by using the dpkg-reconfigure program.

  Template: debconf/priority
  Type: select
  Choices: critical, high, medium, low
  Default: medium
  Description: Ignore questions with a priority less than..
   Packages that use debconf for configuration prioritize the questions they
   might ask you. Only questions with a certain priority or higher are
   actually shown to you; all less important questions are skipped.
   .
   You can select the lowest priority of question you want to see:
     - 'critical' is for items that will probably break the system
       without user intervention.
     - 'high' is for items that don't have reasonable defaults.
     - 'medium' is for normal items that have reasonable defaults.
     - 'low' is for trivial items that have defaults that will work in the
       vast majority of cases.
   .
   For example, this question is of medium priority, and if your priority
   were already 'high' or 'critical', you wouldn't see this question.
   .
   If you are new to the Debian GNU/Linux system choose 'critical' now, so
   you only see the most important questions.
  ============================= [ End ] ====================================

Only some fields have to be translated, current list is:
  * Description
  * Choices (if template type is select or multiselect)
  * Default (except when template type is select, multiselect, note or
    boolean)

Localized entries have a -ll (or -ll_LL) suffix, e.g.
  Choices-fr: critique, haut, moyen, bas
  Choices-pt_BR: crítica, alta, média, baixa

  3.  Drawbacks of native i18n
      ------------------------

This predefined list is sometimes inaccurate, some entries should not be
translated (see e.g. XFree questions) whereas others should be (e.g.
Default field has to be localized when answer is locale dependent).

Other drawbacks:
  * Encoding is not specified, so problems will occur as more people
    are using UTF-8 fonts instead of their common locale.
    Next Debian installer (but not Debian packages) should ship
    UTF-8 encoded templates, but nobody knows how to proceed.
  * There is no information on last translator, and who to contact
    for fixes.
  * With large descriptions, finding changes in original text may
    become painful.
  * Master and localized templates are similar, thus developers often
    merge localized templates files instead of keeping them separate.
    

  4.  Debconf templates files: i18n with po-debconf
      ---------------------------------------------

For po-debconf I chose a scheme similar to i18n of .desktop files:
translatable entries are prepended with an underscore.  In order
to follow common rules, templates file should have been renamed into
templates.in.  But here is the first problem, since this filename is
seen by debconf-utils tools as a translated templates file.
For this reason another filename is in use: _templates.  Hopefully
there should be no clash then.

Long fields are split into small pieces:
  * first line is kept alone
  * PO msgids are split by paragraph, and text is reformatted to remove
    spurious newlines

  ==================== [ Sample _templates file ] ==========================
  Template: debconf/showold
  Type: boolean
  Default: false
  _Description: Show all old questions again and again?
   Debconf normally only asks you any given question once. Then it remembers
   your answer and never asks you that question again. If you prefer, debconf
   can ask you questions over and over again, each time you upgrade or
   reinstall a package that asks them.
   .
   Note that no matter what you choose here, you can see old questions again
   by using the dpkg-reconfigure program.

  Template: debconf/priority
  Type: select
  _Choices: critical, high, medium, low
  Default: medium
  _Description: Ignore questions with a priority less than..
   Packages that use debconf for configuration prioritize the questions they
   might ask you. Only questions with a certain priority or higher are
   actually shown to you; all less important questions are skipped.
   .
   You can select the lowest priority of question you want to see:
     - 'critical' is for items that will probably break the system
       without user intervention.
     - 'high' is for items that don't have reasonable defaults.
     - 'medium' is for normal items that have reasonable defaults.
     - 'low' is for trivial items that have defaults that will work in the
       vast majority of cases.
   .
   For example, this question is of medium priority, and if your priority
   were already 'high' or 'critical', you wouldn't see this question.
   .
   If you are new to the Debian GNU/Linux system choose 'critical' now, so
   you only see the most important questions.
  ============================= [ End ] ====================================

An fr.po file might look like:

  ====================== [ Matching fr.po file ] ===========================
  msgid ""
  msgstr ""
  "Project-Id-Version: templates for package: debconf 1.2.0\n"
  "POT-Creation-Date: 2002-08-26 14:41+0200\n"
  "PO-Revision-Date: 2002-08-26 15:26+0200\n"
  "Last-Translator: Denis Barbier <address@hidden>\n"
  "Language-Team: french <address@hidden>\n"
  "MIME-Version: 1.0\n"
  "Content-Type: text/plain; charset=ISO-8859-1\n"
  "Content-Transfer-Encoding: 8bit\n"
  
  msgid "Show all old questions again and again?"
  msgstr "Poser de nouveau les anciennes questions ?"
  
  msgid ""
  "Debconf normally only asks you any given question once. Then it "
  "remembers your answer and never asks you that question again. If you "
  "prefer, debconf can ask you questions over and over again, each time "
  "you upgrade or reinstall a package that asks them."
  msgstr ""
  "Normalement, debconf ne pose chaque question qu'une seule fois. "
  "Ensuite, il se  souvient de la réponse que vous avez donnée, et ne "
  "repose jamais cette question. Si vous préférez, debconf peut reposer "
  "chaque question encore et encore, chaque fois qu'un paquet ayant "
  "besoin de cette réponse est installé ou mis à jour."
  
  msgid ""
  "Note that no matter what you choose here, you can see old questions "
  "again by using the dpkg-reconfigure program."
  msgstr ""
  "Notez que quel que soit votre choix ici, vous pouvez revoir la "
  "configuration  d'un paquet avec le programme dpkg-reconfigure."
  
  msgid "critical, high, medium, low"
  msgstr "critique, haut, moyen, bas"
  
  ============================= [ snip ] ====================================

We need tools to:
  * convert templates files between native and po-debconf formats
  * extract translatable text into a POT file
  * merge _templates and *.po into templates
  * ease life of developers and translators by providing very high-level
    commands

When converting to po-debconf format, package maintainer has to specify
translatable entries and PO file encoding.  Unless the latter is done,
gettext won't be able to process PO files and final templates cannot
be generated.

  5.  Po-debconf internals
      --------------------

Following programs are shipped by po-debconf:
  * debconf2po
      Convert from native to po-debconf formats.  Performed only once.
  * debconf2pot
      Extract translatable text from _templates and writes templates.pot.
      Wrapper to intltool-extract
  * debconf2po-update
      Update existing *.po files after generating a new templates.pot.
      Wrapper to intltool-update
  * po2debconf
      Merge _templates and *.po files into templates.
      Wrapper to intltool-merge
  * dh_installpodebconf
      Debhelper script to run po2debconf when building binary packages.
  * TODO: po2debconf-revert
      Convert from po-debconf to native formats.

Before looking into these programs in depth, please note that
  a.  For some intltool operations, a POTFILES.in file is mandatory.
  b.  As other Debian files may later use a similar i18n scheme (e.g.
      control file might be l10n'ed one day ;)), PO files are put in
      a subdirectory arbitrarily named debian/po-templates/

debconf2po:
  * Read templates files and generate po-templates/*.po
  * TODO: improve handling of fuzzy entries, they are currently ignored
    and should be marked as fuzzy
  * Generate po-templates/POTFILES.in; this file must not be deleted
  * Generate _templates
  * When done, package maintainer must
    a. edit _templates and check accuracy of marked entries
    b. fix encoding specification within po-templates/*.po
       (TODO: might be done automatically)
    c. delete old templates, templates.xx and templates.xx_XX files
       Otherwise dh_installdebconf will process these files.

debconf2pot
  * Read _templates and generate po-templates/templates.pot

debconf2po-update
  * Read po-templates/POTFILES.in
  * Parse files found in po-templates/POTFILES.in
  * Generate po-templates/templates.pot
  * Merge po-templates/templates.pot with existing po-templates/*.po

po2debconf
  * Read po-templates/*.po files
  * Merge _templates and up to date translations, output is sent
    to stdout

dh_installpodebconf (run from top-level directory)
  * If debian/_templates does not exist, do nothing
  * run po2debconf with adequate arguments
  * send output into package build directory

  6.  Roadmap
      -------

Here is my current roadmap:
  * Add patched intltool scripts until they are shipped by upstream
  * Add regression tests
  * Write documentation
  * Debianize it and send an ITP

  7.  Discussion
      ----------

All comments on this presentation are welcome.  There is another intereting
topic we have to consider: do we want homogeneous behaviours?
In other words, do we want to require that all our po2xx and xx2po scripts
follow the same naming scheme and require the same command-line arguments?

Denis