Re: gettext 0.10.37 msgfmt charset naming compatibility on Solaris 8

[Bruno Haible]

Paul Eggert writes:

> > > How about instead modifying config.charset so that it will output the
> > > tables in question?
> > 
> > That would be twice as much maintenance work, for little gain.
> 
> You're right that it's a bit more maintenance work, if config.charset
> is maintained by hand.  But it's not that much more maintenance work,
> as the reverse tables are not that hard to derive.  I can even modify
> config.charset to generate the reverse tables automatically from the
> forward tables

As I already explained, this is not sufficient, because often iconv()
supports encodings which are not present as installed locales.

I can assure you that maintaining a table of iconv encoder quality for
many (OS, encoding) pairs is a lot of work. I won't do it. It's better
to rely on autoconf.

> And I'm not sure I see why it's little gain.  It decouples GNU
> applications from requiring libiconv, and that is a real gain.

Compared to config.charset's usefulness - namely the ability to go
further than the ISO C APIs in terms of advanced text handling, in a
locale aware way -, this is little gain.

> That exact syntax won't scale well to east Asian codesets, as the
> bitmask will be too long.  I guess this could be fixed, though

Sure. Use boolean expressions instead, that describe the part of the
256x256 matrix to be checked.

> (if you're willing to settle for spot checks)

No spot checks; that'd be gambling, not a quality check.

> This is a bit discouraging, I'm afraid.  In effect, you're suggesting
> that every GNU application run extensive tests on every vendor's
> conversion tables when the application is built.

We also run tests whether vfork() works, and whether shared libraries
work, when configuring GNU programs.

> I wonder whether it's really necessary.  Conversions don't have to
> be perfect; they just have to be good enough.

Remember gettext is operating on PO files - files which are maintained
by humans (translators), and every iconv bug will show up as corrupted
PO files.

> For example, for this application we needn't check every possible
> character; just the characters that are in the .po files.

That would mean pushing the checking work from configure time to run
time. This is not good.

> As things stand I may have to start recommending using --disable-nls
> on all hosts other than GNU/Linux hosts, and that is also
> discouraging.

It would be more constructive if you ran glibc's test suite (in
particular tst-table-from.c and tst-table-to.c) on your preferred
vendor's OS - I guess, that's Solaris - and reported to them the
various bugs you notice.

Bruno