Re: [PATCH v2 1/8] tests: Make sure LANG is set properly for iso9660_test

[Thomas Schmitt]

Hi,

Glenn Washburn wrote:
> LANG must be set to something that supports international characters,
> otherwise xorriso will refuse to include the file with name having
> international characters, causing the test to fail.

Can you tell me the exact error message from xorriso ?
I have some difficulties with reproducing the situation.

Your description is plausible, because libisofs takes its character set
default from nl_langinfo(3) with item CODESET and uses iconv(3) to
convert the file names to the output charset.

I would like to bring up for discussion an alternative remedy by these
xorrisofs options:

  -input-charset UTF-8 -output-charset UTF-8

-----------------------------------------------------------------------

I am quite sure that it will work, but have problems with creating the
situation where it shall be the remedy.
On a system with configured LANG=en_US.UTF-8 i fail to reproduce
the issue with LANG=C or LANG="" unless i add the explicit demand to
convert to UTF-8.

My disappointingly unproblematic xorriso test run is

  ( LANG=  xorrisofs -o test.iso 'ÄÖÜß' )

with various attempts to set the locale charset away from UTF-8:
LANG, LANGUAGE, LC_ALL set to empty text or to "C".
LANG= causes nl_langinfo(CODESET) to return "ANSI_X3.4-1968" but iconv(3)
throws no error when it shall convert "\303\204\303\226\303\234\303\237".

I get warning messages if i tell xorriso to aim for output charset UTF-8
while LANG is empty and the file name contains non-ASCII characters:

  (LANG= xorrisofs -o test.iso -output-charset UTF-8 'ÄÖÜß')

yields three times

  libisofs: WARNING : Charset conversion error. Cannot convert ÄÖÜß from 
ANSI_X3.4-1968 to UTF-8
  libisofs: NOTE :  > Caused by: Charset conversion error

Inspection of test.iso shows that the file name 'ÄÖÜß' was defaulted to
name '________'.

Adding the other charset option

  (LANG= xorrisofs -o test.iso \
            -input-charset UTF-8 -output-charset UTF-8 'ÄÖÜß')

yields a run without warning messages resulting in Rock Ridge file name
'ÄÖÜß'.

So i assume that my proposal is a valid alternative to setting LANG=UTF-8.
Whether it is preferrable over setting LANG to UTF-8 would have to be
discussed.
At least it would be nice for me to know why LANG= causes trouble in
the GRUB tests but not on my command line.
(I see nothing in the xorriso runs of tests/util/grub-fs-tester.in which
would be equivalent to the run which fails to convert 'ÄÖÜß' here.)


Have a nice day :)

Thomas