Austin Group questions on iconv()

[Eric Blake]

In today's Austin Group meeting, the folks discussing POSIX had a
question for Bruno and/or anyone else with an idea on how the
standards should approach a difference in behavior between Solaris and
GNU iconv() implementations.

For context, today's meeting minutes:
https://posix.rhansen.org/p/2023-03-09 around line 1635

and the bugs leading to the question:

https://austingroupbugs.net/view.php?id=1635
 "0001635: iconv: please be more explicit in input-not-convertible case"
 still open - iconv() resulting in EILSEQ not because of input
 encoding error but because of output being unable to encode the
 transliteration

https://austingroupbugs.net/view.php?id=1007
 "0001007: iconv function not allowed to fail to convert valid sequences"
 resolved at https://austingroupbugs.net/view.php?id=1007#c3330,
 standardizing the //IGNORE, //TRANSLIT, and //NON_IDENTICAL_DISCARD
 modifiers

It seems that bug 1635 is saying that the Solaris implementation
provides a conversion that application writers can use to get reliable
output but does not provide some desired features, and the standard
should change to acknowledge that the GNU implementation provides some
of those desired features.  However, the GNU implementation includes
some ambiguities that make it unreliable.  It seems to ask us to
change the standard to allow a modified version of the GNU iconv()
function that could be reliably interpreted by an appication writer.
For example, overloading EILSEQ to mean that there was an invalid
character in the input stream or that there was no transliteration
available in the output codeset to convert that input character makes
it impossible for an application to determine which of those two
problems caused iconv() to fail.

Can we get an explanation on how an application writer is supposed to
write code to reliably use the iconv() in GNU libc, given the above
example?  Can we get help in identifying exactly what changes need to
be made to POSIX (after bugid:1007 has been integrated) to allow GNU
behavior and get reliable results without breaking applications that
currently work with the Solaris iconv() interface.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org