Re: Question regarding incomplete UTF-8 arguments.

[Tetsuo Handa]

I tried

  echo -ne '\xe3' | LANG=C xargs valgrind idn -u
  echo -ne '\xe3' | LANG=en_US xargs valgrind idn -u
  echo -ne '\xe3' | LANG=en_US.UTF-8 xargs valgrind idn -u
  echo -ne '\xe3' | LANG=C xargs valgrind idn -a
  echo -ne '\xe3' | LANG=en_US xargs valgrind idn -a
  echo -ne '\xe3' | LANG=en_US.UTF-8 xargs valgrind idn -a
  echo -ne '\xe3' | LANG=C xargs valgrind idn2
  echo -ne '\xe3' | LANG=en_US xargs valgrind idn2
  echo -ne '\xe3' | LANG=en_US.UTF-8 xargs valgrind idn2

cases and got below results.

Regards.

address@hidden:~$ idn --version
idn (GNU Libidn) 1.25
Packaged by Debian (1.25-2)
Copyright (C) 2012 Simon Josefsson.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Simon Josefsson.
address@hidden:~$ idn2 --version
idn2 (idn2) 0.8
Packaged by Debian (0.8-2)
Copyright (C) 2011 Simon Josefsson.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Simon Josefsson.
address@hidden:~$ echo -ne '\xe3' | LANG=C xargs valgrind idn -u
==3277== Memcheck, a memory error detector
==3277== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==3277== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==3277== Command: idn -u ・
==3277==
idn: could not convert from ANSI_X3.4-1968 to UTF-8
==3277==
==3277== HEAP SUMMARY:
==3277==     in use at exit: 6 bytes in 2 blocks
==3277==   total heap usage: 8 allocs, 6 frees, 8,409 bytes allocated
==3277==
==3277== LEAK SUMMARY:
==3277==    definitely lost: 0 bytes in 0 blocks
==3277==    indirectly lost: 0 bytes in 0 blocks
==3277==      possibly lost: 0 bytes in 0 blocks
==3277==    still reachable: 6 bytes in 2 blocks
==3277==         suppressed: 0 bytes in 0 blocks
==3277== Rerun with --leak-check=full to see details of leaked memory
==3277==
==3277== For counts of detected and suppressed errors, rerun with: -v
==3277== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
address@hidden:~$ echo -ne '\xe3' | LANG=en_US xargs valgrind idn -u
==3282== Memcheck, a memory error detector
==3282== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==3282== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==3282== Command: idn -u ・
==3282==
idn: could not convert from ANSI_X3.4-1968 to UTF-8
==3282==
==3282== HEAP SUMMARY:
==3282==     in use at exit: 6 bytes in 2 blocks
==3282==   total heap usage: 26 allocs, 24 frees, 13,220 bytes allocated
==3282==
==3282== LEAK SUMMARY:
==3282==    definitely lost: 0 bytes in 0 blocks
==3282==    indirectly lost: 0 bytes in 0 blocks
==3282==      possibly lost: 0 bytes in 0 blocks
==3282==    still reachable: 6 bytes in 2 blocks
==3282==         suppressed: 0 bytes in 0 blocks
==3282== Rerun with --leak-check=full to see details of leaked memory
==3282==
==3282== For counts of detected and suppressed errors, rerun with: -v
==3282== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
address@hidden:~$ echo -ne '\xe3' | LANG=en_US.UTF-8 xargs valgrind idn -u
==3287== Memcheck, a memory error detector
==3287== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==3287== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==3287== Command: idn -u ・
==3287==
==3287== Invalid read of size 1
==3287==    at 0x403CA48: stringprep_utf8_to_ucs4 (in 
/usr/lib/i386-linux-gnu/libidn.so.11.6.8)
==3287==    by 0x40864D2: (below main) (libc-start.c:226)
==3287==  Address 0x4218e4b is 1 bytes after a block of size 2 alloc'd
==3287==    at 0x402BB7A: malloc (in 
/usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==3287==    by 0x40EAAF0: strdup (strdup.c:43)
==3287==    by 0x40864D2: (below main) (libc-start.c:226)
==3287==
==3287== Invalid read of size 1
==3287==    at 0x403CA02: stringprep_utf8_to_ucs4 (in 
/usr/lib/i386-linux-gnu/libidn.so.11.6.8)
==3287==    by 0x8049493: ??? (in /usr/bin/idn)
==3287==    by 0x40864D2: (below main) (libc-start.c:226)
==3287==  Address 0x4218e4a is 0 bytes after a block of size 2 alloc'd
==3287==    at 0x402BB7A: malloc (in 
/usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==3287==    by 0x40EAAF0: strdup (strdup.c:43)
==3287==    by 0x40864D2: (below main) (libc-start.c:226)
==3287==
==3287== Invalid read of size 1
==3287==    at 0x403CA02: stringprep_utf8_to_ucs4 (in 
/usr/lib/i386-linux-gnu/libidn.so.11.6.8)
==3287==    by 0x403E871: idna_to_unicode_8z4z (in 
/usr/lib/i386-linux-gnu/libidn.so.11.6.8)
==3287==    by 0x40864D2: (below main) (libc-start.c:226)
==3287==  Address 0x4218e4a is 0 bytes after a block of size 2 alloc'd
==3287==    at 0x402BB7A: malloc (in 
/usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==3287==    by 0x40EAAF0: strdup (strdup.c:43)
==3287==    by 0x40864D2: (below main) (libc-start.c:226)
==3287==
　
==3287==
==3287== HEAP SUMMARY:
==3287==     in use at exit: 6 bytes in 2 blocks
==3287==   total heap usage: 45 allocs, 43 frees, 2,253 bytes allocated
==3287==
==3287== LEAK SUMMARY:
==3287==    definitely lost: 4 bytes in 1 blocks
==3287==    indirectly lost: 2 bytes in 1 blocks
==3287==      possibly lost: 0 bytes in 0 blocks
==3287==    still reachable: 0 bytes in 0 blocks
==3287==         suppressed: 0 bytes in 0 blocks
==3287== Rerun with --leak-check=full to see details of leaked memory
==3287==
==3287== For counts of detected and suppressed errors, rerun with: -v
==3287== ERROR SUMMARY: 4 errors from 3 contexts (suppressed: 0 from 0)
address@hidden:~$ echo -ne '\xe3' | LANG=C xargs valgrind idn -a
==3292== Memcheck, a memory error detector
==3292== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==3292== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==3292== Command: idn -a ・
==3292==
idn: could not convert from ANSI_X3.4-1968 to UTF-8
==3292==
==3292== HEAP SUMMARY:
==3292==     in use at exit: 6 bytes in 2 blocks
==3292==   total heap usage: 8 allocs, 6 frees, 8,409 bytes allocated
==3292==
==3292== LEAK SUMMARY:
==3292==    definitely lost: 0 bytes in 0 blocks
==3292==    indirectly lost: 0 bytes in 0 blocks
==3292==      possibly lost: 0 bytes in 0 blocks
==3292==    still reachable: 6 bytes in 2 blocks
==3292==         suppressed: 0 bytes in 0 blocks
==3292== Rerun with --leak-check=full to see details of leaked memory
==3292==
==3292== For counts of detected and suppressed errors, rerun with: -v
==3292== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
address@hidden:~$ echo -ne '\xe3' | LANG=en_US xargs valgrind idn -a
==3297== Memcheck, a memory error detector
==3297== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==3297== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==3297== Command: idn -a ・
==3297==
idn: could not convert from ANSI_X3.4-1968 to UTF-8
==3297==
==3297== HEAP SUMMARY:
==3297==     in use at exit: 6 bytes in 2 blocks
==3297==   total heap usage: 26 allocs, 24 frees, 13,220 bytes allocated
==3297==
==3297== LEAK SUMMARY:
==3297==    definitely lost: 0 bytes in 0 blocks
==3297==    indirectly lost: 0 bytes in 0 blocks
==3297==      possibly lost: 0 bytes in 0 blocks
==3297==    still reachable: 6 bytes in 2 blocks
==3297==         suppressed: 0 bytes in 0 blocks
==3297== Rerun with --leak-check=full to see details of leaked memory
==3297==
==3297== For counts of detected and suppressed errors, rerun with: -v
==3297== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
address@hidden:~$ echo -ne '\xe3' | LANG=en_US.UTF-8 xargs valgrind idn -a
==3302== Memcheck, a memory error detector
==3302== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==3302== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==3302== Command: idn -a ・
==3302==
==3302== Invalid read of size 1
==3302==    at 0x403CA48: stringprep_utf8_to_ucs4 (in 
/usr/lib/i386-linux-gnu/libidn.so.11.6.8)
==3302==    by 0x40864D2: (below main) (libc-start.c:226)
==3302==  Address 0x4218e4b is 1 bytes after a block of size 2 alloc'd
==3302==    at 0x402BB7A: malloc (in 
/usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==3302==    by 0x40EAAF0: strdup (strdup.c:43)
==3302==    by 0x40864D2: (below main) (libc-start.c:226)
==3302==
==3302== Invalid read of size 1
==3302==    at 0x403CA02: stringprep_utf8_to_ucs4 (in 
/usr/lib/i386-linux-gnu/libidn.so.11.6.8)
==3302==    by 0x804965B: ??? (in /usr/bin/idn)
==3302==    by 0x40864D2: (below main) (libc-start.c:226)
==3302==  Address 0x4218e4a is 0 bytes after a block of size 2 alloc'd
==3302==    at 0x402BB7A: malloc (in 
/usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==3302==    by 0x40EAAF0: strdup (strdup.c:43)
==3302==    by 0x40864D2: (below main) (libc-start.c:226)
==3302==

==3302==
==3302== HEAP SUMMARY:
==3302==     in use at exit: 6 bytes in 2 blocks
==3302==   total heap usage: 46 allocs, 44 frees, 2,269 bytes allocated
==3302==
==3302== LEAK SUMMARY:
==3302==    definitely lost: 4 bytes in 1 blocks
==3302==    indirectly lost: 2 bytes in 1 blocks
==3302==      possibly lost: 0 bytes in 0 blocks
==3302==    still reachable: 0 bytes in 0 blocks
==3302==         suppressed: 0 bytes in 0 blocks
==3302== Rerun with --leak-check=full to see details of leaked memory
==3302==
==3302== For counts of detected and suppressed errors, rerun with: -v
==3302== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
address@hidden:~$ echo -ne '\xe3' | LANG=C xargs valgrind idn2
==3307== Memcheck, a memory error detector
==3307== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==3307== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==3307== Command: idn2 ・
==3307==
==3307== Conditional jump or move depends on uninitialised value(s)
==3307==    at 0x403D88F: ??? (in /usr/lib/libidn2.so.0.0.8)
==3307==    by 0x403FD4D: ??? (in /usr/lib/libidn2.so.0.0.8)
==3307==    by 0x403FE12: ??? (in /usr/lib/libidn2.so.0.0.8)
==3307==    by 0x408A4D2: (below main) (libc-start.c:226)
==3307==
?
==3307==
==3307== HEAP SUMMARY:
==3307==     in use at exit: 262 bytes in 3 blocks
==3307==   total heap usage: 12 allocs, 9 frees, 17,027 bytes allocated
==3307==
==3307== LEAK SUMMARY:
==3307==    definitely lost: 260 bytes in 2 blocks
==3307==    indirectly lost: 2 bytes in 1 blocks
==3307==      possibly lost: 0 bytes in 0 blocks
==3307==    still reachable: 0 bytes in 0 blocks
==3307==         suppressed: 0 bytes in 0 blocks
==3307== Rerun with --leak-check=full to see details of leaked memory
==3307==
==3307== For counts of detected and suppressed errors, rerun with: -v
==3307== Use --track-origins=yes to see where uninitialised values come from
==3307== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
address@hidden:~$ echo -ne '\xe3' | LANG=en_US xargs valgrind idn2
==3312== Memcheck, a memory error detector
==3312== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==3312== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==3312== Command: idn2 ・
==3312==
==3312== Conditional jump or move depends on uninitialised value(s)
==3312==    at 0x403D88F: ??? (in /usr/lib/libidn2.so.0.0.8)
==3312==    by 0x403FD4D: ??? (in /usr/lib/libidn2.so.0.0.8)
==3312==    by 0x403FE12: ??? (in /usr/lib/libidn2.so.0.0.8)
==3312==    by 0x408A4D2: (below main) (libc-start.c:226)
==3312==
?
==3312==
==3312== HEAP SUMMARY:
==3312==     in use at exit: 262 bytes in 3 blocks
==3312==   total heap usage: 30 allocs, 27 frees, 21,838 bytes allocated
==3312==
==3312== LEAK SUMMARY:
==3312==    definitely lost: 260 bytes in 2 blocks
==3312==    indirectly lost: 2 bytes in 1 blocks
==3312==      possibly lost: 0 bytes in 0 blocks
==3312==    still reachable: 0 bytes in 0 blocks
==3312==         suppressed: 0 bytes in 0 blocks
==3312== Rerun with --leak-check=full to see details of leaked memory
==3312==
==3312== For counts of detected and suppressed errors, rerun with: -v
==3312== Use --track-origins=yes to see where uninitialised values come from
==3312== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
address@hidden:~$ echo -ne '\xe3' | LANG=en_US.UTF-8 xargs valgrind idn2
==3317== Memcheck, a memory error detector
==3317== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==3317== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==3317== Command: idn2 ・
==3317==
idn2: lookup: could not convert string to UTF-8
==3317==
==3317== HEAP SUMMARY:
==3317==     in use at exit: 6 bytes in 2 blocks
==3317==   total heap usage: 31 allocs, 29 frees, 1,982 bytes allocated
==3317==
==3317== LEAK SUMMARY:
==3317==    definitely lost: 0 bytes in 0 blocks
==3317==    indirectly lost: 0 bytes in 0 blocks
==3317==      possibly lost: 0 bytes in 0 blocks
==3317==    still reachable: 6 bytes in 2 blocks
==3317==         suppressed: 0 bytes in 0 blocks
==3317== Rerun with --leak-check=full to see details of leaked memory
==3317==
==3317== For counts of detected and suppressed errors, rerun with: -v
==3317== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)