Re: [bug-gnu-libiconv] iconv tool buffers entire input file

[Jeffrey Walton]

On Fri, Dec 7, 2012 at 9:24 AM, Yongzheng Wu <address@hidden> wrote:
> On Fri, Dec 7, 2012 at 9:13 AM, Bruno Haible <address@hidden> wrote:
>> Hi,
>>
>> Yongzheng Wu wrote:
>>> The iconv tool buffers the entire input file in memory. This causes a
>>> problem with big files. This problem has been raised by others before.
>>> http://serverfault.com/questions/50919/how-to-run-command-on-parts-of-input-file
>>> and http://keithdevens.com/weblog/archive/2011/Sep/22/iconv. I found a
>>> comment in iconv_prog.c, "Since we have to deal with arbitrary
>>> encodings we must read the whole text in a buffer and process it in
>>> one step."
>>
>> Your report is about the 'iconv' program from GNU libc. Here we are on the
>> mailing list for GNU libiconv, which is a different package. The 'iconv'
>> program from GNU libiconv does not have the behaviour you report.
>>
>>> My question is, can this be fixed?
>>
>> Yes, this can be fixed, with fixed-size buffers and careful handling of
>> the unprocessed bytes at the end of a block.
>>
>> I encourage you to report this to the GNU libc bug tracker, at
>> http://sourceware.org/bugzilla/
>>
>> Bruno
>>
>
> Thank you for pointing it out. I just found out that the same bug was
> filed 4 years ago there, and status is still "new".
> http://sourceware.org/bugzilla/show_bug.cgi?id=6050
Ouch! It looks like its time to prepare/submit a patch.

Hopefully, it will be acknowledged and incorporated before 2016.

Jeff