bug-gnu-libiconv
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [bug-gnu-libiconv] iconv tool buffers entire input file


From: Jeffrey Walton
Subject: Re: [bug-gnu-libiconv] iconv tool buffers entire input file
Date: Thu, 6 Dec 2012 20:55:14 -0500

On Thu, Dec 6, 2012 at 8:13 PM, Bruno Haible <address@hidden> wrote:
> Hi,
>
> Yongzheng Wu wrote:
>> The iconv tool buffers the entire input file in memory. This causes a
>> problem with big files. This problem has been raised by others before.
>> http://serverfault.com/questions/50919/how-to-run-command-on-parts-of-input-file
>> and http://keithdevens.com/weblog/archive/2011/Sep/22/iconv. I found a
>> comment in iconv_prog.c, "Since we have to deal with arbitrary
>> encodings we must read the whole text in a buffer and process it in
>> one step."
>
> Your report is about the 'iconv' program from GNU libc. Here we are on the
> mailing list for GNU libiconv, which is a different package. The 'iconv'
> program from GNU libiconv does not have the behaviour you report.
>
>> My question is, can this be fixed?
>
> Yes, this can be fixed, with fixed-size buffers and careful handling of
> the unprocessed bytes at the end of a block.
>
> I encourage you to report this to the GNU libc bug tracker, at
> http://sourceware.org/bugzilla/
Good catch Bruno.

I spent last night and this morning thinking about gaps in my negative
test cases, and why I did not see a problem.

Jeff



reply via email to

[Prev in Thread] Current Thread [Next in Thread]