Re: csplit: memory exhausted

[lukekendall]

On  2 Feb, Jim Meyering wrote:
>  address@hidden wrote:
> > I'm using Red Hat Linux 7.2 and just updated the textutils package from
> > version 2.0.14 to 2.0.21, but the error "csplit: memory exhausted" is
> > still occurring intermittently in a shell script.  Mostly it fails,
> > sometimes it works.
>  
>  [I've redirected to the bug-coreutils mailing list]
>  
>  Thanks for the report.
>  That has been fixed since coreutils-5.1.0.
>  
>    ftp://alpha.gnu.org/gnu/coreutils/coreutils-5.1.2.tar.gz
>    ftp://alpha.gnu.org/gnu/coreutils/coreutils-5.1.2.tar.bz2
>    (coreutils is the union of fileutils, textutils, and sh-utils)
>  
>  FYI, although 5.1.2 is a `test' release, no significant
>  problems have been reported against it.

Hi Jim

While the 5.1.2 test release fixed my problem, I've been having a
disturbing problem that until today I could ignore.  Specifically, if
csplit is fed a lot of input (800kb) from stdin, it would sometimes
corrupt a tiny amount of the output.

E.g. one line was output as:
address@hidden@ch patterns particularly archaic?'<P>
instead of:
her speech patterns particularly archaic?'<P>
(showing the odd characters as nvi displays them).

The workaround for me is to write the input to a temporary file:

        ### Major bad bug in csplit: piping the input straight through
        ### corrupts the output stream, as detected by chcnt below.
        ### By writing it to a temporary file we don't trigger the bug.
        #if sed 's/<BR>/<P>/' "$f" | \
        #    csplit --prefix=$TMP --elide-empty-files -n 3 \
        #       --silent - "/^$PATTERN/" "{*}"
        sed 's/<BR>/<P>/' "$f" > $TMP-expanded
        if csplit --prefix=$TMP --elide-empty-files -n 3 \
                --silent $TMP-expanded "/^$PATTERN/" "{*}"

$ csplit --version
csplit (coreutils) 5.1.2
Written by Stuart Kemp and David MacKenzie.

Copyright (C) 2004 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


Just thought I'd let you know.  (I'm running under Linux 2.4.23, Red Hat
7.2)

Regards,

luke