Re: awk leaks memory on RHEL5_64

[Jeetendra Mirchandani]

Hi Aharon,

Thanks for point out the mistake. I have been using gawk for 4-5 years now,
and never realized that the array would initialize an element even if I
referenced a non-existent member.

Sorry about the incorrect bug report.

But I have to say, this is very non-intuitive behavior.

$ awk 'BEGIN{a[1]; a[2]; for(i in a){print i}}'
1
2

Regards,
Jeetu


On Fri, Jan 8, 2010 at 3:33 AM, Aharon Robbins <address@hidden> wrote:

> Greetings. Re this:
>
> > Date: Thu, 7 Jan 2010 19:11:04 -0800
> > Subject: awk leaks memory on RHEL5_64
> > From: Jeetendra Mirchandani <address@hidden>
> > To: address@hidden
> >
> > Hi,
> >
> > I have discovered that awk leaks memory on a rhel5_64 box
> >
> > awk '{if(FILENAME=="file1")interested[$1]=1; else
> if(interested[$1])print}'
> > file1 file2
> >
> >
> > Here file1 is about 40MB in size, with about 500K records. file2 has
> about
> > 220MM records and is about 9GB in size.
> >
> > awk process grows to 13GB + in size
> >
> > $ cat /etc/redhat-release
> > Red Hat Enterprise Linux Server release 5 (Tikanga)
> >
> > $ uname -r
> > 2.6.18-53.1.14.el5
> >
> > $ awk --version
> > GNU Awk 3.1.5
> >
> > $ rpm -q gawk
> > gawk-3.1.5-14.el5
> >
> > $ rpm -q glibc
> > glibc-2.5-12
>
> Gawk is not leaking memory; your program is not doing what you think
> it's doing.  Try:
>
>   awk '{if(FILENAME=="file1")interested[$1]=1; else if($1 in
> interested)print}'
>
> Then read up about associative arrays in the gawk manual.
>
> Arnold
>