Re: make: *** virtual memory exhausted. Stop.

[Ted Stern]

On 23 Jun 2003, Paul D. Smith wrote:
> %% Ted Stern <address@hidden> writes:
>
>   ts> My version of the "make: *** virtual memory exhausted.  Stop." bug
>   ts> occurs from trying to include 1000's of files.  It doesn't matter
>   ts> how large the files are --- what is important is the length of the
>   ts> files' pathnames.  If I allow 128 mega words (1GB) for the qsub
>   ts> job, the error occurs with about 5000 80 character filenames.  If
>   ts> I allow 16MW or 128MB for the job, the error occurs with about
>   ts> 1500 included files of about 80-character pathnames each.
>
>   ts> In earlier correspondence with Paul Smith, he indicated that he
>   ts> thought there may be a memory leak in read.c.
>
>   ts> I have verified that the bug has not been fixed in current CVS.
>
>   ts> Sure would be nice if somebody with access to purify could run it
>   ts> on GNU make ...
>
> I have run the GNU make test suite with Purify on Solaris, and valgrind
> on Linux, and neither have reported any memory leaks with the latest CVS
> code.
>
> If you have a test case that can show this problem pls. let me know.
> Thanks.

I am attaching three files that should give an idea of how to re-create the
problem.

"touchempty.ksh" creates a hierarchy of 1000 directories, each containing 10
*.mk files.

The Makefile tries to include all the 10K makefiles.

The qsub script runs the make job in a batch environment with total memory for
all processes limited by the QSUB "-lM" option.  To duplicate this outside of
a batch environment you may have to play with your shell's "limit" command.

The test directory in which I set this up was named "/ptmp/stern/testmake/",
21 leading characters, so to really duplicate this you should give your top
directory an absolute pathname of 21 characters or more.

Ted
-- 
 Ted Stern                                 Applications Group
 Cray Inc.                               office: 206-701-2182
 411 First Avenue South, Suite 600         cell: 206-383-1049
 Seattle, WA 98104-2860                     FAX: 206-701-2500

 Frango ut patefaciam -- I break that I may reveal
 (The Paleontological Society motto, equally apropos for debugging)

touchempty.ksh
Description: touchempty.ksh

Makefile
Description: Makefile

testmake.qsub
Description: testmake.qsub