Re: Problem with sort ... textutils-2.0.21 on TRU64 v5.1a

[Bob Proulx]

Parkin Frank - fparki <address@hidden> [2002-08-23 16:27:22 +0100]:

> When sorting large files I (root) receive the following message ...

When you say "large files" do you mean files greater than 2GB?
Today's jargon calls files which exceed 2^31 bits large files and they
take special handling in 32 bit programs.  Of course 64 bit programs
should handle those fine up to 2^63 bits.

And does that also imply that you don't get that error message when
sorting small files?  It works up to a larger file size?

> /usr/local/bin/sort: cannot create temporary file: /sorttest/sortapuED: File 
> exists

Hmm...  Normally sort will call mkstemp() with a default temporary
directory set it /tmp/sortXXXXXX.  I assume you are overriding the
temporary directory and placing it in /sorttest/?

> I have tried various command combinations with different temporary areas
> (all 777 perm). 
> 
> Compaq sort and syncsort are working correctly on the same system. 
> 
> Can you help?

There are several possible problems.  We need to determine several
things to move forward.

1. The exact version of sort you are using.  'sort --version'

2. Is GNU sort on True64 compiled as a 32-bit program or a 64-bit
   program?

3. The size of the file you are trying to sort.  'ls -l'

4. The size and number of the temporary files being generated.  Can
   you run 'ls -l' on the temporary directory while this is running
   and catch the size of the files there in process?  Also, are there
   a lot of them or just one or just none?  This is tricky to do since
   when sort exits it will clean everything up and remove those
   files.  You have to catch it in motion.

5. The amount of physical ram on your system in round numbers.

The temporary file size is related to the amount of memory that the
sort program thinks is available.  On some systems it can't tell and
has to guess.  If it has to guess then it guesses very conservatively
in terms of memory but that means that it uses more disk space and
therefore has a longer run time.  The guess size IIRC is 8MB and when
sorting a big files can create a lot of temporaries.  Instead of not
being able to create a temporary file you might be actually running
out of temporary files.  Just a guess and more information will be
needed to tell for sure.

Bob