sort with a large buffer size (-S)

[Ryan Morin]

I routinely use sort on very large files and generally use a buffer size of 
1500M to accomplish this.  I have some large memory 64-bit machines but am not 
able to utilize this memory to improve sorting efficiency.  In other words, I 
would like to be able to set -S to allow sort to consume much more memory to 
save the necessity of writing large temp files to disk (unless necessary).  I 
presume there is a way to compile sort to allow it to utilize more than 4G of 
memory on an x86_64 machine but have not been able to determine how.  Here is 
what I am currently encountering:

ls -l temp
-rw-rw-r--  1 rmorin  users  807714374 May 29 16:15 temp

sort -S 4000M temp > temp.sort
sort(45777) malloc: *** mmap(size=4194308096) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug

I work on x86_64 Macs and also x86_64 machines running RHEL.

I would appreciate any advice on the matter.

Kindest regards,

Ryan Morin