bug#10877: Wimpy external files.

[Rogier Wolff]

On Fri, Feb 24, 2012 at 09:28:02AM -0800, Paul Eggert wrote:
> I think 'sort' by default uses 1/8 of physical memory.
> It sounds like that calculation isn't working on your host;
> if so, it'd be nice if you could dive into the code and see why.

OK. I'll look into it. The server where I observed the 20M files has
8G RAM, and is currently very busy with its primary task (serving
files).

The server runs 32bit userspace and an PAE kernel. 

I checked if I can reproduce the problem on my workstation that
would be available for testing. 

My workstation has 8G ram and is fully 64-bit. (i.e. kernel +
userspace).

On my workstation I get 34M files.... So problem reproduced. It is
using 30 times smaller files than what you predict it would.

Analysis so far: 
-S 2% results in about 160Mbytes of core memory used, and temporary
files of about 114Mbytes. 

My conclusion is that physmem_total() is working. 

After a bit of instrumentation I get: 


default sort size: avail=1946.183594M, total=8001.156250M, mem=1946.183594M
size is now: 1946M
after rlimit size is now: 1946M
after margin size is now: 973M
after rlimitrss size is now: 973M
returning: 973M

so also the function "default_sort_size" seems to be working. I have
to leave now. More later. 

        Roger.


-- 
** address@hidden ** http://www.BitWizard.nl/ ** +31-15-2600998 **
**    Delftechpark 26 2628 XH  Delft, The Netherlands. KVK: 27239233    **
*-- BitWizard writes Linux device drivers for any device you may have! --*
The plan was simple, like my brother-in-law Phil. But unlike
Phil, this plan just might work.