Re: cp: performance improvement with small files

[Pádraig Brady]

Hauke Laging wrote:
> Hello,
> 
> I just read an interesting hint in the German shell Usenet group 
> (<address@hidden>). As I could not find anything about 
> that point in your mailing list archive I would like to mention it here.
> 
> The author claims that he achieved a huge performance increase (more than 
> factor 10) when copying a big amount of small files (1-10 KiB) by sorting 
> by inode numbers first. This probably reduces the disk access time which 
> becomes the dominating factor for small files.

disk seeks have mostly missed the computer performance increases over time,
and hence why they're increasing being noticed as the bottleneck.
However I think mechanical disks will quickly become a thing of the past,
with the onset of solid state disks.

I've noticed myself a large performance gain in the few filesystems I've
tested by sorting by inode, so that the disk head seeks in 1 direction only.
One can see this in the findup component of fslint here for example:
http://code.google.com/p/fslint/source/browse/tags/2.26/fslint/findup#158
I've also found though that sorting by path is only a little worse.
Sorting by md5sum is very bad though :)

Pádraig.