bug-coreutils
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

cp: performance improvement with small files


From: Hauke Laging
Subject: cp: performance improvement with small files
Date: Fri, 23 May 2008 15:23:54 +0200
User-agent: KMail/1.9.6 (enterprise 20070904.708012)

Hello,

I just read an interesting hint in the German shell Usenet group 
(<address@hidden>). As I could not find anything about 
that point in your mailing list archive I would like to mention it here.

The author claims that he achieved a huge performance increase (more than 
factor 10) when copying a big amount of small files (1-10 KiB) by sorting 
by inode numbers first. This probably reduces the disk access time which 
becomes the dominating factor for small files.

Of course, this kind of sorting could be (transparently) done in cp, too. 
When reading the directory contents you might count the number (and share) 
of small files and determine whether such sorting makes sense for the 
respective data. And certainly this decision is based on the assumption 
that the respective file system places the inodes on disk according to 
their number. I don't know if that is correct for all file systems. If 
not, cp might check that first.


The same goes for mv, of course, when moving between volumes (and maybe 
other programs that access inodes of many files in certain situations).


Best regards,

Hauke




reply via email to

[Prev in Thread] Current Thread [Next in Thread]