[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
cp: performance improvement with small files
From: |
Hauke Laging |
Subject: |
cp: performance improvement with small files |
Date: |
Fri, 23 May 2008 15:23:54 +0200 |
User-agent: |
KMail/1.9.6 (enterprise 20070904.708012) |
Hello,
I just read an interesting hint in the German shell Usenet group
(<address@hidden>). As I could not find anything about
that point in your mailing list archive I would like to mention it here.
The author claims that he achieved a huge performance increase (more than
factor 10) when copying a big amount of small files (1-10 KiB) by sorting
by inode numbers first. This probably reduces the disk access time which
becomes the dominating factor for small files.
Of course, this kind of sorting could be (transparently) done in cp, too.
When reading the directory contents you might count the number (and share)
of small files and determine whether such sorting makes sense for the
respective data. And certainly this decision is based on the assumption
that the respective file system places the inodes on disk according to
their number. I don't know if that is correct for all file systems. If
not, cp might check that first.
The same goes for mv, of course, when moving between volumes (and maybe
other programs that access inodes of many files in certain situations).
Best regards,
Hauke
- cp: performance improvement with small files,
Hauke Laging <=