Merging (-m/--merge) slower than sorting irrespective of --parallel option

[Vlad ROMASCANU]

Using sort (GNU coreutils) 8.21

Linux xxxx 2.6.18-274.12.1.el5 #1 SMP Tue Nov 29 13:37:46 EST 2011 x86_64 x86_64 x86_64 GNU/Linux

CentOS release 5.7 (Final)

16 cores / Pentium III Xeon (Cascades) / .18um / X5550 @ 2.67GHz / L1 data cache: 32K, 8-way, 64 byte lines / data TLB: 4K pages, 4-way, 64 entries / L2 unified cache: 256K, 8-way

Pre-sorting every file in the coreutils-8.21 source distro then using "sort --files0-from" on the sorted files unexpectedly outperforms "sort --merge --files0-from" by more than 100% (i.e. "sort --merge" is more than 2x SLOWER than non-merge.)  The gap narrows but still exists when disabling parallelism (--parallel=1.)  FWIW ~20% of the lines being sorted are identical (isolated newlines), and the same relative performance is observed with other datasets I tried (I used the coreutils sourcetree as it is readily available.)

SO: is this normal?

Here's what I did:

$ export LC_ALL=C

$ export LANG=C

$ cd /tmp/coreutils-8.21

$ mkdir /tmp/coreutils-8.21-sorted

$ find . -type d | sort | while read d ; do mkdir /tmp/coreutils-8.21-sorted/$d ; done

$ find . -type f | while read f ; do sort $f > /tmp/coreutils-8.21-sorted/$f ; done

$ cd /tmp/coreutils-8.21-sorted

$ find . -type f -print0 >lst

$ free

             total       used       free     shared    buffers     cached

Mem:      24674780    5897556   18777224          0     780368    4337448

-/+ buffers/cache:     779740   23895040

Swap:     26738680          0   26738680

Now,

$ time sort -m --files0-from lst >/dev/null

real    0m2.410s

user    0m1.466s

sys     0m0.943s

$ time sort --files0-from lst >/dev/null

real    0m0.813s

user    0m1.891s

sys     0m0.274s

Disabling parallelism,

$ time sort --parallel=1 -m --files0-from lst >/dev/null

real    0m2.545s

user    0m1.561s

sys     0m0.984s

$ time sort --parallel=1 --files0-from lst >/dev/null

real    0m1.748s

user    0m1.600s

sys     0m0.148s

And using -s:

$ time sort --parallel=1 -m -s --files0-from lst >/dev/null

real    0m2.502s

user    0m1.577s

sys     0m0.925s

$ time sort --parallel=1 -s --files0-from lst >/dev/null

real    0m1.826s

user    0m1.620s

sys     0m0.206s

Anyone have any idea why these non-intuitive results?  Why does "sort -m" appear so slow even with --parallel=1?

Should this go to address@hidden?

Thank you!

Vlad.