Re: Degraded performance in cat + patch

[Pádraig Brady]

Pádraig Brady wrote:
> Pádraig Brady wrote:
>> Jim Meyering wrote:
>>> >From 6dd9c564a0cba6eec95102f091c6692a5ab48876 Mon Sep 17 00:00:00 2001
>>> From: Jim Meyering <address@hidden>
>>> Date: Fri, 6 Mar 2009 10:27:43 +0100
>>> Subject: [PATCH] cat: use larger buffer sizes to reduce read/write-syscall 
>>> overhead
>>>
>>> * src/cat.c (max): Remove definition.  Use MAX from system.h instead.
>>> (compute_buffer_size): New function.
>>> (main): Use it, to compute larger input and output buffer sizes
>>> derived from st_blksize, now typically 32KiB rather than 4KiB.
>>> Suggestion from Tzvi Rotshtein.
>> That sounds like previously cat did not derive from st_blksize
>> and that st_blksize is typically 32KiB :) Suggested log message:
>>
>> * src/cat.c (max): Remove definition.  Use MAX from system.h instead.
>> (compute_buffer_size): New function to compute the input and output
>> buffer sizes, which are now set at 8 times st_blksize with
>> a minimum of 32KiB. Previously the typical block sizes used were
>> 1KiB for pipes and 4KiB for files.
>> (main): Use it.
>> This was seem to increase throughput by up to 50%.
>> Suggestion from Tzvi Rotshtein.
> 
> Oops :) accurate one below I think:
> 
> * src/cat.c (max): Remove definition.  Use MAX from system.h instead.
> (compute_buffer_size): New function to compute the input and output
> buffer sizes, which are now set at 8 times st_blksize with a maximum
> of 32KiB. Previously the typical block sizes used were 1KiB for pipes
> and 4KiB for files, and now will be 8KiB and 32KiB respectively.
> (main): Use it.
> This was seem to increase throughput by up to 50%.
> Suggestion from Tzvi Rotshtein.

Actually reading your preformance results more closely showed
the throughput actually doubled? That surprises me.
Why such a huge syscall overhead? Testing with dd on a
1.7GHz pentium-m with 2.6.24.5-85.fc8 shows much less:

$ truncate -s2G test.cat

$ dd bs=4x1024 if=test.cat of=/dev/null
2147483648 bytes (2.1 GB) copied, 6.57765 s, 326 MB/s
$ dd bs=32x1024 if=test.cat of=/dev/null
2147483648 bytes (2.1 GB) copied, 5.74548 s, 374 MB/s

So trying with cat...

$ /usr/bin/time ./cat test.cat >/dev/null
0.29user 5.28system 0:06.55elapsed 85%CPU (0avgtext+0avgdata 0maxresident)k
472inputs+0outputs (2major+149minor)pagefaults 0swaps

Notice that we're waiting for something even though there is no data on disk!
That seems like a separate kernel issue, anyway reducing the file size
to fit in the page cache on this 2GB RAM machine and repeating...

$ truncate -s1G test.cat

$ /usr/bin/time ./cat-4K test.cat >/dev/null
0.13user 1.30system 0:01.44elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+150minor)pagefaults 0swaps

$ /usr/bin/time ./cat-32K test.cat >/dev/null
0.01user 1.17system 0:01.19elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+157minor)pagefaults 0swaps

Again that's a modest improvement, though worth changing the buffer size for.
Is there some massive syscall overhead in rawhide kernels or something at the 
moment?

cheers,
Pádraig.