[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RFC: enable potential checksum output buffering improvements
From: |
Pádraig Brady |
Subject: |
RFC: enable potential checksum output buffering improvements |
Date: |
Sun, 5 Aug 2012 19:31:09 +0100 |
Perhaps useful for some future release?
I noticed one of my scripts for processing checksums
of many small files was using lots of CPU.
In particular 'sed' was showing up in top.
I replaced 'sed' with 'tee dbg | sed' and then
tee started using the CPU.
This indicated that the previous process in the pipe
(md5sum) was causing excessive context switching.
Note the Linux kernel behavior of encouraging wakup
of the process reading from a pipe when the pipe is written to,
does seem sensible to me.
I confirmed reduced CPU usage by removing the
setvbuf(..,_IOLBF,..) call in md5sum.
Now we only use this to ensure atomic line output,
and the existing setvbuf() interface seems rich enough
to express just this requirement, by also specifying a
buffer size to use with _IOLBF. So I proposed such
an augmentation of the stdio implementation in glibc here:
http://sourceware.org/bugzilla/show_bug.cgi?id=14433
This patch would enable such logic, but could
be applied at any stage.
cheers,
Pádraig.
- RFC: enable potential checksum output buffering improvements,
Pádraig Brady <=