[PATCH] stdio buffer flushing on redirection

[Tom Alsberg]

Hi there.

A curious issue I had with a shell script writes output to a pipe, 
turns out (after quite a while of investigation, and noticing that it 
works on systems where /bin/sh is not bash) to be due to the way 
bash writes output on redirection.

Some bash builtins (e.g. echo) write output using stdio functions 
(e.g. printf, putchar), however redirection is performed at the Unix 
file descriptor level (using dup).  The thing is that since stdio 
handles buffering on its own, output of builtins may go to the wrong 
place if redirection is done.  While bash correctly takes care to 
flush the buffers after output in e.g. the echo builtin, it does not 
handle the case of output failure.

In case fflush fails, under some implementations, the output may 
remain in the stdout buffer, and when printing something later, after 
redirecting stdout to some file, the previous output (which should not 
have gone to this file) will remain in the buffer, and be printed 
together with the new output into the target file.

I managed to reduce the bug to a simple example - take the following 
script:

#!/usr/bin/env bash
trap "" 1
while true; do
       echo "FOO"
       echo "BAR" >> /tmp/barlog
done

and run it within an xterm.  As long as the terminal is still open, 
all FOO output goes to the terminal, while all BAR output goes to 
/tmp/barlog.  Close the xterm, though, and (at least with glibc 2.7 as 
the system C library), the FOO output will fail to go to the terminal 
(which no longer exists), and will instead also go to /tmp/barlog.

I believe if the terminal was closed, one would expect output to it to 
be discarded, and not go to some other unrelated file, where it may 
interfere.  If somebody can check this and let me know if you can 
reproduce the problem, or confirm that this is indeed the problem...

It does not appear easy to fix the bug in a portable way, since in 
general mixing I/O in the Unix file descriptor level and in the stdio 
level is not so well defined.  However, 4.4BSD-based systems provide 
the fpurge function, and Solaris and glibc the (identical, except for 
the return value) __fpurge function, which clears all pending output 
in a stdio FILE * buffer.  Calling those functions when redirecting 
stdout/stderr solves the problem on systems where they are supported - 
output that previously failed to go to stdout will be discarded and 
not go to the redirect target.

Attached is a small patch I wrote for bash-3.2 that (with a small 
change to configure.in and config.h.in) checks for those functions on 
configure, and if they are available uses them whenever redirecting 
stdout/stderr.  That seems to do the trick here, but let me know if it
works for you, or if there is a more correct way to do this.

I would be glad to have that patch applied to upstream bash - that 
will save us the trouble of applying it locally every time.  Of course 
it would be even better to solve that problem in a more portable 
manner, but I suspect that may require more substantial modifications 
to parts of the code.

 Cheers,
 -- Tom

--
 Tom Alsberg - hacker (being the best description fitting this space)
 Web page:      http://www.cs.huji.ac.il/~alsbergt/
DISCLAIMER:  The above message does not even necessarily represent what
my fingers have typed on the keyboard, save anything further.

bash-3.2.redirflush.patch
Description: Text Data