race condition in GNU "tail -f"

[Ken Raeburn]

It appears that there is a small window between when GNU "tail -f"
reads the last bytes of the file (adjusting the current offset of the
file descriptor to the then-current end of the file) and when it calls
fstat() to get the size of the file.

The fstat size is saved away as the current size of the file.  The
tail_forever loop calls fstat and checks the return size against the
saved current size of the file.

Any data added during that window will not be displayed until
*another* change is made to the file size, normally by appending more
data.  At that point, the data added during the window and the data
added later will be read and displayed, and the combined size added to
tail's notion of the current file size.

But the earlier fstat call reported a size that would've included the
data added during the window, so that data has now been counted twice.
So, the next time fstat is called to check for new data, if the file
hasn't been modified, or if the added data is smaller than the data
added during the first window, the file will appear to have shrunk;
tail will complain and adjust the computed current size back to the
real current size, without displaying any output, so some data might
never be seen.

If the file has been modified by adding more data than was added
during the initial window, I believe it'll all be displayed, the
computed current size updated and still out of sync, so the
out-of-sync state can probably be maintained until the file stops
growing fast enough.

Since tail is just reading and displaying a few lines, normally, and
the file is already open and actively being accessed, and especially
if the file is on local disk, one would think that this theoretical
race condition wouldn't ever cause a problem.  And even if it did, a
little missed data probably wouldn't ever be noticed.

As it happens, I'm working on a project that has an expect script
which fires up "tail -f" on a log file, then appends a cookie value to
the log file, and waits for the cookie to show up in the tail output
so we know when tail has reached the end of the file.  We've hit this
window quite a number of times; enough that it makes our testing
unreliable on our slower systems.

I've got some workarounds to try, but tail still needs fixing...  My
suggestion would be to use lseek to get the current file position as
the initial computed file size for the tail_forever loop, and ignore
the size returned by fstat.  Or, check if they're different and read
some more from the file until caught up.

Ken