Re: reading all data from FIFO with 'dd' without blocking

[Assaf Gordon]

Hello Bernhard,

Thanks for the reply.

On 11/19/2014 08:05 AM, Bernhard Voelker wrote:
> On 11/19/2014 01:40 AM, Assaf Gordon wrote:
> > Hello,
> >
> > I'm looking for a way to read all available data from a FIFO without blocking, 
> > using coreutils (or other unix shell commands?).
<...>
> > A better solution, using 'timeout' and 'cat' is:
> > ===
<...>
> >     while read A ; do
> >        ## Consume any events that happen in the next 1 second
> >        timeout 1 cat > /dev/null
> >
> >        ## call external program
> >        my-program
> >     done < myfifo
> > ===
>
> I'm still not sure what exactly you want to avoid, but what about
> something around these lines:
>
>    while timeout 10 head -n 20 ; do
>      my-program
>    done < myfifo
>
> This ensures that 'my-program' is run every 10s while running more
> often when 20 lines have appeared in the fifo.

With my 'timeout' + 'cat' I already get a 1s timeout, I was looking for 
something quicker than 1s .

> But this still somehow contradicts to the idea of inotify ...
> can't you read the inotify output from within 'my-program'?

No - it's not really 'my' program...

What I'm trying to do is have a mechanism to run to GNU Savannah Wiki locally, 
and rebuild the static HTMLs every time a markdown file is updated.
If I was using Github's "Jekyll" system, it has a built-in "--watch" option.
But IkiWiki (on which Savannah's Wiki depends) doesn't.

The technical problem is that an update to a markdown file is actually multiple 
'inotify' events:
A temporary file is created, then saved, then the real file is updated, then 
saved (at least in when editing files in vim).
So one logical update translates to multiple file-system events (=multiple 
output lines in the fifo). they all happen in a quick burst.
I want to consume/discard all these lines, and run the wiki update once.

Again, with the 'timeout + cat' method above, I have a working solution with 1s 
delay between the time I update a markdown file and the time the website 
renders.
It's not bad, but I'm looking for something quicker.


As an added bonus, I'm also contemplating using a similar inotify approach for 
the new mechanism to update the web pages on 'www.gnu.org'.
Currently there's a cron job that runs every few minutes and checks for updates 
in a certain directory. Sounds like 'inotify' would be a good substitute. But 
that's just a thought - not sure how stable 'inofiywait' is.

Thanks,
 - Assaf