[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: line buffering in pipes (was: RFC: Safely using xargs -P$NUM childre
From: |
Egmont Koblinger |
Subject: |
Re: line buffering in pipes (was: RFC: Safely using xargs -P$NUM children's output? Need a new tool?) |
Date: |
Thu, 2 May 2019 21:22:23 +0200 |
Hi Assaf,
On Thu, May 2, 2019 at 9:14 PM Assaf Gordon <address@hidden> wrote:
> The easiest way to avoid that is to use "stdbuf" (from coreutils),
> forcing a flush after each line. Assuming the lines are short enough
> (and file's output should be short enough), it should work:
>
> find /usr -print0 | xargs -0r -P199 -n16 stdbuf -oL file | ...
I don't think this is robust enough. If many "stdbuf -oL file"
processes decide to produce a reasonably sized output pretty much at
the same time, it might still suddenly clog the pipe and result in a
short write in one of them. Or am I missing something?
cheers,
egmont
Re: line buffering in pipes, Assaf Gordon, 2019/05/02