Re: GNU Parallel Bug Reports Bug in parallel using --pipe

[Ole Tange]

On Wed, Sep 7, 2011 at 2:09 AM, Bill Wyatt <address@hidden> wrote:
>
> I have a bug that I found in 20110722, and is also in the latest release.

I believe the bug can be summed up as:

  echo | parallel -j2 -N1 --pipe md5sum -c

should only give one error (from md5sum) instead of 2.

  echo | parallel -r -j2 -N1 --pipe md5sum -c

should give no errors.

The bug is confirmed.

There are several parts of this bug:

= Checking for empty lines =

-r implies we should never output empty lines. I do not see an
efficient way to do that (--pipe is too slow as it is), but maybe it
is acceptable with an even slower --pipe when using -r.

This part should be doable, and can probably be done with a
strategicly placed s/^\s*\n//gm;

= Not spawning unneeded processes =

Currently GNU Parallel spawns a new process when a process dies.

It will require some redesign to get it to not spawn more processes if
there is no more data to process. So do not expect a quick fix for
this.


/Ole