Re: GNU Parallel Bug Reports env_parallel is unable to accept input via process substitution

[Ole Tange]

On Mon, Jul 3, 2017 at 10:15 PM, Coby Viner <address@hidden> wrote:

> env_parallel is unable to accept input via process substitution.

Thanks for the bug report.

> address@hidden:~] env_parallel --colsep '\t' "echo f1={1} f2={2} g1={3}
> g2={4}" :::: <(perl -e 'printf "f1\tf2\nA\tB\nC\tD\n"') :::: <(perl -e
> 'printf "g1\tg2\nE\tF\nG\tH\n"')
> parallel: Error: Cannot open input file `/dev/fd/63': No such file or
> directory.

This works for me. So there is something in your environment that is
different from mine.

> address@hidden:~] uname -a
>
> Linux mordor 2.6.32-642.6.2.el6.x86_64 #1 SMP Wed Oct 26 06:52:09 UTC 2016
> x86_64 GNU/Linux

2.6.32, eh? This leads me to believe we are dealing with a Bash
problem, because I feel we have different versions of Bash:

$ bash --version
GNU bash, version 4.4.7(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

I have now tested on Bash version 3.2.25 and 4.2.39, and that gives
the same problem as you see.

For version 4.2.39 the problem lies in the command 'which'. If you
remove these lines, it works:

#    if which parallel | grep 'no parallel in' >/dev/null; then
#       echo 'env_parallel: Error: parallel must be in $PATH.' >&2
#       return 255
#    fi
#    if which parallel >/dev/null; then
#       true which on linux
#    else
#       echo 'env_parallel: Error: parallel must be in $PATH.' >&2
#       return 255
#    fi

Now why on EARTH 'which' would cause this problem, I have no idea.

For version 3.2.25 you also need to comment these:

#    if perl -e 'exit grep { /^--record-env$/ } @ARGV' -- "$@"; then
#       true skip
#    else
#       (_names_of_ALIASES;
#        _names_of_FUNCTIONS;
#        _names_of_VARIABLES) |
#           cat > $HOME/.parallel/ignored_vars
#       return 0
#    fi

and:

#    if `which true` >/dev/null ; then
       `which parallel` "$@";
       _parallel_exit_CODE=$?
       unset PARALLEL_ENV;
       return $_parallel_exit_CODE
#    else
#       unset PARALLEL_ENV;
#       echo "env_parallel: Error: Your environment is too big." >&2
#       echo "env_parallel: Error: Try running this in a clean
environment once:" \
>&2
#       echo "env_parallel: Error:   env_parallel --record-env" >&2
#       echo "env_parallel: Error: And the use '--env _'" >&2
#       echo "env_parallel: Error: For details see: man env_parallel"
>&2
#
#       return 255
#    fi

'which' clearly seems to be the major culprit here.

> env_parallel does not appear able to accept input via process substitution
> from the parent shell. Perhaps this is prohibitively difficult to
> accomplish?

There are limitations to env_parallel. But this ought not to be one of them.

It ought to be possible to code a 'which' replacement, so maybe it can
be brought to work on these old systems, too.

Thanks for giving env_parallel a spin.


/Ole