Re: 5.0 regression: Script stuck when waiting in trap

[Chet Ramey]

On 6/9/19 8:58 AM, mwnx wrote:
> On Thu, Jun 06, 2019 at 03:18:16PM -0400, Chet Ramey wrote:
>>>    If ID is not given, waits for all currently active child
>>>    *processes*, and the return status is zero.
>>
>> OK, so the wording is the issue? What would work better? We could use
>> the "known to the shell" wording POSIX does, or "background jobs" or
>> even "running jobs" that some other shells use.
> 
> "running jobs" seems fine to me, but with bash 5 I guess it would
> have to be extended to "running jobs and the last started process
> substitution if one exists and it is still running".

Well, that's more wordy than I like, but it's on the right track.


>>> 2. `wait` takes into account only the last process substitution but
>>>    takes into account all processes of other kinds, not just the
>>>    last of their kind (and this is not documented in the help or the
>>>    man page).
>>
>> So you'd now like wait without arguments to wait on all process
>> substitutions?
> 
> Well, I guess I kind of liked the old behaviour (i.e. not waiting
> for process substitutions at all). It just seemed less confusing.

Before this, there was no way to wait for a process substitution at all,
and if you're going to allow wait to wait for a single one, then you
might was well allow wait without arguments to wait for it as well.


>> It seems like a reasonable compromise between waiting for none
>> (pre-bash-5.0 behavior) and waiting for all. If you are interested
>> in waiting for all the process substitutions, you can collect the
>> PIDs by getting $! after each one starts, and wait for them
>> explicitly.
> 
> ...but I don't really see the use case. If the PID can be retrieved
> from `$!`, why not just `wait $!`? Because one might want to wait
> for all background jobs _and_ the last process substitution? 

Because `wait' without arguments waits for all processes "known to the
shell" (POSIX wording), and the last process substitution is certainly
known.

Sure,
> this might help in this case:
> 
>     ... >(...) & ...; wait
> 
> But it won't in the following case, with a second process
> substitution:
> 
>     ... >(...) >(...) & ...; wait
> 
> I guess what I'm trying to get at is that this new feature of `wait`
> is lacking in generality, and that makes it confusing _and_ not all
> that useful in, well, the general case.

There are three cases.

1. A process substitution that is associated with a file descriptor that
   persists between commands (exec 4< <(some command)).

2. A process substitution that is associated with a file descriptor that
   exists for the duration of a shell command. This is most often a shell
   function or builtin, but a group command is a special case.

3. A process substitution that is associated with a command that runs in
   a subshell (e.g., ls > >(some other command)).

The shell will wait for all process substitutions in case 1, some, but not
all, in case 2, and none in case 3 (since those are not children of the
shell).

The original report was about a deadlock caused by case 2.

> 
> And, by the way, I get an error message with the following command
> in bash 5.0.7:
> 
>      $ echo >(sleep 5) >(sleep 1); pstree -p $BASHPID; wait
>      /dev/fd/63 /dev/fd/62
>      bash(10730)-+-bash(30435)---sleep(30437)
>                  |-bash(30436)---sleep(30439)
>                  `-pstree(30438)
>      bash: wait_for: No record of process 30435
> 
> I also found out, while tinkering around, that it is not possible to
> wait for any process substitution but the last one, even though this
> should technically be possible in the underlying C implementation
> since in my test examples the process substitution in question is
> indeed a (direct) child process of the shell:

It's a limitation of the current implementation. The shell closes file
descriptors to process substitutions as soon as it no longer needs them,
since not doing so will generally result in that process never exiting.
It doesn't remember the process substitution after closing the last file
descriptor to it, though it could, and should for the general case. It
will wait for running process substitutions to which it has open file
descriptors.

The shell assumes that the majority of commands used in process
substitutions do not persist very long after having their standard input
pipe closed, and will exit very quickly after consuming the available input
and reading EOF.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/