Infinite loop with error trap and subshell

[Samir Aguiar]

Hi,

We have a script that prints the call stack inside an ERR trap. The code 
invokes the `caller` built-in in a while loop that is piped to another 
while loop, which uses `read` to parse the output of `caller` and 
pretty-print it. However, after updating to Bash 5.1 the script now 
leads to an infinite number of processes being spawn and the error 
handler being called over and over. Prior to that version this did not 
happen.

My colleague Philipp Gesang and I have come up with a sample script to 
reproduce the problem. The ERR trap will create a flag file when it gets 
called again to indicate that infinite forking *would* occur, in which 
case it will exit with 99. An exit code of 0 means that nothing wrong 
would happen. The invocation of `/bin/false` (or similar) inside the 
while loop is mandatory for the problem to be triggered.

---------------------------
    #/bin/bash
    shopt -s extdebug

    bisect_abort=0
    error_flag_file="triggered_error"
    rm -f "$error_flag_file"

    handle_error()
    {
        [ $bisect_abort -gt 1 ] && touch "$error_flag_file" && return
        bisect_abort=$((bisect_abort+1))

        while :; do
            # need to produce an nonzero return code
            /bin/false
            # only a single iteration is needed
            break
        done | cat
    }

    trap handle_error ERR

    /bin/false

    # error code path reached on bash-5.1.x
    [ -f "$error_flag_file" ] && rm -f "$error_flag_file" && exit 99

    # normal exit path reached on bash-5.0.x
    exit 0
---------------------------

On bash 5.1.8 I get an exit code of 99, but on 5.0 I get 0.

This behavior was introduced in commit 8f576adedb5 (dev branch), more 
specifically in `execute_cmd.c`:

---------------------------
  -#if 0        /* TAG:bash-5.1 */
  +
     /* We are in a subshell, so forget that we are running a trap 
handler or
        that the signal handler has changed (we haven't changed it!) */
     if (running_trap > 0)
       {
         run_trap_cleanup (running_trap - 1);
  -      running_trap = 0;
  +      running_trap = 0;              /* XXX - maybe leave this */
       }
---------------------------

We have managed to work around it by changing the first while loop so 
that it will not produce an error in the last iteration, but I was 
wondering - is this expected behavior? Should we avoid subshells inside 
traps?

Thanks in advance.

Kind regards,
Samir Aguiar