Re: Subprocess API.

[Chris Vine]

On Mon, 25 Sep 2017 19:14:22 +0200
Mathieu Othacehe <address@hidden> wrote:
> Hi Chris,
> 
> > This works exactly as you would expect from its POSIX equivalents
> > and has the advantage that you can read from the pipe as the
> > sub-process is proceeding rather than just collect at the end.  
> 
> Thank you ! Following your suggestion, I ended-up with :
> 
> --8<---------------cut here---------------start------------->8---
> (let* ((err-pipe (pipe))
>        (out-pipe (pipe))
>        (read-out (car out-pipe))
>        (write-out (cdr out-pipe))
>        (read-err (car err-pipe))
>        (write-err (cdr err-pipe))
>        (pid (run-concurrently+
>              (apply tail-call-program "...")
>              (write-out 1)
>              (write-err 2)))
>        (ret (status:exit-val (cdr (waitpid pid)))))
>   (close-port write-out)
>   (close-port write-err)
>   (let ((output (read-string read-out))
>         (error (read-string read-err)))
>     (close-port read-out)
>     (close-port read-errs 
>     (case ret
>       ((0) output)
>       (else (raise ...)))))
> --8<---------------cut here---------------end--------------->8---
> 
> which seems to work. However, run-concurrently+ uses "primitive-fork"
> which is forbiden in a multi-thread context (sadly, mine).
> 
> Do you have any idea on how to overcome this ?

Any launching of a new process requires a fork and if (as appears to
be your intention) you want to replace the process image with a new
one, an exec.  As you appear to know, POSIX allows only
async-signal-safe functions to be called in a multi-threaded program
between the fork and the exec, although glibc does relax this somewhat.

Since anything you do in guile between the fork and the exec has the
potential to allocate memory, that appears to mean that, as you say, you
cannot call primitive-fork in a guile program at a time when it is
running more than one thread.  If so, I do not know how to circumvent
that: you could consider launching the new process in C code via the
guile FFI so you can ensure that no non-async-signal-safe code is
called at the wrong time; but presumably you would still have by some
means to prevent the garbage collector from being able to start a
memory reclaiming run in the new process after the fork and before the
exec, and again I do not know how you would do that.  You would also
need to block system asyncs before forking (and unblock after the fork
in the original process) but that is trivial to do.

As regards your code, if you do not need to distinguish between stdout
and stderr, you would do better to have only one pipe and use the write
port of the pipe for both of descriptors 1 and 2.  That means that you
could read the pipe while the new process is proceeding rather than
after it has finished (which risks filling up the pipe): just loop
reading the read end of the pipe until an eof-object is received, and
then call waitpid after that.

Chris