Re: parallel make and I/O of child processes

[Ralf Wildenhues]

* Stefano Lattarini wrote on Tue, Jan 25, 2011 at 11:02:11AM CET:
> So, what about the attached updated patch?

I have some nits, otherwise I'll let Eric approve this.

> Subject: [PATCH] docs: another parallel make issue
> 
> * doc/autoconf.texi (Parallel Make): Document that some make
> implementations, when run in parallel mode, connect stdout and
> stderr of child processes to pipes or temporary files, and might
> re-route stderr of spawned processes to stout.  Also document
> that FreeBSD make in parallel mode reuses the same shell for
> multiple commands within one recipe (like NetBSD make does).

> --- a/doc/autoconf.texi
> +++ b/doc/autoconf.texi
> @@ -19475,13 +19475,52 @@ sh
>  @cindex Parallel @command{make}
>  
>  Support for parallel execution in @command{make} implementation varies.
> -Generally, using GNU make is your best bet.  When NetBSD
> address@hidden is invoked with @address@hidden, it will reuse the
> -same shell for multiple commands within one recipe.  This can have
> -unexpected address@hidden that GNU make has
> -heuristics to avoid spawning a shell at all if the command is deemed
> -safe to be executed directly.} For example, change of directories or
> -variables persist between commands:
> +Generally, using GNU make is your best bet.
> +
> +Some make implementations (among them, FreeBSD @command{make}, NetBSD
> address@hidden, and Solaris @command{dmake}), when invoked with a
> address@hidden@var{N}} option, connect the standard output and standard
> +error of all their children processes to pipes or temporary regular

s/children/child/  (hasn't this been reviewed already?)

> +files.  This can lead to slightly but subtly different semantics in

s/slightly but//

> +the behaviour of the spawned processes.  For example, even if the
> address@hidden standard output is a connected to a tty, a command

s/is a/is/

> +like @code{test -t 1} issued in a recipe will return a failure within
> +those make implementations:

How about: ... is connected to a tty, the recipe commands will not be:

> address@hidden
> +$ @kbd{cat Makefile}
> +all:
> +        @@test -t 1 && echo "Is a tty" || echo "Is not a tty"
> +$ @kbd{make -j 2} # FreeBSD make
> +Is not a tty
> +$ @kbd{make -j 2} # NetBSD make
> +--- all ---
> +Is not a tty
> +$ @kbd{dmake -j 2} # Solaris dmake
> address@hidden --> 1 job
> address@hidden --> Job output
> +Is not a tty
> address@hidden example
> +
> address@hidden
> +while on the other hand:
> +
> address@hidden
> +$ @kbd{make -j 2} # GNU make, Heirloom make
> +Is a tty
> address@hidden example
> +
> +Furthermore, parallel run of those @command{make} implementations will

"a parallel run" or "parallel runs"

> +route standard error from commands that they spawn into their own

s/that they spawn //

> +standard output, and may remove leading whitespace from output lines.
> +
> +And there's more about NetBSD or FreeBSD @command{make}.  When run in

This sentence is newspaper style.  How about just
  When NetBSD or FreeBSD @command{make} are run in parallel mode, ...

> +parallel mode, they will reuse the same shell for multiple commands
> +within one recipe.  This can have unexpected
> address@hidden that GNU make has heuristics to avoid
> +spawning a shell at all if the command is deemed safe to be executed
> +directly.} For example, change of directories or variables persist
> +between commands:

Footnotes are really ugly, we should think of them as costing 5 bucks or
so each time we use them.  How about just omitting the footnote here,
since this isn't about GNU make otherwise at all?  Also, I don't know
what the immediate connection here is.

FWIW, newish GNU make supports .ONESHELL for reusing shells between
recipe commands.

>  @example
>  all:
> @@ -19518,12 +19557,9 @@ Another consequence of this is that, if one command 
> in a recipe uses
>  @code{exit 0} to indicate a successful exit, the shell will be gone
>  and the remaining commands of this recipe will not be executed.
>  
> -The above example also shows additional status output NetBSD
> address@hidden produces in parallel mode for targets being updated.
> -
> -Furthermore, parallel NetBSD @command{make} will route standard error
> -from commands that it spawns into its own standard output, and may
> -remove leading whitespace from output lines.
> +The above example also shows additional status output produced by NetBSD
> address@hidden@footnote{but @emph{not} by FreeBSD @command{make}} in

I'd just replace the footnote with the same text in parentheses.

> +parallel mode for targets being updated.
>  
>  You can avoid these issues by using the @option{-B} option to enable
>  compatibility semantics.  However, that will effectively also disable

Cheers,
Ralf