Should make handle non-blocking jobserver and use socket pair instead?

[Juha-Matti Tilli]

Hello,

(Please CC me, I’m not on the list)

I’m creating a new build system that is supposed to solve many of the
issues of standard make. Most programmers who create alternative build
systems (SCons, Rake, Shake) do so by selecting an alternative
programming language (Python, Ruby, Haskell), whereas I’m doing it in
C. Most programmers also create the system by incompatibility and
making it wildly different from make, whereas I’m keeping most of the
features in make that are actually good. That includes the jobserver.

My build system can fork recursive sub-makes, and has support for the
jobserver. While creating the support for jobserver, I noticed a
problem in GNU make handling of jobserver.

GNU make expects the jobserver to be blocking, whereas a program based
on an event-loop architecture should really be using non-blocking file
descriptors.

If the jobserver is set to be non-blocking (I’m creating the jobserver
in my build system and passing it to GNU make), GNU make gives this
error:

make: *** read jobs pipe: Resource temporarily unavailable.  Stop.
make: *** Waiting for unfinished jobs....

I’m wondering whether this error exit() is the right thing to do. In
the programming world, there is a lot of variability and different
choices. Some prefer C, some C++, some Rust. Similarly, some may
prefer GNU make and some may prefer alternative build systems. In my
opinion, GNU make, including parallel make, should attempt to be as
compatible as possible with all other build systems, because not every
programmer wants to use blocking file descriptors. The jobserver is
created by the top-level build system always, and it may be something
else than GNU make. Thus, I'm proposing a "be liberal in what you
accept; be conservative in what you do" approach. Now GNU make doesn't
accept a non-blocking jobserver.

Due to the reasons given in https://cr.yp.to/unix/nonblock.html I
cannot turn on O_NONBLOCK even after dup() because the flag hits the
entire ofile, not just the fd, and because GNU make does not work with
a non-blocking jobserver. I see two solutions:

1. Create a socket pair instead of a pipe so that I can use
MSG_DONTWAIT on a blocking socket

2. Use a select/poll + read pair, and immediately before the read,
turn on a 10 millisecond interval timer just in case some sub-make won
the race to acquire the token

Both of these solutions seem to work, and for compatibility with old
GNU make variants, I really have to use either solution for the next
10 years or so. In case the jobserver was created by make and not my
tool, the MSG_DONTWAIT doesn’t work, so in that case I have to revert
to the second solution. My current approach is to try MSG_DONTWAIT
first and in case of ENOTSOCK use the interval timer with read().

However, I think the issue should be fixed in GNU make too.
Specifically, I think GNU make should read(), and if it gives EAGAIN,
do a select()/poll() and read() again, iterating as long as read()
returns EAGAIN. Of course, some systems may not have a functioning
select()/poll(), so the special EAGAIN handling would apply only to
those systems that fully support select()/poll().

In fact, I argue that GNU make should too create a SOCK_STREAM socket
pair instead of a pipe, if the socket pair facility is available.
Socket pairs are superior in many aspects, including the possibility
to use MSG_DONTWAIT. I have tested that GNU make works perfectly with
a jobserver that is actually a socket pair instead of a pipe, but then
that jobserver needs to be created externally, as the GNU make
creating the jobserver creates a pipe and not a socket pair. As a
hack, a wrapper program could do it and then call GNU make.

What are your opinions of this? I can prepare a patch / patches for
GNU make if it would be acceptable. Of course I won't be wasting any
time reading GNU make sources if the improvement proposal is rejected.

BR, Juha-Matti