Parallel build issue with bison 3.4.1 and make 3.81

[Thomas Petazzoni]

Hello,

As part of the Buildroot [0] project, we have a system of build
machines that continuously build a large number of open-source
packages. We recently updated Bison to 3.4.1, and started seeing build
failures:

  http://autobuild.buildroot.net/?reason=host-bison-3.4.1

The error looks like this:

make[2]: Entering directory 
`/home/test/autobuild/run/instance-3/output/build/host-bison-3.4.1'
  LEX      examples/c/reccalc/scan.stamp
make[2]: Entering directory 
`/home/test/autobuild/run/instance-3/output/build/host-bison-3.4.1'
  LEX      examples/c/reccalc/scan.stamp
make[2]: Leaving directory 
`/home/test/autobuild/run/instance-3/output/build/host-bison-3.4.1'
mv: cannot stat `examples/c/reccalc/scan.stamp.tmp': No such file or directory
make[2]: *** [examples/c/reccalc/scan.stamp] Error 1

The problem is caused by the following piece of Makefile logic in
examples/c/reccalc/local.mk:

%D%/scan.c %D%/scan.h: %D%/scan.stamp
        @test -f $@ || rm -f %D%/scan.stamp
        @test -f $@ || $(MAKE) $(AM_MAKEFLAGS) %D%/scan.stamp

%D%/scan.stamp: %D%/scan.l
        $(AM_V_LEX)rm -f $@ $@.tmp
        $(AM_V_at)$(MKDIR_P) %D%
        $(AM_V_at)touch $@.tmp
        $(AM_V_at)$(LEX) -o%D%/scan.c --header-file=%D%/scan.h 
$(srcdir)/%D%/scan.l
        $(AM_V_at)mv $@.tmp $@

The problem is that scan.c and scan.h already depend on scan.stamp, so
it triggers the scan.stamp: scan.l rule. But then in addition, the
commands in the scan.c/scan.h rule also run a sub-make to re-create
scan.stamp, and those two sub-make invocations are executed in
parallel, causing a race.

In practice, it means that the scan.stamp: scan.l rule is executed 3
times! One time because it's a dependency of the scan.c/scan.h rule,
and two more times due to the two sub-makes.

This seems suboptimal, but also creates a race condition causing the
build failures.

One interesting point though is that this issue only appears with our
build machine that has a fairly old version of make (3.81), and
apparently not on other build machines.

Would it be possible to fix this issue ?

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com