Re: Parallel build issue with bison 3.4.1 and make 3.81

[Akim Demaille]

Hi Thomas,

> Le 2 août 2019 à 09:43, Thomas Petazzoni <address@hidden> a écrit :
> 
> Hello,
> 
> As part of the Buildroot [0] project, we have a system of build
> machines that continuously build a large number of open-source
> packages.

Great!

> The problem is caused by the following piece of Makefile logic in
> examples/c/reccalc/local.mk:
> 
> %D%/scan.c %D%/scan.h: %D%/scan.stamp
>        @test -f $@ || rm -f %D%/scan.stamp
>        @test -f $@ || $(MAKE) $(AM_MAKEFLAGS) %D%/scan.stamp
> 
> %D%/scan.stamp: %D%/scan.l
>        $(AM_V_LEX)rm -f $@ $@.tmp
>        $(AM_V_at)$(MKDIR_P) %D%
>        $(AM_V_at)touch $@.tmp
>        $(AM_V_at)$(LEX) -o%D%/scan.c --header-file=%D%/scan.h 
> $(srcdir)/%D%/scan.l
>        $(AM_V_at)mv $@.tmp $@
> 
> The problem is that scan.c and scan.h already depend on scan.stamp, so
> it triggers the scan.stamp: scan.l rule. But then in addition, the
> commands in the scan.c/scan.h rule also run a sub-make to re-create
> scan.stamp, and those two sub-make invocations are executed in
> parallel, causing a race.

Yes.  I'm aware of the problem, but it was not causing any harm.  So far.
Yet I continuously use concurrent builds (including on the CI).

> In practice, it means that the scan.stamp: scan.l rule is executed 3
> times!

Yes.  And it's on purpose.  Run "info automake 'Multiple Outputs'" to get
the details of why things are this way (or see
https://www.gnu.org/software/automake/manual/html_node/Multiple-Outputs.html).

> One interesting point though is that this issue only appears with our
> build machine that has a fairly old version of make (3.81), and
> apparently not on other build machines.


Well, it seems kinda useless to launch the same job several times, maybe
that was addressed in more recent versions of GNU Make.

> Would it be possible to fix this issue ?

I thought Automake did not support headers generated by Flex, but it appear
it does.  Could you please confirm that this patch fixes your problem?
(I can't check it on the CI, for some unrelated reason.)

Cheers!



commit 192f9f3f093c5f123a1759366d990bab111cdd6b
Author: Akim Demaille <address@hidden>
Date:   Sat Aug 3 18:57:24 2019 +0200

    examples: rely on ylwrap
    
    Reported by Thomas Petazzoni.
    https://lists.gnu.org/archive/html/bug-bison/2019-08/msg00000.html
    
    * examples/c/reccalc/scan.l: Generate scan.h.
    * examples/c/reccalc/local.mk: Remove dedicated rules, leave the
    handling of multiple outputs to Automake's ylwrap.

diff --git a/THANKS b/THANKS
index 14a26d76..5ee4b29a 100644
--- a/THANKS
+++ b/THANKS
@@ -171,6 +171,7 @@ Sum Wu                    address@hidden
 Théophile Ranquet         address@hidden
 Thiru Ramakrishnan        address@hidden
 Thomas Jahns              address@hidden
+Thomas Petazzoni          address@hidden
 Tim Josling               address@hidden
 Tim Landscheidt           address@hidden
 Tim Van Holder            address@hidden
diff --git a/examples/c/reccalc/local.mk b/examples/c/reccalc/local.mk
index bb64b214..2d99fa62 100644
--- a/examples/c/reccalc/local.mk
+++ b/examples/c/reccalc/local.mk
@@ -37,18 +37,6 @@ endif FLEX_WORKS
 DASH = -
 %D%/reccalc$(DASH)parse.o: %D%/scan.h
 
-%D%/scan.c %D%/scan.h: %D%/scan.stamp
-       @test -f $@ || rm -f %D%/scan.stamp
-       @test -f $@ || $(MAKE) $(AM_MAKEFLAGS) %D%/scan.stamp
-
-%D%/scan.stamp: %D%/scan.l
-       $(AM_V_LEX)rm -f $@ $@.tmp
-       $(AM_V_at)$(MKDIR_P) %D%
-       $(AM_V_at)touch $@.tmp
-       $(AM_V_at)$(LEX) -o%D%/scan.c --header-file=%D%/scan.h 
$(srcdir)/%D%/scan.l
-       $(AM_V_at)mv $@.tmp $@
-
-
 EXTRA_DIST += %D%/reccalc.test %D%/scan.l
 dist_reccalc_DATA = %D%/parse.y %D%/scan.l %D%/Makefile %D%/README.md
 CLEANFILES += %D%/parse.[ch] %D%/parse.output %D%/scan.[ch] %D%/*.stamp
diff --git a/examples/c/reccalc/scan.l b/examples/c/reccalc/scan.l
index fbb781aa..f8ae829b 100644
--- a/examples/c/reccalc/scan.l
+++ b/examples/c/reccalc/scan.l
@@ -3,7 +3,7 @@
 /* Disable Flex features we don't need, to avoid warnings. */
 %option nodefault noinput nounput noyywrap
 
-%option reentrant
+%option reentrant header-file="scan.h"
 
 %{
 #include <assert.h>