Re: preparing for a new release

[Jim Meyering]

On Sun, Nov 15, 2020 at 6:52 PM Karl Berry <karl@freefriends.org> wrote:
>
> Hi Jim - (and thanks for the test runs and logs, Gavin)
>
>       make -k check TESTS='t/txinfo-no-clutter' \
>                     AM_TESTSUITE_MAKE="make -j$(( 2*$(nproc) + 1 ))"
>
> I know little about programming for parallel (auto)make, but I'm
> guessing that with the above invocation, the make commands inside the
> test are also being run in parallel?
>
> As far as I can see, this test is simply not set up to support that. It
> essentially runs, for every format being built by texi2dvi:
>   make FMT && make clean
>
> Thus it seems to me that one make clean could be cleaning the .dvi while
> the next make ps is trying to use it.
>
> And/or, maybe dependencies are missing. all-local is depending on
> "ps pdf dvi html", so I suppose they'll get spawned in parallel.  And the
> only dependency I'm seeing is the suffix rule, .dvi.ps, so maybe the
> purportedly intermediate .dvi is getting removed by one of the parallel
> makes?  So I tried adding
>
> .PRECIOUS: %.dvi
>
> to both Makefile.am's, and the fancy make command above then succeeded.
> Does that get around the problem for you?  Doesn't exactly seem like the
> best fix, though, since ordinarily one would want the intermediate file
> to be deleted. Maybe it would be cleaner to just force serialization of
> the makes inside this test? Is that possible?
>
> Further:
>
> 1) I am not smart enough to tell what's actually happening from the log
> since there's no indication of which make is doing what (and I can't
> even imagine how that could be indicated). How does one ever debug these
> things for sure?
>
> 2) I had not seen AM_TESTSUITE_MAKE before. I was running tests simply
> with make -j12 TARGET, and there the test does not fail -- not
> surprisingly, I guess, since then (I think) the makes inside the test
> are not being run in parallel.
>
> 3) I can't see why this is anything new. That test has not changed since
> 2012. The mentioned bug
> (https://debbugs.gnu.org/cgi/bugreport.cgi?bug=11146) describes the
> commit at that time as improving the test wrt parallelism.  Thus I
> surmise no one has ever tried with a parallel AM_TESTSUITE_MAKE before now.
>
> 4) I feel surprised that this parallelism does not cause trouble with
> other tests that involve suffix rules, like yacc or lex. I don't see
> .PRECIOUS being used in any other test, aside from precious.sh. But
> maybe no other test is doing quite what txinfo-no-clutter is.

Thanks for investigating. I've just fixed that with the attached (add
.NOTPARALLEL):

0001-tests-avoid-missing-.dvi-failure-with-parallel-tests.patch
Description: Binary data