Re: make check(s) pre-release problems

[Zack Weinberg]

On 2022-10-04 6:58 PM, Karl Berry wrote:
> With Zack's latest Python fixes, I was hoping to move towards an
> Automake release, but I find myself stymied by apparently random and
> unreproducible test failures. I haven't exhausted every conceivable
> avenue yet, but I thought I would write in hopes that others (Zack, past
> Automake developers, anyone else ...) could give it a try, and/or have
> some insights.
>
> For me, running a parallel make check (with or without parallelizing the
> "internal" makes), or make distcheck, fails some tests, e.g., nodef,
> nodef2, testsuite-summary-reference-log. The exact tests that fail
> changes from run to run. Running the tests on their own succeeds. Ok, so
> it's something in the parallelism. But why? And how to debug?

I can't reproduce this problem myself, but my first thought is that some 
of the tests, when run concurrently, could be overwriting each other's 
files somehow.  I can think of two ways to investigate that hypothesis: 
look for tests that write files outside a directory dedicated to that 
test, and, after a failed test run, look for files that are corrupted,
then try to figure out which tests would be stomping on those files.

> Perhaps easier to debug: there are two targets to be run before making a
> release, check-no-trailing-backslash-in-recipes and check-cc-no-c-o,
> to try to ensure no reversion wrt these features. A special shell and
> compiler are configured, respectively (shell scripts that check the
> behavior).

I'm running these targets now and will report what I get.

zw