[FYI] Fix bug in test `backcompat4.test' (stale autom4te cache) (was: testsuite failure: backcompat4.test)

[Stefano Lattarini]

On Monday 08 November 2010, Ralf Wildenhues wrote:
> Hello Stefano,
Hi Ralf.

Wouldn't this report have been better suited for bug-automake, especially
now that you've gone through the hassle of setting that up as a real bug
tracker?

Anyway ...
 
> backcompat4.test is failing for me, see below.  This is with Autoconf
> 2.68.
> 
> Cheers,
> Ralf
> 
> FAIL: backcompat4.test (exit: 1)
> ================================
> 
> /tmp/automake/build/tests:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
> === Running test ../../automake/tests/backcompat4.test
> + pwd
> /tmp/automake/build/tests/backcompat4.dir
> + set -e
> + mkdir sub
> + :
> + :
> + :
> + :
> + :
> + :
> + :
> + ls -l
> total 56
> -rw-r----- 1 ralf    ralf       75 2010-11-08 09:44 configure.in
> -rwxr-x--- 1 ralf    ralf    20001 2010-11-08 09:44 depcomp
> -rw-r----- 1 ralf    ralf        0 2010-11-08 09:44 foo1.in
> -rw-r----- 1 ralf    ralf        0 2010-11-08 09:44 foo2.in
> -rw-r----- 1 ralf    ralf        0 2010-11-08 09:44 foo3.in
> -rw-r----- 1 ralf    ralf        0 2010-11-08 09:44 foo.in
> -rwxr-x--- 1 ralf    ralf    13781 2010-11-08 09:44 install-sh
> -rw-r----- 1 ralf    ralf        0 2010-11-08 09:44 Makefile.am
> -rwxr-x--- 1 ralf    ralf    11419 2010-11-08 09:44 missing
> drwxr-x--- 2 ralf    ralf     4096 2010-11-08 09:44 sub
> -rw-r----- 1 ralf    ralf        0 2010-11-08 09:44 zardoz
> + cat
> + aclocal-1.11a -Werror
> + automake-1.11a --foreign -Werror -Wall Makefile
> + mv -f Makefile.in Makefile.acf
> + cat
> + aclocal-1.11a -Werror
> + automake-1.11a --foreign -Werror -Wall Makefile
> + mv -f Makefile.in Makefile.aco
> + diff Makefile.acf Makefile.aco
> + cat
> + aclocal-1.11a -Werror
> + automake-1.11a --foreign -Werror -Wall Makefile
> + mv -f Makefile.in Makefile.acf
> + cat
> + aclocal-1.11a -Werror
> + automake-1.11a --foreign -Werror -Wall Makefile
> + mv -f Makefile.in Makefile.aco
> + diff Makefile.acf Makefile.aco
> + cat
> + aclocal-1.11a -Werror
> + automake-1.11a --foreign -Werror -Wall Makefile
> + mv -f Makefile.in Makefile.acf
> + cat
> + aclocal-1.11a -Werror
> + automake-1.11a --foreign -Werror -Wall Makefile
> + mv -f Makefile.in Makefile.aco
> + diff Makefile.acf Makefile.aco
> 36,37c36,37
> <     $(srcdir)/Makefile.in $(srcdir)/foo1.in \
> <     $(top_srcdir)/configure depcomp install-sh missing
> ---
> >     $(srcdir)/Makefile.in $(srcdir)/zardoz $(top_srcdir)/configure \
> >     depcomp install-sh missing
> 173c173
> < foo: $(top_builddir)/config.status $(srcdir)/foo1.in
> ---
> > foo: $(top_builddir)/config.status $(srcdir)/zardoz
> + exit_status=1
> + set +e
> + cd /tmp/automake/build/tests
> + test 0 != 0
> + echo backcompat4: exit 1
> backcompat4: exit 1
> + exit 1
> 

Yes, I'm experiencing similar non-deterministic failures:

 $ while true; do make check TESTS=backcompat4.test; done 2>&1 | grep -E 
'^(PASS|FAIL):'
 PASS: backcompat4.test
 PASS: backcompat4.test
 PASS: backcompat4.test
 PASS: backcompat4.test
 PASS: backcompat4.test
 FAIL: backcompat4.test
 PASS: backcompat4.test
 PASS: backcompat4.test
 PASS: backcompat4.test
 PASS: backcompat4.test
 PASS: backcompat4.test
 PASS: backcompat4.test
 FAIL: backcompat4.test
 PASS: backcompat4.test
 PASS: backcompat4.test
 PASS: backcompat4.test
 PASS: backcompat4.test
 PASS: backcompat4.test
 PASS: backcompat4.test
 PASS: backcompat4.test
 PASS: backcompat4.test
 PASS: backcompat4.test
 FAIL: backcompat4.test
 ^C

I think it's due to the fact that backcompat4.test don't remove
the old autom4te cache directory before re-running aclocal and
automake.  With the attached patch applied, I've not encounterd
the intermittent failure anymore in 100 tries.  So I've applied
the patch as a "bug-fixing patch" (as explained in HACKING),
merged to master, and pushed.

Regards,
   Stefano


From 928b446009f9c9abd47b41d7075d1113b7d9b369 Mon Sep 17 00:00:00 2001
From: Stefano Lattarini <address@hidden>
Date: Tue, 9 Nov 2010 11:58:23 +0100
Subject: [PATCH] Fix bug in test `backcompat4.test' (stale autom4te cache).

* tests/backcompat4.test: Remove stale autom4te cache directory
before re-running aclocal and automake.  Also, since we are at
it, be more verbose in displaying contents of generated files,
to ease debuggability.
Reported by Ralf Wildenhues.
---
 ChangeLog              |    9 +++++++++
 tests/backcompat4.test |    4 ++++
 2 files changed, 13 insertions(+), 0 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 6bbb49f..5bc575b 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,12 @@
+2010-11-09  Stefano Lattarini  <address@hidden>
+
+       Fix bug in test `backcompat4.test' (stale autom4te cache).
+       * tests/backcompat4.test: Remove stale autom4te cache directory
+       before re-running aclocal and automake.  Also, since we are at
+       it, be more verbose in displaying contents of generated files,
+       to ease debuggability.
+       Reported by Ralf Wildenhues.
+
 2010-11-06  Stefano Lattarini  <address@hidden>
 
        New tests on obsoleted usages of automake/autoconf macros (such
diff --git a/tests/backcompat4.test b/tests/backcompat4.test
index 1bee059..f0625ea 100755
--- a/tests/backcompat4.test
+++ b/tests/backcompat4.test
@@ -41,20 +41,24 @@ for args in \
   'foo:foo1.in:foo2.in:foo3.in' \
   'foo:foo1.in:foo2.in:sub/bar.in:foo3.in' \
 ; do
+  rm -rf autom4te*.cache aclocal.m4
   cat >configure.in <<END
 AC_INIT([$me], [1.0])
 AM_INIT_AUTOMAKE
 AC_CONFIG_FILES([$args])
 AC_OUTPUT
 END
+  cat configure.in # for debugging
   $ACLOCAL
   $AUTOMAKE Makefile
   mv -f Makefile.in Makefile.acf
+  rm -rf autom4te*.cache aclocal.m4
   cat >configure.in <<END
 AC_INIT([$me], [1.0])
 AM_INIT_AUTOMAKE
 AC_OUTPUT([$args])
 END
+  cat configure.in # for debugging
   $ACLOCAL
   $AUTOMAKE Makefile
   mv -f Makefile.in Makefile.aco
-- 
1.7.1