Re: prepare Automake's test suite for parallelization

[Ralf Wildenhues]

* Ralf Wildenhues wrote on Sun, Nov 04, 2007 at 08:16:32PM CET:
> * Benoit SIGOURE wrote:
> > Most modern OSes randomize PIDs, so it's 
> > going to be cumbersome.  My personal opinion would be to use something 
> > based on the test name.  If you want, you can try to combine the test-name 
> > and the PID?  Or maybe increment a number for each test and use that?  
> > (this will need to deal nicely with parallel make check though, so it might 
> > not be trivial).
> 
> Using a number is ugly for parallel, as you say.  Based on test name (or
> log name) would be ok.

Here's an updated patch with test directories named after the test:
  foo23.test -> foo23.dir

I'll probably want to revisit the `keep_testdirs' "feature" later, but
for now I'll just leave that in.

Applied and pushed.

BTW, there is one thing about the Automake test suite that has been
bothering me for a long time: you often cannot interrupt it with ^C.
I usually keep it pressed until one of the repeated interrupts finally
kills the testing.  I've never been able to spot why exactly that is,
I assume the non-interruptability of aclocal is a problem here.
I don't even know why that is the case.  Hmm.

Cheers,
Ralf

    Run each test in a process-private subdirectory.
    
    * tests/defs.in: Employ a trap to clean up at the end of the
    test in case the test succeeded.  This needs to be revisited
    because we would like to keep the output of XPASSing tests.
    Also, note when we were interrupted by a signal.  Code taken
    from Autoconf's _AC_INIT_PREPARE.
    If $keep_testdirs is set, keep them even for successful tests.
    * tests/Makefile.am (check-clean-local): New target.  Remove all
    tests subdirs, but also make them writable first.  The latter
    was already needed without per-test subdirs.
    (distclean-local): Depend upon check-clean-local.
    * tests/Makefile.in: Regenerate.
    * tests/.cvsignore, tests/.gitignore: Adjust.

diff --git a/ChangeLog b/ChangeLog
index fa941b4..b79015e 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,19 @@
 2007-11-10  Ralf Wildenhues  <address@hidden>
 
+       Run each test in a process-private subdirectory.
+       * tests/defs.in: Employ a trap to clean up at the end of the
+       test in case the test succeeded.  This needs to be revisited
+       because we would like to keep the output of XPASSing tests.
+       Also, note when we were interrupted by a signal.  Code taken
+       from Autoconf's _AC_INIT_PREPARE.
+       If $keep_testdirs is set, keep them even for successful tests.
+       * tests/Makefile.am (check-clean-local): New target.  Remove all
+       tests subdirs, but also make them writable first.  The latter
+       was already needed without per-test subdirs.
+       (distclean-local): Depend upon check-clean-local.
+       * tests/Makefile.in: Regenerate.
+       * tests/.cvsignore, tests/.gitignore: Adjust.
+
        Fix default includes ordering to be `-I. -I$(srcdir) ...' again.
        * automake.in (handle_compile): Put -I$(srcdir) before include
        paths for config headers, as was done before Automake 1.10, but
diff --git a/HACKING b/HACKING
index 5a3ed33..9f2f1f4 100644
--- a/HACKING
+++ b/HACKING
@@ -119,6 +119,8 @@
 
 * Make sure each test file is executable
 
+* Use `keep_testdirs=yes' to keep test directories for successful
+  tests also.
 
 ================================================================
 = Release procedure
diff --git a/tests/.cvsignore b/tests/.cvsignore
index c28dc16..69d3d43 100644
--- a/tests/.cvsignore
+++ b/tests/.cvsignore
@@ -5,6 +5,6 @@ automake
 defs
 diffs
 log
-testSubDir
+*.dir
 aclocal-*
 automake-*
diff --git a/tests/.gitignore b/tests/.gitignore
index 0859387..f1c3bff 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -1,4 +1,4 @@
 aclocal-*
 automake-*
 defs
-testSubDir
+*.dir
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 81a0382..3c586a1 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -627,5 +627,8 @@ EXTRA_DIST = ChangeLog-old $(TESTS)
 # Each test case depends on defs, aclocal, and automake.
 check_SCRIPTS = defs aclocal-$(APIVERSION) automake-$(APIVERSION)
 
-distclean-local:
-       -rm -rf testSubDir
+distclean-local: check-clean-local
+
+check-clean-local:
+       -chmod -R u+rwx *.dir
+       -rm -rf *.dir
diff --git a/tests/Makefile.in b/tests/Makefile.in
index 999ed24..b446130 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -1023,8 +1023,11 @@ uninstall-am:
        pdf-am ps ps-am uninstall uninstall-am
 
 
-distclean-local:
-       -rm -rf testSubDir
+distclean-local: check-clean-local
+
+check-clean-local:
+       -chmod -R u+rwx *.dir
+       -rm -rf *.dir
 # Tell versions [3.59,3.63) of GNU make to not export all variables.
 # Otherwise a system limit (for SysV at least) may be exceeded.
 .NOEXPORT:
diff --git a/tests/defs.in b/tests/defs.in
index 78cf37a..d79d874 100644
--- a/tests/defs.in
+++ b/tests/defs.in
@@ -235,16 +235,35 @@ case "$srcdir" in
     ;;
 esac
 
-chmod -R a+rwx testSubDir > /dev/null 2>&1
-rm -rf testSubDir > /dev/null 2>&1
-mkdir testSubDir
+curdir=`pwd`
+testSubDir=$me.dir
+chmod -R u+rwx $testSubDir > /dev/null 2>&1
+rm -rf $testSubDir > /dev/null 2>&1
+mkdir $testSubDir
+
+trap 'exit_status=$?
+  cd "$curdir"
+  case $exit_status,$keep_testdirs in
+  0,)
+    chmod -R a+rwx $testSubDir > /dev/null 2>&1
+    rm -rf "$testSubDir" ;;
+  esac
+  test "$signal" != 0 &&
+    echo "$as_me: caught signal $signal"
+  echo "$as_me: exit $exit_status"
+  exit $exit_status
+' 0
+for signal in 1 2 13 15; do
+  trap 'signal='$signal'; { (exit 1); exit 1; }' $signal
+done
+signal=0
 
 # Copy in some files we need.
 for file in install-sh missing depcomp; do
-   cp $srcdir/../lib/$file testSubDir/$file || exit 1
+   cp $srcdir/../lib/$file $testSubDir/$file || exit 1
 done
 
-cd ./testSubDir
+cd ./$testSubDir
 
 # Build appropriate environment in test directory.  Eg create
 # configure.in, touch all necessary files, etc.