[PATCH] parallel-tests: new recognized test result 'ERROR'

automake-patches

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[PATCH] parallel-tests: new recognized test result 'ERROR'

From:	Stefano Lattarini
Subject:	[PATCH] parallel-tests: new recognized test result 'ERROR'
Date:	Thu, 30 Jun 2011 16:58:21 +0200
User-agent:	KMail/1.13.3 (Linux/2.6.30-2-686; KDE/4.4.4; i686; ; )

I've applied the attached patch to the 'GSoC/experimental/test-results-work'
branch  Note that this is not an FYI, since that branch is temporary, and can
thus be amended and modified.  Reviews are welcome!

Thanks,
  Stefano

From 2df72384ab57d165c7e1da0f88e513171a0035e8 Mon Sep 17 00:00:00 2001
Message-Id: <address@hidden>
From: Stefano Lattarini <address@hidden>
Date: Thu, 30 Jun 2011 14:55:43 +0200
Subject: [PATCH] parallel-tests: new recognized test result 'ERROR'

* lib/am/check.am ($(TEST_SUITE_LOG)): Recognize a new test result
`ERROR'.  Use it when encountering unreadable test logs (previously
a simple `FAIL' was used in this situations).
* lib/test-driver: Set the global test result to `ERROR' when the
test exit status is 99.  When doing colorized output, color `ERROR'
results in magenta.
* doc/automake.texi (Log files generation and test results
recording): Update by also listing `ERROR' among the list of valid
`:test-results:' arguments.
* tests/trivial-test-driver: Update.
* tests/parallel-tests.test: Likewise.
* tests/parallel-tests-harderror.test: Likewise.
* tests/parallel-tests-no-spurious-summary.test: Likewise.
* tests/test-driver-global-log.test: Likewise.
* tests/test-driver-recheck.test: Likewise.
* tests/test-driver-custom-multitest-recheck.test: Likewise.
* tests/test-driver-custom-multitest-recheck2.test: Likewise.
* tests/test-driver-custom-multitest.test: Likewise.
* tests/test-driver-custom-no-html.test: Likewise.
* tests/test-driver-end-test-results.test: Likewise.
* tests/color.test: Likewise, and make stricter.
* tests/color2.test: Likewise, and improve syncing with color.test.
* tests/parallel-tests-exit-statuses.test: New test.
* tests/parallel-tests-console-output.test: New test.
* tests/Makefile.am (TESTS): Update.
---
 ChangeLog                                        |   29 ++++++
 doc/automake.texi                                |    8 +-
 lib/am/check.am                                  |    8 +-
 lib/test-driver                                  |    5 +-
 tests/Makefile.am                                |    2 +
 tests/Makefile.in                                |    6 +-
 tests/color.test                                 |   48 ++++++----
 tests/color2.test                                |   51 ++++++----
 tests/parallel-tests-console-output.test         |  104 +++++++++++++++++++++
 tests/parallel-tests-exit-statuses.test          |  108 ++++++++++++++++++++++
 tests/parallel-tests-harderror.test              |    6 +-
 tests/parallel-tests-no-spurious-summary.test    |    2 +
 tests/parallel-tests.test                        |    9 +-
 tests/test-driver-custom-multitest-recheck.test  |   27 +++---
 tests/test-driver-custom-multitest-recheck2.test |   56 ++++++++---
 tests/test-driver-custom-multitest.test          |   33 ++++---
 tests/test-driver-custom-no-html.test            |    2 +-
 tests/test-driver-end-test-results.test          |    1 +
 tests/test-driver-global-log.test                |    4 +-
 tests/test-driver-recheck.test                   |    2 +-
 tests/trivial-test-driver                        |   11 +-
 21 files changed, 419 insertions(+), 103 deletions(-)
 create mode 100755 tests/parallel-tests-console-output.test
 create mode 100755 tests/parallel-tests-exit-statuses.test

diff --git a/ChangeLog b/ChangeLog
index ab7ba2f..22e7087 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,32 @@
+2011-06-30  Stefano Lattarini  <address@hidden>
+
+       parallel-tests: new recognized test result 'ERROR'
+       * lib/am/check.am ($(TEST_SUITE_LOG)): Recognize a new test result
+       `ERROR'.  Use it when encountering unreadable test logs (previously
+       a simple `FAIL' was used in this situations).
+       * lib/test-driver: Set the global test result to `ERROR' when the
+       test exit status is 99.  When doing colorized output, color `ERROR'
+       results in magenta.
+       * doc/automake.texi (Log files generation and test results
+       recording): Update by listing `ERROR' too among the list of valid
+       `:test-results:' arguments.
+       * tests/trivial-test-driver: Update.
+       * tests/parallel-tests.test: Likewise.
+       * tests/parallel-tests-harderror.test: Likewise.
+       * tests/parallel-tests-no-spurious-summary.test: Likewise.
+       * tests/test-driver-global-log.test: Likewise.
+       * tests/test-driver-recheck.test: Likewise.
+       * tests/test-driver-custom-multitest-recheck.test: Likewise.
+       * tests/test-driver-custom-multitest-recheck2.test: Likewise.
+       * tests/test-driver-custom-multitest.test: Likewise.
+       * tests/test-driver-custom-no-html.test: Likewise.
+       * tests/test-driver-end-test-results.test: Likewise.
+       * tests/color.test: Likewise, and make stricter.
+       * tests/color2.test: Likewise, and improve syncing with color.test.
+       * tests/parallel-tests-exit-statuses.test: New test.
+       * tests/parallel-tests-console-output.test: New test.
+       * tests/Makefile.am (TESTS): Update.
+
 2011-06-29  Stefano Lattarini  <address@hidden>
 
        parallel-tests: make parsing of test results safer
diff --git a/doc/automake.texi b/doc/automake.texi
index 1e251db..16804cd 100644
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -9248,10 +9248,10 @@ leading whitespace will not be ignored.
 
 @c Keep this in sync with lib/am/check-am:$(TEST_SUITE_LOG).
 The only recognized test results are currently @code{PASS}, @code{XFAIL},
address@hidden, @code{FAIL} and @code{XPASS}.  These results, when declared
-with @code{:test-result:}, can be optionally followed by text holding
-the name and/or a brief description of the corresponding test; the
address@hidden harness will ignore such extra text when
address@hidden, @code{FAIL}, @code{XPASS} and @code{ERROR}.  These results,
+when declared with @code{:test-result:}, can be optionally followed by
+text holding the name and/or a brief description of the corresponding
+test; the @option{parallel-tests} harness will ignore such extra text when
 generating @file{test-suite.log} and preparing the testsuite summary.
 Also, @code{:test-result:} can be used with a special ``pseudo-result''
 @code{END}, that will instruct the testsuite harness to stop scanning
diff --git a/lib/am/check.am b/lib/am/check.am
index f2528e0..76609b5 100644
--- a/lib/am/check.am
+++ b/lib/am/check.am
@@ -66,7 +66,7 @@ include inst-vars.am
 ## appended.
 ##
 ## In addition to the magic "exit 77 means SKIP" feature (which was
-## imported from automake), there is a magic "exit 99 means FAIL" feature
+## imported from automake), there is a magic "exit 99 means ERROR" feature
 ## which is useful if you need to issue a hard error no matter whether the
 ## test is XFAIL or not.  You can disable this feature by setting the
 ## variable DISABLE_HARD_ERRORS to a nonempty value.
@@ -153,7 +153,7 @@ $(TEST_SUITE_LOG): $(TEST_LOGS)
 ## Readable test logs.
        list2=`for f in $$list; do test ! -r $$f || echo $$f; done`; \
 ## Each unreadable test log counts as a failed test.
-       results1=`for f in $$list; do test -r $$f || echo FAIL; done`; \
+       results1=`for f in $$list; do test -r $$f || echo ERROR; done`; \
 ## Now we're going to extract the outcome of all the testcases from the
 ## test logs.
        results2=''; \
@@ -187,6 +187,10 @@ $(TEST_SUITE_LOG): $(TEST_LOGS)
        skip=`echo "$$results" | grep -c '^SKIP'`;                      \
        xfail=`echo "$$results" | grep -c '^XFAIL'`;                    \
        xpass=`echo "$$results" | grep -c '^XPASS'`;                    \
+       error=`echo "$$results" | grep -c '^ERROR'`;                    \
+## FIXME: for the moment, we count errors as failures, otherwise the code
+## that display the testsuite summary will become too much complicated.
+       fail=`expr $$fail + $$error`;                                   \
        failures=`expr $$fail + $$xpass`;                               \
        all=`expr $$all - $$skip`;                                      \
        if test "$$all" -eq 1; then tests=test; All=;                   \
diff --git a/lib/test-driver b/lib/test-driver
index 145bbea..aa39c5b 100755
--- a/lib/test-driver
+++ b/lib/test-driver
@@ -85,9 +85,10 @@ if test $color_tests = yes; then
   grn='[0;32m' # Green.
   lgn='[1;32m' # Light green.
   blu='[1;34m' # Blue.
+  mgn='[0;35m' # Magenta.
   std='[m'     # No color.
 else
-  red= grn= lgn= blu= std=
+  red= grn= lgn= blu= mgn= std=
 fi
 
 tmpfile=$logfile-t
@@ -109,7 +110,7 @@ case $estatus:$expect_failure in
   0:yes) col=$red; res=XPASS;;
   0:*)   col=$grn; res=PASS ;;
   77:*)  col=$blu; res=SKIP ;;
-  99:*)  col=$red; res=FAIL ;;
+  99:*)  col=$mgn; res=ERROR;;
   *:yes) col=$lgn; res=XFAIL;;
   *:*)   col=$red; res=FAIL ;;
 esac
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 9c0605d..64299b8 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -731,6 +731,8 @@ parallel-tests-log-override-recheck.test \
 parallel-tests-empty-testlogs.test \
 parallel-test-driver-install.test \
 parallel-tests-no-spurious-summary.test \
+parallel-tests-exit-statuses.test \
+parallel-tests-console-output.test \
 test-driver-end-test-results.test \
 test-driver-custom-no-extra-driver.test \
 test-driver-custom.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index dc03179..5ef82da 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -991,6 +991,8 @@ parallel-tests-log-override-recheck.test \
 parallel-tests-empty-testlogs.test \
 parallel-test-driver-install.test \
 parallel-tests-no-spurious-summary.test \
+parallel-tests-exit-statuses.test \
+parallel-tests-console-output.test \
 test-driver-end-test-results.test \
 test-driver-custom-no-extra-driver.test \
 test-driver-custom.test \
@@ -1400,7 +1402,7 @@ $(TEST_SUITE_LOG): $(TEST_LOGS)
        nlinit=`echo 'nl="'; echo '"'`; eval "$$nlinit"; unset nlinit; \
        list='$(TEST_LOGS)'; \
        list2=`for f in $$list; do test ! -r $$f || echo $$f; done`; \
-       results1=`for f in $$list; do test -r $$f || echo FAIL; done`; \
+       results1=`for f in $$list; do test -r $$f || echo ERROR; done`; \
        results2=''; \
        exec 5<&0; \
        for f in $$list2; do \
@@ -1421,6 +1423,8 @@ $(TEST_SUITE_LOG): $(TEST_LOGS)
        skip=`echo "$$results" | grep -c '^SKIP'`;                      \
        xfail=`echo "$$results" | grep -c '^XFAIL'`;                    \
        xpass=`echo "$$results" | grep -c '^XPASS'`;                    \
+       error=`echo "$$results" | grep -c '^ERROR'`;                    \
+       fail=`expr $$fail + $$error`;                                   \
        failures=`expr $$fail + $$xpass`;                               \
        all=`expr $$all - $$skip`;                                      \
        if test "$$all" -eq 1; then tests=test; All=;                   \
diff --git a/tests/color.test b/tests/color.test
index ecd4f94..bcaf17b 100755
--- a/tests/color.test
+++ b/tests/color.test
@@ -22,18 +22,24 @@
 TERM=ansi
 export TERM
 
-red='[0;31m'
-grn='[0;32m'
-lgn='[1;32m'
-blu='[1;34m'
-std='[m'
+esc=''
+red="$esc[0;31m"
+grn="$esc[0;32m"
+lgn="$esc[1;32m"
+blu="$esc[1;34m"
+mgn="$esc[0;35m"
+std="$esc[m"
+
+# Tho old serial testsuite driver doesn't distinguish between failures
+# and hard errors.
+test x"$parallel_tests" = x"yes" || mgn=$red
 
 # Check that grep can parse nonprinting characters.
 # BSD 'grep' works from a pipe, but not a seekable file.
 # GNU or BSD 'grep -a' works on files, but is not portable.
-case `echo "$std" | grep .` in
+case `echo "$std" | $EGREP .` in
   "$std") ;;
-  *) echo "$me: grep can't parse nonprinting characters" >&2; Exit 77;;
+  *) echo "$me: egrep can't parse nonprinting characters" >&2; Exit 77;;
 esac
 
 cat >>configure.in <<END
@@ -43,7 +49,7 @@ END
 cat >Makefile.am <<'END'
 AUTOMAKE_OPTIONS = color-tests
 TESTS = $(check_SCRIPTS)
-check_SCRIPTS = pass fail skip xpass xfail
+check_SCRIPTS = pass fail skip xpass xfail error
 XFAIL_TESTS = xpass xfail
 END
 
@@ -62,9 +68,14 @@ cat >skip <<END
 exit 77
 END
 
+cat >error <<END
+#! /bin/sh
+exit 99
+END
+
 cp fail xfail
 cp pass xpass
-chmod +x pass fail skip xpass xfail
+chmod +x pass fail skip xpass xfail error
 
 $ACLOCAL
 $AUTOCONF
@@ -75,21 +86,20 @@ $AUTOMAKE --add-missing
 test_color ()
 {
   # Not a useless use of cat; see above comments about grep.
-  cat stdout | grep ": pass" | $FGREP "$grn"
-  cat stdout | grep ": fail" | $FGREP "$red"
-  cat stdout | grep ": xfail" | $FGREP "$lgn"
-  cat stdout | grep ": xpass" | $FGREP "$red"
-  cat stdout | grep ": skip" | $FGREP "$blu"
+  cat stdout | $EGREP ": pass" | $FGREP "$grn"
+  cat stdout | $EGREP ": fail" | $FGREP "$red"
+  cat stdout | $EGREP ": xfail" | $FGREP "$lgn"
+  cat stdout | $EGREP ": xpass" | $FGREP "$red"
+  cat stdout | $EGREP ": skip" | $FGREP "$blu"
+  cat stdout | $EGREP ": error" | $FGREP "$mgn"
 }
 
 test_no_color ()
 {
   # Not a useless use of cat; see above comments about grep.
-  cat stdout | grep ": pass" | $FGREP "$grn" && Exit 1
-  cat stdout | grep ": fail" | $FGREP "$red" && Exit 1
-  cat stdout | grep ": xfail" | $FGREP "$lgn" && Exit 1
-  cat stdout | grep ": xpass" | $FGREP "$red" && Exit 1
-  cat stdout | grep ": skip" | $FGREP "$blu" && Exit 1
+  cat stdout \
+    | $EGREP ": (pass|fail|xfail|xpass|skip|error)" \
+    | $FGREP "$esc" && Exit 1
   :
 }
 
diff --git a/tests/color2.test b/tests/color2.test
index 4f8862e..f8996ce 100755
--- a/tests/color2.test
+++ b/tests/color2.test
@@ -22,18 +22,24 @@
 TERM=ansi
 export TERM
 
-red='[0;31m'
-grn='[0;32m'
-lgn='[1;32m'
-blu='[1;34m'
-std='[m'
+esc=''
+red="$esc[0;31m"
+grn="$esc[0;32m"
+lgn="$esc[1;32m"
+blu="$esc[1;34m"
+mgn="$esc[0;35m"
+std="$esc[m"
+
+# Tho old serial testsuite driver doesn't distinguish between failures
+# and hard errors.
+test x"$parallel_tests" = x"yes" || mgn=$red
 
 # Check that grep can parse nonprinting characters.
 # BSD 'grep' works from a pipe, but not a seekable file.
 # GNU or BSD 'grep -a' works on files, but is not portable.
-case `echo "$std" | grep .` in
+case `echo "$std" | $EGREP .` in
   "$std") ;;
-  *) echo "$me: grep can't parse nonprinting characters" >&2; Exit 77;;
+  *) echo "$me: egrep can't parse nonprinting characters" >&2; Exit 77;;
 esac
 
 # Check that we have a working expect program.
@@ -56,7 +62,7 @@ END
 cat >Makefile.am <<'END'
 AUTOMAKE_OPTIONS = color-tests
 TESTS = $(check_SCRIPTS)
-check_SCRIPTS = pass fail skip xpass xfail
+check_SCRIPTS = pass fail skip xpass xfail error
 XFAIL_TESTS = xpass xfail
 END
 
@@ -75,33 +81,38 @@ cat >skip <<END
 exit 77
 END
 
+cat >error <<END
+#! /bin/sh
+exit 99
+END
+
 cp fail xfail
 cp pass xpass
-chmod +x pass fail skip xpass xfail
+chmod +x pass fail skip xpass xfail error
 
 $ACLOCAL
-$AUTOMAKE -a
 $AUTOCONF
+$AUTOMAKE --add-missing
+
 ./configure
 
 test_color ()
 {
   # Not a useless use of cat; see above comments about grep.
-  cat stdout | grep ": pass" | $FGREP "$grn"
-  cat stdout | grep ": fail" | $FGREP "$red"
-  cat stdout | grep ": xfail" | $FGREP "$lgn"
-  cat stdout | grep ": xpass" | $FGREP "$red"
-  cat stdout | grep ": skip" | $FGREP "$blu"
+  cat stdout | $EGREP ": pass" | $FGREP "$grn"
+  cat stdout | $EGREP ": fail" | $FGREP "$red"
+  cat stdout | $EGREP ": xfail" | $FGREP "$lgn"
+  cat stdout | $EGREP ": xpass" | $FGREP "$red"
+  cat stdout | $EGREP ": skip" | $FGREP "$blu"
+  cat stdout | $EGREP ": error" | $FGREP "$mgn"
 }
 
 test_no_color ()
 {
   # Not a useless use of cat; see above comments about grep.
-  cat stdout | grep ": pass" | $FGREP "$grn" && Exit 1
-  cat stdout | grep ": fail" | $FGREP "$red" && Exit 1
-  cat stdout | grep ": xfail" | $FGREP "$lgn" && Exit 1
-  cat stdout | grep ": xpass" | $FGREP "$red" && Exit 1
-  cat stdout | grep ": skip" | $FGREP "$blu" && Exit 1
+  cat stdout \
+    | $EGREP ": (pass|fail|xfail|xpass|skip|error)" \
+    | $FGREP "$esc" && Exit 1
   :
 }
 
diff --git a/tests/parallel-tests-console-output.test 
b/tests/parallel-tests-console-output.test
new file mode 100755
index 0000000..5c59b72
--- /dev/null
+++ b/tests/parallel-tests-console-output.test
@@ -0,0 +1,104 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# parallel-tests: some checks on console output about testsuite
+# advancement.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+XFAIL_TESTS = sub/xpass.test xfail.test error.test
+TESTS = $(XFAIL_TESTS) fail.test pass.test a/b/skip.test sub/error2.test
+pass.log: fail.log
+error.log: pass.log
+sub/xpass.log: error.log
+sub/error2.log: xfail.log
+a/b/skip.log: sub/error2.log
+END
+
+cat > exp <<'END'
+FAIL: fail.test
+PASS: pass.test
+ERROR: error.test
+XPASS: sub/xpass.test
+XFAIL: xfail.test
+ERROR: sub/error2.test
+SKIP: a/b/skip.test
+END
+
+mkdir sub a a/b
+
+cat > pass.test << 'END'
+#!/bin/sh
+exit 0
+END
+cp pass.test sub/xpass.test
+
+cat > fail.test << 'END'
+#!/bin/sh
+exit 1
+END
+
+cat > xfail.test << 'END'
+#!/bin/sh
+# The sleep should ensure expected execution order of tests
+# even when make is run in parallel mode.
+# FIXME: quotes below required by maintainer-check
+sleep '10'
+exit 1
+END
+
+cat > error.test << 'END'
+#!/bin/sh
+exit 99
+END
+cp error.test sub/error2.test
+
+cat > a/b/skip.test << 'END'
+#!/bin/sh
+exit 77
+END
+
+chmod a+x pass.test fail.test xfail.test sub/xpass.test \
+          a/b/skip.test error.test sub/error2.test
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+for vpath in : false; do
+  if $vpath; then
+    mkdir build
+    cd build
+    srcdir=..
+  else
+    srcdir=.
+  fi
+  $srcdir/configure
+  $MAKE check >stdout && { cat stdout; Exit 1; }
+  cat stdout
+  LC_ALL=C grep '^[A-Z][A-Z]*:' stdout > got
+  cat got
+  diff $srcdir/exp got
+  cd $srcdir
+done
+
+:
diff --git a/tests/parallel-tests-exit-statuses.test 
b/tests/parallel-tests-exit-statuses.test
new file mode 100755
index 0000000..531526a
--- /dev/null
+++ b/tests/parallel-tests-exit-statuses.test
@@ -0,0 +1,108 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check parallel-tests features: normal and special exit statuses
+# in the test scripts.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+# $failure_statuses should be defined to the list of all integers between
+# 1 and 255 (inclusive), excluded 77 and 99.
+# Let's use `seq' if available, it's faster than the loop.
+failure_statuses=`seq 1 255 \
+  || { i=1; while test $i -le 255; do echo $i; i=\`expr $i + 1\`; done; }`
+failure_statuses=`
+  for i in $failure_statuses; do
+    test $i -eq 77 || test $i -eq 99 || echo $i
+  done | tr "$nl" ' '`
+# For debugging.
+echo "failure_statuses: $failure_statuses"
+# Sanity check.
+test `for st in $failure_statuses; do echo $st; done | wc -l` -eq 253 \
+  || fatal_ "initializing list of exit statuses for simple failures"
+
+cat > Makefile.am <<END
+LOG_COMPILER = ./do-exit
+fail_tests = $failure_statuses
+TESTS = 0 77 99 $failure_statuses
+\$(TESTS):
+END
+
+cat > do-exit <<'END'
+#!/bin/sh
+echo "$0: $1"
+case $1 in
+  [0-9]|[0-9][0-9]|[0-9][0-9][0-9]) st=$1;;
+  */[0-9]|*/[0-9][0-9]|*/[0-9][0-9][0-9]) st=`echo x"$1" | sed 's|.*/||'`;;
+  *) st=99;;
+esac
+exit $st
+END
+chmod a+x do-exit
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+{
+  echo PASS: 0
+  echo SKIP: 77
+  echo ERROR: 99
+  for st in $failure_statuses; do
+    echo "FAIL: $st"
+  done
+} | LC_ALL=C sort > exp-fail
+
+sed 's/^FAIL:/XFAIL:/' exp-fail | LC_ALL=C sort > exp-xfail-1
+sed '/^ERROR:/d' exp-xfail-1 > exp-xfail-2
+
+sort exp-fail
+sort exp-xfail-1
+sort exp-xfail-2
+
+./configure
+
+st=1
+$MAKE check >stdout && st=0
+cat stdout
+cat test-suite.log
+test $st -gt 0 || Exit 1
+LC_ALL=C grep '^[A-Z][A-Z]*:' stdout | LC_ALL=C sort > got-fail
+diff exp-fail got-fail
+
+st=1
+XFAIL_TESTS="$failure_statuses 99" $MAKE -e check >stdout && st=0
+cat stdout
+cat test-suite.log
+test $st -gt 0 || Exit 1
+LC_ALL=C grep '^[A-Z][A-Z]*:' stdout | LC_ALL=C sort > got-xfail-1
+diff exp-xfail-1 got-xfail-1
+
+st=0
+XFAIL_TESTS="$failure_statuses" TESTS="0 77 $failure_statuses" \
+  $MAKE -e check >stdout || st=$?
+cat stdout
+cat test-suite.log
+test $st -eq 0 || Exit 1
+LC_ALL=C grep '^[A-Z][A-Z]*:' stdout | LC_ALL=C sort > got-xfail-2
+diff exp-xfail-2 got-xfail-2
+
+:
diff --git a/tests/parallel-tests-harderror.test 
b/tests/parallel-tests-harderror.test
index 8680e29..f30fd52 100755
--- a/tests/parallel-tests-harderror.test
+++ b/tests/parallel-tests-harderror.test
@@ -69,13 +69,13 @@ DISABLE_HARD_ERRORS=x $MAKE check
 
 $MAKE check DISABLE_HARD_ERRORS='' && Exit 1
 cat test-suite.log
-grep '^FAIL: foo\.test .*exit.*99' test-suite.log
+grep '^ERROR: foo\.test .*exit.*99' test-suite.log
 
 cd sub
 # The `-e' is wanted here.
 DISABLE_HARD_ERRORS='' $MAKE -e check && Exit 1
 cat test-suite.log
-grep '^FAIL: bar\.test .*exit.*99' test-suite.log
+grep '^ERROR: bar\.test .*exit.*99' test-suite.log
 cd ..
 
 # Check the distributions.
@@ -90,7 +90,7 @@ $AUTOMAKE Makefile
 ./config.status Makefile
 VERBOSE=yes $MAKE check && Exit 1
 grep '^FAIL' test-suite.log && Exit 1
-grep '^FAIL: bar\.test .*exit.*99' sub/test-suite.log
+grep '^ERROR: bar\.test .*exit.*99' sub/test-suite.log
 
 echo 'DISABLE_HARD_ERRORS = zardoz' >> sub/Makefile
 VERBOSE=yes $MAKE check
diff --git a/tests/parallel-tests-no-spurious-summary.test 
b/tests/parallel-tests-no-spurious-summary.test
index 43c54a5..8b9547a 100755
--- a/tests/parallel-tests-no-spurious-summary.test
+++ b/tests/parallel-tests-no-spurious-summary.test
@@ -32,10 +32,12 @@ cat > foo.test <<'END'
 #! /bin/sh
 echo :test-result:XFAIL
 echo :test-result: SKIP
+echo :test-result:ERROR
 exit 0
 END
 cat > bar.test <<'END'
 #! /bin/sh
+echo :test-result: ERROR
 echo :test-result:FAIL
 echo :test-result: XPASS
 exit 0
diff --git a/tests/parallel-tests.test b/tests/parallel-tests.test
index f046c3f..95f78e7 100755
--- a/tests/parallel-tests.test
+++ b/tests/parallel-tests.test
@@ -68,10 +68,13 @@ $AUTOMAKE -a
 
 $MAKE check >stdout && { cat stdout; Exit 1; }
 cat stdout
-# There should be two errors: bar.test is a hard error.
-test `grep -c '^FAIL' stdout` -eq 2
+# There should be one failure and one hard error.
+test `grep -c '^FAIL:' stdout` -eq 1
+test `grep -c '^ERROR:' stdout` -eq 1
 test -f mylog.log
-test `grep -c '^FAIL' mylog.log` -eq 2
+cat mylog.log
+test `grep -c '^FAIL:' mylog.log` -eq 1
+test `grep -c '^ERROR:' mylog.log` -eq 1
 test -f baz.log
 test -f bar.log
 test -f foo.log
diff --git a/tests/test-driver-custom-multitest-recheck.test 
b/tests/test-driver-custom-multitest-recheck.test
index e0f010f..c178fa0 100755
--- a/tests/test-driver-custom-multitest-recheck.test
+++ b/tests/test-driver-custom-multitest-recheck.test
@@ -49,7 +49,7 @@ echo PASS:
 if test -f b.ok; then
   echo PASS:
 else
-  echo FAIL:
+  echo ERROR:
 fi
 : > b.run
 END
@@ -97,14 +97,15 @@ do_recheck ()
 
 do_count ()
 {
-  pass=ERR fail=ERR xpass=ERR xfail=ERR skip=ERR
+  pass=ERR fail=ERR xpass=ERR xfail=ERR skip=ERR error=ERR
   eval "$@"
-  $EGREP '(PASS|FAIL|XPASS|XFAIL|SKIP)' stdout || : # For debugging.
+  $EGREP '(PASS|FAIL|XPASS|XFAIL|SKIP|ERROR)' stdout || : # For debugging.
   test `grep -c '^PASS:' stdout` -eq $pass
   test `grep -c '^FAIL:' stdout` -eq $fail
   test `grep -c '^XPASS:' stdout` -eq $xpass
   test `grep -c '^XFAIL:' stdout` -eq $xfail
   test `grep -c '^SKIP:' stdout` -eq $skip
+  test `grep -c '^ERROR:' stdout` -eq $error
 }
 
 for vpath in : false; do
@@ -129,7 +130,7 @@ for vpath in : false; do
   test ! -r c.log
   test ! -r d.run
   test ! -r d.log
-  do_count pass=0 fail=0 xpass=0 xfail=0 skip=0
+  do_count pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
 
   : Run the tests for the first time.
   $MAKE check >stdout && { cat stdout; Exit 1; }
@@ -140,7 +141,7 @@ for vpath in : false; do
   test -f b.run
   test -f c.run
   test -f d.run
-  do_count pass=3 fail=3 xpass=1 xfail=1 skip=1
+  do_count pass=3 fail=2 xpass=1 xfail=1 skip=1 error=1
 
   : Let us make b.test pass.
   echo OK > b.ok
@@ -151,7 +152,7 @@ for vpath in : false; do
   test -f b.run
   test -f c.run
   test -f d.run
-  do_count pass=2 fail=2 xpass=1 xfail=1 skip=1
+  do_count pass=2 fail=2 xpass=1 xfail=1 skip=1 error=0
 
   : Let us make the first part of c.test pass.
   echo OK > c.pass
@@ -160,7 +161,7 @@ for vpath in : false; do
   test ! -r b.run
   test -f c.run
   test -f d.run
-  do_count pass=1 fail=1 xpass=1 xfail=1 skip=1
+  do_count pass=1 fail=1 xpass=1 xfail=1 skip=1 error=0
 
   : Let us make also the second part of c.test pass.
   echo KO > c.xfail
@@ -169,7 +170,7 @@ for vpath in : false; do
   test ! -r b.run
   test -f c.run
   test -f d.run
-  do_count pass=1 fail=1 xpass=0 xfail=2 skip=1
+  do_count pass=1 fail=1 xpass=0 xfail=2 skip=1 error=0
 
   : Nothing changed, so only d.test should be run.
   for i in 1 2; do
@@ -178,7 +179,7 @@ for vpath in : false; do
     test ! -r b.run
     test ! -r c.run
     test -f d.run
-    do_count pass=0 fail=1 xpass=0 xfail=0 skip=1
+    do_count pass=0 fail=1 xpass=0 xfail=0 skip=1 error=0
   done
 
   : Let us make d.test run more testcases, and experience _more_ failures.
@@ -190,14 +191,16 @@ for vpath in : false; do
     echo XPASS: xp
     echo FAIL: f 3
     echo FAIL: f 4
+    echo ERROR: e 1
     echo PASS: p 2
+    echo ERROR: e 2
 END
   do_recheck --fail
   test ! -r a.run
   test ! -r b.run
   test ! -r c.run
   test -f d.run
-  do_count pass=2 fail=4 xpass=1 xfail=0 skip=2
+  do_count pass=2 fail=4 xpass=1 xfail=0 skip=2 error=2
 
   : Let us finally make d.test pass.
   echo : > d.extra
@@ -206,7 +209,7 @@ END
   test ! -r b.run
   test ! -r c.run
   test -f d.run
-  do_count pass=0 fail=0 xpass=0 xfail=0 skip=1
+  do_count pass=0 fail=0 xpass=0 xfail=0 skip=1 error=0
 
   : All tests have been successful or skipped, nothing should be re-run.
   do_recheck --pass
@@ -214,7 +217,7 @@ END
   test ! -r b.run
   test ! -r c.run
   test ! -r d.run
-  do_count pass=0 fail=0 xpass=0 xfail=0 skip=0
+  do_count pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
 
   cd $srcdir
 
diff --git a/tests/test-driver-custom-multitest-recheck2.test 
b/tests/test-driver-custom-multitest-recheck2.test
index a18bdd8..d2e979e 100755
--- a/tests/test-driver-custom-multitest-recheck2.test
+++ b/tests/test-driver-custom-multitest-recheck2.test
@@ -56,7 +56,13 @@ END
 
 cat > c.test << 'END'
 #! /bin/sh
-echo XPASS: xp
+if test -f c.err; then
+  echo ERROR: xxx
+elif test -f c.ok; then
+  echo PASS: ok
+else
+  echo XPASS: xp
+fi
 : > c.run
 END
 
@@ -68,14 +74,15 @@ $AUTOMAKE
 
 do_count ()
 {
-  pass=ERR fail=ERR xpass=ERR xfail=ERR skip=ERR
+  pass=ERR fail=ERR xpass=ERR xfail=ERR skip=ERR error=ERR
   eval "$@"
-  $EGREP '(PASS|FAIL|XPASS|XFAIL|SKIP)' stdout || : # For debugging.
+  $EGREP '(PASS|FAIL|XPASS|XFAIL|SKIP|ERROR)' stdout || : # For debugging.
   test `grep -c '^PASS:' stdout` -eq $pass
   test `grep -c '^FAIL:' stdout` -eq $fail
   test `grep -c '^XPASS:' stdout` -eq $xpass
   test `grep -c '^XFAIL:' stdout` -eq $xfail
   test `grep -c '^SKIP:' stdout` -eq $skip
+  test `grep -c '^ERROR:' stdout` -eq $error
 }
 
 for vpath in : false; do
@@ -96,7 +103,7 @@ for vpath in : false; do
   test -f a.run
   test -f b.run
   test -f c.run
-  do_count pass=2 fail=1 xpass=1 xfail=0 skip=1
+  do_count pass=2 fail=1 xpass=1 xfail=0 skip=1 error=0
 
   rm -f *.run
 
@@ -104,7 +111,7 @@ for vpath in : false; do
   for var in TESTS TEST_LOGS; do
     env "$var=" $MAKE -e recheck >stdout || { cat stdout; Exit 1; }
     cat stdout
-    do_count pass=0 fail=0 xpass=0 xfail=0 skip=0
+    do_count pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
     test ! -r a.run
     test ! -r b.run
     test ! -r c.run
@@ -115,7 +122,7 @@ for vpath in : false; do
   env TESTS=a.test $MAKE -e recheck >stdout \
     || { cat stdout; Exit 1; }
   cat stdout
-  do_count pass=0 fail=0 xpass=0 xfail=0 skip=0
+  do_count pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
   test ! -r a.run
   test ! -r b.run
   test ! -r c.run
@@ -128,7 +135,7 @@ for vpath in : false; do
   test ! -r a.run
   test -f b.run
   test ! -r c.run
-  do_count pass=0 fail=0 xpass=0 xfail=1 skip=1
+  do_count pass=0 fail=0 xpass=0 xfail=1 skip=1 error=0
 
   rm -f *.run
 
@@ -136,35 +143,54 @@ for vpath in : false; do
   TEST_LOGS=b.log $MAKE -e recheck >stdout \
     || { cat stdout; Exit 1; }
   cat stdout
-  do_count pass=0 fail=0 xpass=0 xfail=0 skip=0
+  do_count pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
   test ! -r a.run
   test ! -r b.run
   test ! -r c.run
   TESTS='a.test b.test' $MAKE -e recheck >stdout \
     || { cat stdout; Exit 1; }
   cat stdout
-  do_count pass=0 fail=0 xpass=0 xfail=0 skip=0
+  do_count pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
   test ! -r a.run
   test ! -r b.run
   test ! -r c.run
 
-  # An XPASS should count a failure.
+  : No need to re-run a.test anymore, but c.test should be rerun,
+  : as it contained an XPASS.  And this time, make it fail with
+  : an hard error.
+  : > c.err
   env TEST_LOGS='a.log c.log' $MAKE -e recheck >stdout \
     && { cat stdout; Exit 1; }
   cat stdout
-  do_count pass=0 fail=0 xpass=1 xfail=0 skip=0
+  do_count pass=0 fail=0 xpass=0 xfail=0 skip=0 error=1
   test ! -r a.run
   test ! -r b.run
   test -f c.run
-  rm -f *.run
-  env TESTS='c.test b.test' $MAKE -e recheck >stdout \
-    && { cat stdout; Exit 1; }
+
+  rm -f *.run *.err
+
+  : c.test contained and hard error the last time, so it should be re-run.
+  : This time, make it pass
+  : > c.ok
+  env TESTS='c.test a.test' $MAKE -e recheck >stdout \
+    || { cat stdout; Exit 1; }
   cat stdout
-  do_count pass=0 fail=0 xpass=1 xfail=0 skip=0
+  do_count pass=1 fail=0 xpass=0 xfail=0 skip=0 error=0
   test ! -r a.run
   test ! -r b.run
   test -f c.run
 
+  rm -f *.run *.err *.ok
+
+  : Nothing should be rerun anymore, as all tests have been evenatually
+  : succesfull.
+  $MAKE recheck >stdout || { cat stdout; Exit 1; }
+  cat stdout
+  do_count pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
+  test ! -r a.run
+  test ! -r b.run
+  test ! -r c.run
+
   cd $srcdir
 
 done
diff --git a/tests/test-driver-custom-multitest.test 
b/tests/test-driver-custom-multitest.test
index a1c1010..2366b5b 100755
--- a/tests/test-driver-custom-multitest.test
+++ b/tests/test-driver-custom-multitest.test
@@ -40,7 +40,7 @@ TESTS = \
   pass-fail.t \
   pass4-skip.t \
   pass3-skip2-xfail.t \
-  pass-xpass-fail-xfail-skip.t
+  pass-xpass-fail-xfail-skip-error.t
 END
 
 expected_pass=10
@@ -48,6 +48,7 @@ expected_fail=5
 expected_skip=4
 expected_xfail=2
 expected_xpass=1
+expected_error=1
 
 cat > pass.t << 'END'
 echo %% pass %%
@@ -99,13 +100,14 @@ mv || echo "XFAIL: mv cannot read users' mind yet"
 exit 127
 END
 
-cat > pass-xpass-fail-xfail-skip.t << 'END'
+cat > pass-xpass-fail-xfail-skip-error.t << 'END'
 echo PASS:
 echo FAIL:
 echo XFAIL:
 echo XPASS:
 echo SKIP:
-echo %% pass-xpass-fail-xfail-skip %%
+echo ERROR:
+echo %% pass-xpass-fail-xfail-skip-error %%
 END
 
 chmod a+x *.t
@@ -139,25 +141,28 @@ for vpath in : false; do
   cat pass-fail.log
   cat pass4-skip.log
   cat pass3-skip2-xfail.log
-  cat pass-xpass-fail-xfail-skip.log
+  cat pass-xpass-fail-xfail-skip-error.log
   # For debugging.
-  $EGREP '(PASS|FAIL|XPASS|XFAIL|SKIP)' stdout
+  $EGREP '(PASS|FAIL|XPASS|XFAIL|SKIP|ERROR)' stdout
 
   test `grep -c '^PASS:' stdout` -eq $expected_pass
   test `grep -c '^FAIL:' stdout` -eq $expected_fail
   test `grep -c '^XPASS:' stdout` -eq $expected_xpass
   test `grep -c '^XFAIL:' stdout` -eq $expected_xfail
   test `grep -c '^SKIP:' stdout` -eq $expected_skip
+  test `grep -c '^ERROR:' stdout` -eq $expected_error
 
-  grep  '^PASS: pass-xpass-fail-xfail-skip.t\, testcase 1' stdout
-  grep  '^FAIL: pass-xpass-fail-xfail-skip\.t, testcase 2' stdout
-  grep '^XFAIL: pass-xpass-fail-xfail-skip\.t, testcase 3' stdout
-  grep '^XPASS: pass-xpass-fail-xfail-skip\.t, testcase 4' stdout
-  grep  '^SKIP: pass-xpass-fail-xfail-skip\.t, testcase 5' stdout
+  tst=pass-xpass-fail-xfail-skip-error
+  grep  "^PASS: $tst\.t, testcase 1" stdout
+  grep  "^FAIL: $tst\.t, testcase 2" stdout
+  grep "^XFAIL: $tst\.t, testcase 3" stdout
+  grep "^XPASS: $tst\.t, testcase 4" stdout
+  grep  "^SKIP: $tst\.t, testcase 5" stdout
+  grep "^ERROR: $tst\.t, testcase 6" stdout
 
   # Check testsuite summary printed on console.
   sed -e 's/[()]/ /g' -e 's/^/ /' stdout > t
-  grep ' 6 of 18 ' t
+  grep ' 7 of 19 ' t
   grep ' 1 unexpected pass' t
   grep ' 4 test.* not run' t
 
@@ -170,7 +175,7 @@ for vpath in : false; do
   grep '%% fail %%' test-suite.log
   grep '%% fail2 %%' test-suite.log
   grep '%% pass-fail %%' test-suite.log
-  grep '%% pass-xpass-fail-xfail-skip %%' test-suite.log
+  grep '%% pass-xpass-fail-xfail-skip-error %%' test-suite.log
   test `grep -c '%% ' test-suite.log` -eq 4
 
   TESTS='pass.t pass3-skip2-xfail.t' $MAKE -e check >stdout \
@@ -178,11 +183,11 @@ for vpath in : false; do
   cat test-suite.log
   cat stdout
   # For debugging.
-  $EGREP '(PASS|FAIL|XPASS|XFAIL|SKIP)' stdout
+  $EGREP '(PASS|FAIL|XPASS|XFAIL|SKIP|ERROR)' stdout
   test `grep -c '^PASS:' stdout` -eq 4
   test `grep -c '^SKIP:' stdout` -eq 2
   test `grep -c '^XFAIL:' stdout` -eq 1
-  $EGREP '^(FAIL|XPASS)' stdout && Exit 1
+  $EGREP '^(FAIL|XPASS|ERROR)' stdout && Exit 1
 
   cd $srcdir
 
diff --git a/tests/test-driver-custom-no-html.test 
b/tests/test-driver-custom-no-html.test
index 9aceae6..683659b 100755
--- a/tests/test-driver-custom-no-html.test
+++ b/tests/test-driver-custom-no-html.test
@@ -62,6 +62,6 @@ $FGREP 'dummy title' test-suite.log
 # Sanity check: trying to produce HTML output should fail.
 $MAKE check-html >output 2>&1 && { cat output; Exit 1; }
 cat output
-$EGREP 'SEVERE|ERROR' output
+$FGREP SEVERE output
 
 :
diff --git a/tests/test-driver-end-test-results.test 
b/tests/test-driver-end-test-results.test
index 97c1f29..03af51b 100755
--- a/tests/test-driver-end-test-results.test
+++ b/tests/test-driver-end-test-results.test
@@ -57,6 +57,7 @@ PASS: b.test
 :test-result:PASS
 :test-result:END
 :test-result:FAIL
+:test-result:ERROR
 END
 
 cat > c.test <<END
diff --git a/tests/test-driver-global-log.test 
b/tests/test-driver-global-log.test
index cdaaff6..35b6fe1 100755
--- a/tests/test-driver-global-log.test
+++ b/tests/test-driver-global-log.test
@@ -44,12 +44,13 @@ END
 echo SKIP: skip.test > skip.test
 echo FAIL: fail.test > fail.test
 echo XPASS: xpass.test > xpass.test
+echo ERROR: error.test > error.test
 echo :test-result: PASS > fake-pass.test
 echo "$tab $tab$tab" > empty.test
 
 cat > Makefile.am << 'END'
 TEST_LOG_DRIVER = ./dummy-driver
-TESTS = pass.test skip.test fail.test xfail.test xpass.test \
+TESTS = pass.test skip.test fail.test xfail.test xpass.test error.test \
         fake-pass.test empty.test
 END
 
@@ -82,6 +83,7 @@ cat test-suite.log
 grep '^SKIP: skip\.test$' test-suite.log
 grep '^FAIL: fail.test$' test-suite.log
 grep '^XPASS: xpass.test$' test-suite.log
+grep '^ERROR: error.test$' test-suite.log
 grep '^:test-result: PASS$' test-suite.log
 grep "^$tab $tab$tab$" test-suite.log
 $EGREP 'not seen' test-suite.log && Exit 1
diff --git a/tests/test-driver-recheck.test b/tests/test-driver-recheck.test
index 2ae75b9..f9482b1 100755
--- a/tests/test-driver-recheck.test
+++ b/tests/test-driver-recheck.test
@@ -54,7 +54,7 @@ END
   rechecked="$rechecked BAD-$R-1 BAD-$R-2 BAD-$R-3 BAD-$R-4"
 done
 
-for R in FAIL XPASS UNKNOWN; do
+for R in FAIL XPASS ERROR UNKNOWN; do
   echo $R: > $R-1
   echo $R:foo > $R-2
   echo $R: bar baz > $R-3
diff --git a/tests/trivial-test-driver b/tests/trivial-test-driver
index f06eb1c..113e158 100644
--- a/tests/trivial-test-driver
+++ b/tests/trivial-test-driver
@@ -20,9 +20,10 @@
 # test results per test script.
 #
 # The exit status of the wrapped script is ignored.  Lines in its stdout
-# and stderr beginning with `PASS', `FAIL', `XFAIL', `XPASS' or `SKIP'
-# count as a test case result with the obviously-corresponding outcome.
-# Every other line is ignored for what concerns the testsuite outcome.
+# and stderr beginning with `PASS', `FAIL', `XFAIL', `XPASS', `SKIP' or
+# `ERROR' count as a test case result with the obviously-corresponding
+# outcome.  Every other line is ignored for what concerns the testsuite
+# outcome.
 #
 # This script is used at least by the `driver-custom-multitest*.test'
 # tests.
@@ -61,10 +62,10 @@ tmp_res=$log_file-res.tmp
   : > $tmp_res
   while read line; do
     case $line in
-      PASS:*|FAIL:*|XPASS:*|XFAIL:*|SKIP:*)
+      PASS:*|FAIL:*|XPASS:*|XFAIL:*|SKIP:*|ERROR:*)
         i=`expr $i + 1`
         result=`LC_ALL=C expr "$line" : '\([A-Z]*\):.*'`
-        case $result in FAIL|XPASS) st=1;; esac
+        case $result in FAIL|XPASS|ERROR) st=1;; esac
         # Output testcase result to console.
         echo "$result: $test_name, testcase $i"
         # Register testcase outcome for the log file.
-- 
1.7.2.3

[Prev in Thread]

Current Thread

[Next in Thread]

[PATCH] parallel-tests: new recognized test result 'ERROR', Stefano Lattarini <=
- Re: [PATCH] parallel-tests: new recognized test result 'ERROR', Ralf Wildenhues, 2011/06/30

Prev by Date: [FYI] New public temporary branch, about improved handling and parsing of test results
Next by Date: [MERGE REQUEST] merge patches for AM_TESTSUITE_ENVIRONMENT into maint
Previous by thread: [PATCH] {master} coverage: new test on parallel-tests TESTS runtime overriding
Next by thread: Re: [PATCH] parallel-tests: new recognized test result 'ERROR'
Index(es):
- Date
- Thread