[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[FYI 2/2] {test-protocols} tap: non-zero exit status after "Bail out!" s
From: |
Stefano Lattarini |
Subject: |
[FYI 2/2] {test-protocols} tap: non-zero exit status after "Bail out!" should not be reported |
Date: |
Fri, 12 Aug 2011 10:51:50 +0200 |
This new behaviour is more consistent with that of the `prove'
utility. Also, experience with the use of TAP in the Automake's
own test suite has shown that reporting non-zero exit status from
a script after it has issued a "Bail out!" directive is mostly
redundant, more introducing noise rather than conveying useful
information.
* lib/tap-driver (finish): Inline the part of it that was getting
the script exit status ...
(main): ... in here.
* tests/tap-bailout-badexit.test: New test.
* tests/Makefile.am (tap_with_common_setup_tests): Update.
---
ChangeLog | 15 +++++++++
lib/tap-driver | 8 ++--
tests/Makefile.am | 1 +
tests/tap-bailout-badexit.test | 65 ++++++++++++++++++++++++++++++++++++++++
4 files changed, 85 insertions(+), 4 deletions(-)
create mode 100755 tests/tap-bailout-badexit.test
diff --git a/ChangeLog b/ChangeLog
index 4ea0ee7..9a1d601 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,18 @@
+2011-08-12 Stefano Lattarini <address@hidden>
+
+ tap: non-zero exit status after "Bail out!" should not be reported
+ This new behaviour is more consistent with that of the `prove'
+ utility. Also, experience with the use of TAP in the Automake's
+ own test suite has shown that reporting non-zero exit status from
+ a script after it has issued a "Bail out!" directive is mostly
+ redundant, more introducing noise rather than conveying useful
+ information.
+ * lib/tap-driver (finish): Inline the part of it that was getting
+ the script exit status ...
+ (main): ... in here.
+ * tests/tap-bailout-badexit.test: New test.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
2011-08-11 Stefano Lattarini <address@hidden>
tap: a "plan with SKIP" after test results is an error
diff --git a/lib/tap-driver b/lib/tap-driver
index a02557d..b7a8ffb 100755
--- a/lib/tap-driver
+++ b/lib/tap-driver
@@ -235,10 +235,6 @@ sub get_test_exit_message ()
sub finish ()
{
- if (!$cfg{"ignore-exit"} and my $msg = get_test_exit_message)
- {
- testsuite_error $msg;
- }
write_test_results;
close LOG or die "closing $log_file: $!\n";
exit 0;
@@ -444,6 +440,10 @@ sub main (@)
testsuite_error (sprintf "too %s tests run (expected %d, got %d)",
$bad_amount, $planned, $run);
}
+ if (!$cfg{"ignore-exit"} and my $msg = get_test_exit_message)
+ {
+ testsuite_error $msg;
+ }
finish;
}
diff --git a/tests/Makefile.am b/tests/Makefile.am
index ae76516..cf78cb6 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -1144,6 +1144,7 @@ testsuite-summary-count-many.log:
extract-testsuite-summary
tap_with_common_setup_tests = \
tap-autonumber.test \
tap-bailout.test \
+tap-bailout-badexit.test \
tap-color.test \
tap-deps.test \
tap-diagnostic.test \
diff --git a/tests/tap-bailout-badexit.test b/tests/tap-bailout-badexit.test
new file mode 100755
index 0000000..059394c
--- /dev/null
+++ b/tests/tap-bailout-badexit.test
@@ -0,0 +1,65 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Basic TAP test protocol support:
+# - A "Bail out!" directive causes the driver to ignore the exit
+# status of the test script.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+tests='exit.test exit127.test sighup.test sigterm.test'
+
+cat > Makefile.am <<END
+TESTS = $tests
+END
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > exit.test << 'END'
+#!/bin/sh
+echo 'Bail out!'
+exit 1
+END
+
+cat > exit127.test << 'END'
+#!/bin/sh
+echo 'Bail out!'
+exit 127
+END
+
+cat > sighup.test << 'END'
+#!/bin/sh
+echo 'Bail out!'
+kill -1 $$
+END
+
+cat > sigterm.test << 'END'
+#!/bin/sh
+echo 'Bail out!'
+kill -15 $$
+END
+
+chmod a+x $tests
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=4 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=4
+for tst in $tests; do grep "^ERROR: $tst - Bail out!" stdout; done
+$EGREP "ERROR: .*(exit.*status|terminat.*signal)" stdout && Exit 1
+
+:
--
1.7.2.3