[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Automake-commit] [SCM] GNU Automake branch, test-protocols, updated. v1
|
From: |
Stefano Lattarini |
|
Subject: |
[Automake-commit] [SCM] GNU Automake branch, test-protocols, updated. v1.11-1046-gc9ca5f6 |
|
Date: |
Sun, 21 Aug 2011 21:54:02 +0000 |
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU Automake".
http://git.sv.gnu.org/gitweb/?p=automake.git;a=commitdiff;h=c9ca5f61d87cb0578d32981b2ee32118d934ce21
The branch, test-protocols has been updated
via c9ca5f61d87cb0578d32981b2ee32118d934ce21 (commit)
via 1bc8f828ae07b6a3537ef0ca46cd698ba99659e2 (commit)
via fdfe9a1154b9945c71ac92b5c16ffed21e89f2b4 (commit)
via 82bb64f1d21f6993341a725e75aa5aa00c3e00a1 (commit)
via 9f833f79c912cef51f4f2f71f2183574cdc43ee4 (commit)
from 28964e4787b17f0211f9b2aab2776cee163fdb30 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit c9ca5f61d87cb0578d32981b2ee32118d934ce21
Author: Stefano Lattarini <address@hidden>
Date: Sun Aug 21 23:15:45 2011 +0200
tap/awk: correctly extract message from SKIP plans
* lib/tap-driver.sh (/^1\.\.0+[ \t]*#/): Add a forgotten `?' to
the regex used to extract the skip message from a "1..0" TAP plan.
* tests/planskip.test: Check for the just-fixed bug.
* tests/tap-planskip-unplanned-corner.test: Don't be too strict
w.r.t. the formatting of the "skip whole test" message, to avoid
failing due to bugs irrelevant for this test (like the one just
fixed).
commit 1bc8f828ae07b6a3537ef0ca46cd698ba99659e2
Author: Stefano Lattarini <address@hidden>
Date: Sun Aug 21 20:09:34 2011 +0200
tap/awk: support colorized output
* lib/tap-driver.sh: Pass definition of colors to the BEGIN block
of the awk script, using ...
($init_colors): ... this new variable, which supersedes ...
($red, $grn, $lgn, $blu, $mgn, $std): ... these variables, which
have been consequently removed.
($color_tests): We don't have to pass this anymore to the awk
script, so do not convert it to a numeric boolean.
(decorate_result): Implement for real.
With this change, the test 'tap-color.test' now passes.
commit fdfe9a1154b9945c71ac92b5c16ffed21e89f2b4
Author: Stefano Lattarini <address@hidden>
Date: Sun Aug 21 18:15:35 2011 +0200
tap: real (but still incomplete) awk implementation of TAP driver
* lib/tap-driver.sh : Add an incomplete, but mostly working,
implementation of a TAP parser and driver in awk. It doesn't yet
support colorized output, fetching of exit status from test
programs, nor a way to escape TAP directives in TAP result lines,
but passes all the tests of TAP support in the automake testsuite,
apart from the following ones:
- tap-color.test
- tap-escape-directive.test
- tap-exit.test
- tap-missing-plan-and-bad-exit.test
- tap-passthrough-exit.test
- tap-planskip-badexit.test
- tap-planskip-unplanned-corner.test
- tap-signal.test
Tested on Debian GNU/Linux with GNU awk 3.1.7 and 3.0.2, "original
awk" 2010-05-23-1, and mawk 1.3.3-15 (with which also the test
`tap-realtime.test' fails, in addition to those listed above).
commit 82bb64f1d21f6993341a725e75aa5aa00c3e00a1
Author: Stefano Lattarini <address@hidden>
Date: Sun Aug 21 18:56:21 2011 +0200
testsuite: fix spurious failure due to Solaris 'wc' incompatibility
* tests/tap-number-wordboundary.test ($planned): Strip any extra
whitespace from the output of `wc', to cater to Solaris 10.
commit 9f833f79c912cef51f4f2f71f2183574cdc43ee4
Author: Stefano Lattarini <address@hidden>
Date: Sun Aug 21 15:33:03 2011 +0200
coverage: keyword "SKIP" in TAP plan is case-insensitive
* tests/tap-planskip-case-insensitive.test: New test.
* tests/tap-planskip.test: Remove now-duplicated checks.
* tests/Makefile.am (tap_with_common_setup_tests): Update.
-----------------------------------------------------------------------
Summary of changes:
ChangeLog | 58 +++
lib/tap-driver.sh | 514 ++++++++++++++++++--
tests/Makefile.am | 1 +
tests/Makefile.in | 1 +
tests/tap-number-wordboundary.test | 3 +-
...kip.test => tap-planskip-case-insensitive.test} | 25 +-
tests/tap-planskip-unplanned-corner.test | 3 +-
tests/tap-planskip.test | 8 +-
8 files changed, 570 insertions(+), 43 deletions(-)
copy tests/{tap-msg0-planskip.test => tap-planskip-case-insensitive.test} (62%)
diff --git a/ChangeLog b/ChangeLog
index 810cee0..738e10f 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,63 @@
2011-08-21 Stefano Lattarini <address@hidden>
+ tap/awk: correctly extract message from SKIP plans
+ * lib/tap-driver.sh (/^1\.\.0+[ \t]*#/): Add a forgotten `?' to
+ the regex used to extract the skip message from a "1..0" TAP plan.
+ * tests/planskip.test: Check for the just-fixed bug.
+ * tests/tap-planskip-unplanned-corner.test: Don't be too strict
+ w.r.t. the formatting of the "skip whole test" message, to avoid
+ failing due to bugs irrelevant for this test (like the one just
+ fixed).
+
+2011-08-21 Stefano Lattarini <address@hidden>
+
+ tap/awk: support colorized output
+ * lib/tap-driver.sh: Pass definition of colors to the BEGIN block
+ of the awk script, using ...
+ ($init_colors): ... this new variable, which supersedes ...
+ ($red, $grn, $lgn, $blu, $mgn, $std): ... these variables, which
+ have been consequently removed.
+ ($color_tests): We don't have to pass this anymore to the awk
+ script, so do not convert it to a numeric boolean.
+ (decorate_result): Implement for real.
+ With this change, the test 'tap-color.test' now passes.
+
+2011-08-21 Stefano Lattarini <address@hidden>
+
+ tap: real (but still incomplete) awk implementation of TAP driver
+ * lib/tap-driver.sh: Add an incomplete, but mostly working,
+ implementation of a TAP parser and driver in awk. It doesn't yet
+ support colorized output, fetching of exit status from test
+ programs, nor a way to escape TAP directives in TAP result lines,
+ but passes all the tests of TAP support in the automake testsuite,
+ apart from the following ones:
+ - tap-color.test
+ - tap-escape-directive.test
+ - tap-exit.test
+ - tap-missing-plan-and-bad-exit.test
+ - tap-passthrough-exit.test
+ - tap-planskip-badexit.test
+ - tap-planskip-unplanned-corner.test
+ - tap-signal.test
+ Tested on Debian GNU/Linux with GNU awk 3.1.7 and 3.0.2, "original
+ awk" 2010-05-23-1, and mawk 1.3.3-15 (with which also the test
+ `tap-realtime.test' fails, in addition to those listed above).
+
+2011-08-21 Stefano Lattarini <address@hidden>
+
+ testsuite: fix spurious failure due to Solaris 'wc' incompatibility
+ * tests/tap-number-wordboundary.test ($planned): Strip any extra
+ whitespace from the output of `wc', to cater to Solaris 10.
+
+2011-08-21 Stefano Lattarini <address@hidden>
+
+ coverage: keyword "SKIP" in TAP plan is case-insensitive
+ * tests/tap-planskip-case-insensitive.test: New test.
+ * tests/tap-planskip.test: Remove now-duplicated checks.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-21 Stefano Lattarini <address@hidden>
+
coverage: ambiguous use of TAP "TODO"/"SKIP" directives
* tests/tap-ambiguous-directive.test: New test.
* tests/tap-todo-skip-together.test: Reference it in heading
diff --git a/lib/tap-driver.sh b/lib/tap-driver.sh
index 322e3c4..dd2b929 100755
--- a/lib/tap-driver.sh
+++ b/lib/tap-driver.sh
@@ -23,21 +23,23 @@
# bugs to <address@hidden> or send patches to
# <address@hidden>.
-scriptversion=2011-08-17.14; # UTC
+scriptversion=2011-08-21.21; # UTC
# Make unconditional expansion of undefined variables an error. This
# helps a lot in preventing typo-related bugs.
set -u
+me=tap-driver.sh
+
fatal ()
{
- echo "$0: fatal: $*" >&2
+ echo "$me: fatal: $*" >&2
exit 1
}
usage_error ()
{
- echo "$0: $*" >&2
+ echo "$me: $*" >&2
print_usage >&2
exit 2
}
@@ -46,11 +48,11 @@ print_usage ()
{
cat <<END
Usage:
- tap-driver --test-name=NAME --log-file=PATH --trs-file=PATH
- [--expect-failure={yes|no}] [--color-tests={yes|no}]
- [--enable-hard-errors={yes|no}] [--ignore-exit]
- [--diagnostic-string=STRING] [--merge|--no-merge]
- [--comments|--no-comments] [--] TEST-COMMAND
+ tap-driver.sh --test-name=NAME --log-file=PATH --trs-file=PATH
+ [--expect-failure={yes|no}] [--color-tests={yes|no}]
+ [--enable-hard-errors={yes|no}] [--ignore-exit]
+ [--diagnostic-string=STRING] [--merge|--no-merge]
+ [--comments|--no-comments] [--] TEST-COMMAND
The \`--test-name', \`--log-file' and \`--trs-file' options are mandatory.
END
}
@@ -60,28 +62,28 @@ END
test_name= # Used for reporting.
log_file= # Where to save the result and output of the test script.
trs_file= # Where to save the metadata of the test run.
-expect_failure=no
-color_tests=no
-merge=no
-ignore_exit=no
-comments=no
+expect_failure=0
+color_tests=0
+merge=0
+ignore_exit=0
+comments=0
diag_string='#'
while test $# -gt 0; do
case $1 in
--help) print_usage; exit $?;;
- --version) echo "tap-driver $scriptversion"; exit $?;;
+ --version) echo "$me $scriptversion"; exit $?;;
--test-name) test_name=$2; shift;;
--log-file) log_file=$2; shift;;
--trs-file) trs_file=$2; shift;;
--color-tests) color_tests=$2; shift;;
--expect-failure) expect_failure=$2; shift;;
--enable-hard-errors) shift;; # No-op.
- --merge) merge=yes;;
- --no-merge) merge=no;;
- --ignore-exit) ignore_exit=yes;;
- --comments) comments=yes;;
- --no-comments) comments=no;;
- --diag-string) diag_string=$2; shift;;
+ --merge) merge=1;;
+ --no-merge) merge=0;;
+ --ignore-exit) ignore_exit=1;;
+ --comments) comments=1;;
+ --no-comments) comments=0;;
+ --diagnostic-string) diag_string=$2; shift;;
--) shift; break;;
-*) usage_error "invalid option: '$1'";;
esac
@@ -96,21 +98,471 @@ case $expect_failure in
esac
if test $color_tests = yes; then
- red='[0;31m' # Red.
- grn='[0;32m' # Green.
- lgn='[1;32m' # Light green.
- blu='[1;34m' # Blue.
- mgn='[0;35m' # Magenta.
- std='[m' # No color.
+ init_colors='
+ color_map["red"]="[0;31m" # Red.
+ color_map["grn"]="[0;32m" # Green.
+ color_map["lgn"]="[1;32m" # Light green.
+ color_map["blu"]="[1;34m" # Blue.
+ color_map["mgn"]="[0;35m" # Magenta.
+ color_map["std"]="[m" # No color.
+ color_for_result["ERROR"] = "mgn"
+ color_for_result["PASS"] = "grn"
+ color_for_result["XPASS"] = "red"
+ color_for_result["FAIL"] = "red"
+ color_for_result["XFAIL"] = "lgn"
+ color_for_result["SKIP"] = "blu"'
else
- red= grn= lgn= blu= mgn= std=
+ init_colors=''
fi
-# TODO: test script is run here.
-# "$@" | [our magic awk script]
+{
+ # FIXME: this usage loses the test program exit status. We should
+ # probably rewrite the awk script to use the
+ # expression | getline [var]
+ # idiom, which should allow us to obtain the final exit status from
+ # <expression> when closing it.
+ { test $merge -eq 0 || exec 2>&1; "$@"; } \
+ | LC_ALL=C ${AM_TAP_AWK-awk} \
+ -v me="$me" \
+ -v test_script_name="$test_name" \
+ -v log_file="$log_file" \
+ -v trs_file="$trs_file" \
+ -v expect_failure="$expect_failure" \
+ -v merge="$merge" \
+ -v ignore_exit="$ignore_exit" \
+ -v comments="$comments" \
+ -v diag_string="$diag_string" \
+'
+# FIXME: the usages of "cat >&3" below could be optimized whne using
+# FIXME: GNU awk, and/on on systems that supports /dev/fd/.
+
+# Implementation note: in what follows, `result_obj` will be an
+# associative array that (partly) simulates a TAP result object
+# from the `TAP::Parser` perl module.
+
+## ----------- ##
+## FUNCTIONS ##
+## ----------- ##
+
+function fatal(msg)
+{
+ print me ": " msg | "cat >&3"
+ exit 1
+}
+
+function abort(where)
+{
+ fatal("internal error " where)
+}
+
+function close_or_die(fpath, fname)
+{
+ if (close(fpath) != 0)
+ fatal(sprintf("could not close %s \"%s\"", fname, fpath))
+}
+
+# Convert a boolean to a "yes"/"no" string.
+function yn(bool)
+{
+ return bool ? "yes" : "no";
+}
+
+function add_test_result(result)
+{
+ if (!test_results_index)
+ test_results_index = 0
+ test_results_list[test_results_index] = result
+ test_results_index += 1
+ test_results_seen[result] = 1;
+}
+
+# Whether the test script should be re-run by "make recheck".
+function must_recheck()
+{
+ for (k in test_results_seen)
+ if (k != "XFAIL" && k != "PASS" && k != "SKIP")
+ return 1
+ return 0
+}
+
+# Whether the content of the log file associated to this test should
+# be copied into the "global" test-suite.log.
+function copy_in_global_log()
+{
+ for (k in test_results_seen)
+ if (k != "PASS")
+ return 1
+ return 0
+}
+
+# FIXME: this can certainly be improved ...
+function get_global_test_result()
+{
+ if ("ERROR" in test_results_seen)
+ return "ERROR"
+ all_skipped = 1
+ for (k in test_results_seen)
+ if (k != "SKIP")
+ all_skipped = 0
+ if (all_skipped)
+ return "SKIP"
+ if ("FAIL" in test_results_seen || "XPASS" in test_results_seen)
+ return "FAIL"
+ return "PASS";
+}
+
+function stringify_result_obj(obj)
+{
+ if (obj["is_unplanned"] || obj["number"] != testno)
+ return "ERROR"
+
+ if (plan_seen == LATE_PLAN)
+ return "ERROR"
+
+ if (result_obj["directive"] == "TODO")
+ return obj["is_ok"] ? "XPASS" : "XFAIL"
+
+ if (result_obj["directive"] == "SKIP")
+ return obj["is_ok"] ? "SKIP" : COOKED_FAIL;
+
+ if (length(result_obj["directive"]))
+ abort("in function stringify_result_obj()")
+
+ return obj["is_ok"] ? COOKED_PASS : COOKED_FAIL
+}
+
+function decorate_result(result)
+{
+ color_name = color_for_result[result]
+ if (color_name)
+ return color_map[color_name] "" result "" color_map["std"]
+ # If we are not using colorized output, or if we do not know how
+ # to colorize the given result, we should return it unchanged.
+ return result
+}
+
+function report(result, details)
+{
+ if (result ~ /^(X?(PASS|FAIL)|SKIP|ERROR)/)
+ {
+ msg = ": " test_script_name
+ add_test_result(result)
+ }
+ else if (result == "#")
+ {
+ msg = " " test_script_name ":"
+ }
+ else
+ {
+ abort("in function report()")
+ }
+ if (length(details))
+ msg = msg " " details
+ # Output on console might be colorized.
+ print decorate_result(result) msg | "cat >&3";
+ # Log the result in the log file too, to help debugging (this is
+ # especially true when said result is a TAP error or "Bail out!").
+ print result msg;
+}
+
+function testsuite_error(error_message)
+{
+ report("ERROR", "- " error_message)
+}
+
+function handle_tap_result()
+{
+ details = result_obj["number"];
+ if (length(result_obj["description"]))
+ details = details " " result_obj["description"]
+
+ if (plan_seen == LATE_PLAN)
+ {
+ details = details " # AFTER LATE PLAN";
+ }
+ else if (result_obj["is_unplanned"])
+ {
+ details = details " # UNPLANNED";
+ }
+ else if (result_obj["number"] != testno)
+ {
+ details = sprintf("%s # OUT-OF-ORDER (expecting %d)",
+ details, testno);
+ }
+ else if (result_obj["directive"])
+ {
+ details = details " # " result_obj["directive"];
+ if (length(result_obj["explanation"]))
+ details = details " " result_obj["explanation"]
+ }
+
+ report(stringify_result_obj(result_obj), details)
+}
+
+# `skip_reason` should be emprty whenever planned > 0.
+function handle_tap_plan(planned, skip_reason)
+{
+ planned += 0 # Avoid getting confused if, say, `planned` is "00"
+ if (length(skip_reason) && planned > 0)
+ abort("in function handle_tap_plan()")
+ if (plan_seen)
+ {
+ # Error, only one plan per stream is acceptable.
+ testsuite_error("multiple test plans")
+ return;
+ }
+ planned_tests = planned
+ # The TAP plan can come before or after *all* the TAP results; we speak
+ # respectively of an "early" or a "late" plan. If we see the plan line
+ # after at least one TAP result has been seen, assume we have a late
+ # plan; in this case, any further test result seen after the plan will
+ # be flagged as an error.
+ plan_seen = (testno >= 1 ? LATE_PLAN : EARLY_PLAN)
+ # If testno > 0, we have an error ("too many tests run") that will be
+ # automatically dealt with later, so do not worry about it here. If
+ # $plan_seen is true, we have an error due to a repeated plan, and that
+ # has already been dealt with above. Otherwise, we have a valid "plan
+ # with SKIP" specification, and should report it as a particular kind
+ # of SKIP result.
+ if (planned == 0 && testno == 0)
+ {
+ if (length(skip_reason))
+ skip_reason = "- " skip_reason;
+ report("SKIP", skip_reason);
+ }
+}
+
+function extract_tap_comment(line)
+{
+ # FIXME: verify there is not an off-by-one bug here.
+ if (index(line, diag_string) == 1)
+ {
+ # Strip leading `diag_string` from `line`.
+ # FIXME: verify there is not an off-by-one bug here.
+ line = substr(line, length(diag_string) + 1)
+ # And strip any leading and trailing whitespace left.
+ sub("^[ \\t]*", "", line)
+ sub("[ \\t]*$", "", line)
+ # Return what is left (if any).
+ return line;
+ }
+ return "";
+}
+
+# When this function is called, we know that line is a TAP result line,
+# so that it matches the (perl) RE "^(not )?ok\b".
+function setup_result_obj(line)
+{
+ # Get the result, and remove it from the line.
+ result_obj["is_ok"] = (substr(line, 1, 2) == "ok" ? 1 : 0)
+ sub("^(not )?ok[ \\t]*", "", line)
+
+ # If the result has an explicit number, get it and strip it; otherwise,
+ # automatically assing the next progresive number to it.
+ if (line ~ /^[0-9]+$/ || line ~ /^[0-9]+[^a-zA-Z0-9_]/)
+ {
+ match(line, "^[0-9]+")
+ # The final `+ 0` is to normalize numbers with leading zeros.
+ result_obj["number"] = substr(line, 1, RLENGTH) + 0
+ line = substr(line, RLENGTH + 1)
+ }
+ else
+ {
+ result_obj["number"] = testno
+ }
+
+ if (plan_seen == LATE_PLAN)
+ # No further test results are acceptable after a "late" TAP plan
+ # has been seen.
+ result_obj["is_unplanned"] = 1
+ else if (plan_seen && testno > planned_tests)
+ result_obj["is_unplanned"] = 1
+ else
+ result_obj["is_unplanned"] = 0
+
+ # Strip trailing and leading whitespace.
+ sub("^[ \\t]*", "", line)
+ sub("[ \\t]*$", "", line)
+
+ # This will have to be corrected if we have a "TODO"/"SKIP" directive.
+ result_obj["description"] = line
+ result_obj["directive"] = ""
+ result_obj["explanation"] = ""
+
+ # TODO: maybe we should allow a way to escape "#"?
+ if (index(line, "#") == 0)
+ return # No possible directive, nothing more to do.
+
+ # Directives are case-insensitive.
+ rx = "[ \\t]*#[ \\t]*([tT][oO][dD][oO]|[sS][kK][iI][pP])[ \\t]*"
+
+ # See whether we have the directive, and if yes, where.
+ pos = match(line, rx "$")
+ if (!pos)
+ pos = match(line, rx "[^a-zA-Z0-9_]")
+
+ # If there was no TAP directive, we have nothing more to do.
+ if (!pos)
+ return
+
+ # Strip the directive and its explanation (if any) from the test
+ # description.
+ result_obj["description"] = substr(line, 1, pos - 1)
+ # Now remove the test description from the line, that has been dealt
+ # with already.
+ line = substr(line, pos)
+ # Strip the directive, and save its value (normalized to upper case).
+ sub("^[ \\t]*#[ \\t]*", "", line)
+ result_obj["directive"] = toupper(substr(line, 1, 4))
+ line = substr(line, 5)
+ # Now get the explanation for the directive (if any), with leading
+ # and trailing whitespace removed.
+ sub("^[ \\t]*", "", line)
+ sub("[ \\t]*$", "", line)
+ result_obj["explanation"] = line
+}
+
+function write_test_results()
+{
+ print ":global-test-result: " get_global_test_result() > trs_file
+ print ":recheck: " yn(must_recheck()) > trs_file
+ print ":copy-in-global-log: " yn(copy_in_global_log()) > trs_file
+ for (i = 0; i < test_results_index; i += 1)
+ print ":test-result: " test_results_list[i] > trs_file
+ close_or_die(trs_file, "trs file");
+}
+
+## ------- ##
+## SETUP ##
+## ------- ##
+
+BEGIN {
+
+ '"$init_colors"'
+
+ # Properly initialized once the TAP plan is seen.
+ planned_tests = 0
+
+ COOKED_PASS = expect_failure ? "XPASS": "PASS";
+ COOKED_FAIL = expect_failure ? "XFAIL": "FAIL";
+
+ # Enumeration-like constants to remember which kind of plan (if any)
+ # has been seen. It is important that NO_PLAN evaluates "false" as
+ # a boolean.
+ NO_PLAN = 0
+ EARLY_PLAN = 1
+ LATE_PLAN = 2
+
+ testno = 0 # Number of test results seen so far.
+ bailed_out = 0 # Whether a "Bail out!" directive has been seen.
+
+ # Whether the TAP plan has been seen or not, and if yes, which kind
+ # it is ("early" is seen before any test result, "late" otherwise).
+ plan_seen = NO_PLAN
+
+}
+
+## --------- ##
+## PARSING ##
+## --------- ##
+
+{
+ # Copy any input line verbatim into the log file.
+ print
+ # Parsing of TAP input should stop after a "Bail out!" directive.
+ if (bailed_out)
+ next
+}
+
+# TAP test result.
+($0 ~ /^(not )?ok$/ || $0 ~ /^(not )?ok[^a-zA-Z0-9_]/) {
+
+ testno += 1
+ setup_result_obj($0)
+ handle_tap_result()
+ next
+
+}
+
+# TAP plan (normal or "SKIP" without explanation).
+/^1\.\.[0-9]+[ \t]*$/ {
+
+ # The next two lines will put the number of planned tests in $0.
+ sub("^1\\.\\.", "")
+ sub("[^0-9]*$", "")
+ handle_tap_plan($0, "")
+ next
+
+}
+
+# TAP "SKIP" plan, with an explanation.
+/^1\.\.0+[ \t]*#/ {
+
+ # The next lines will put the skip explanation in $0, stripping any
+ # leading and trailing whitespace. This is a little more tricky in
+ # thruth, since we want to also strip a potential leading "SKIP"
+ # string from the message.
+ sub("^[^#]*#[ \\t]*(SKIP[: \\t][ \\t]*)?", "")
+ sub("[ \\t]*$", "");
+ handle_tap_plan(0, $0)
+ next
+
+}
+
+# "Bail out!" magic.
+/^Bail out!/ {
+
+ bailed_out = 1
+ # Get the bailout message (if any), with leading and trailing
+ # whitespace stripped. The message remains stored in `$0`.
+ sub("^Bail out![ \\t]*", "");
+ sub("[ \\t]*$", "");
+ # Format the error message for the
+ bailout_message = "Bail out!"
+ if (length($0))
+ bailout_message = bailout_message " " $0
+ testsuite_error(bailout_message)
+ next
+
+}
+
+(comments != 0) {
+
+ comment = extract_tap_comment($0);
+ if (length(comment))
+ report("#", comment);
+
+}
+
+## -------- ##
+## FINISH ##
+## -------- ##
+
+END {
+
+ # A "Bail out!" directive should cause us to ignore any following TAP
+ # error, as well as a non-zero exit status from the TAP producer.
+ if (!bailed_out)
+ {
+ if (!plan_seen)
+ testsuite_error("missing test plan")
+ else if (planned_tests != testno)
+ {
+ bad_amount = testno > planned_tests ? "many" : "few"
+ testsuite_error(sprintf("too %s tests run (expected %d, got %d)",
+ bad_amount, planned_tests, testno))
+ }
+ }
+ write_test_results()
+
+ exit 0
+}
+'
+
+# TODO: document that we consume the file descriptor 3 :-(
+} 3>&1 >"$log_file" 2>&1
-echo "$0: still to be implemented, sorry" >&2
-exit 255
+test $? -eq 0 || fatal "I/O or internal error"
# Local Variables:
# mode: shell-script
diff --git a/tests/Makefile.am b/tests/Makefile.am
index cd6ed2c..1dbdcb2 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -1195,6 +1195,7 @@ tap-planskip-late.test \
tap-planskip-and-logging.test \
tap-planskip-unplanned.test \
tap-planskip-unplanned-corner.test \
+tap-planskip-case-insensitive.test \
tap-planskip-whitespace.test \
tap-planskip-badexit.test \
tap-planskip-bailout.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index ed955ac..631bd8c 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -1436,6 +1436,7 @@ tap-planskip-late.test \
tap-planskip-and-logging.test \
tap-planskip-unplanned.test \
tap-planskip-unplanned-corner.test \
+tap-planskip-case-insensitive.test \
tap-planskip-whitespace.test \
tap-planskip-badexit.test \
tap-planskip-bailout.test \
diff --git a/tests/tap-number-wordboundary.test
b/tests/tap-number-wordboundary.test
index 5235738..74cddc1 100755
--- a/tests/tap-number-wordboundary.test
+++ b/tests/tap-number-wordboundary.test
@@ -93,7 +93,8 @@ $
\
END
-planned=`wc -l <punctuation`
+# Strip any extra whitespace, for Solaris' wc.
+planned=`wc -l <punctuation | tr -d " $tab"`
echo 1..$planned > all.test
awk '{print "ok " NR $0 }' punctuation >> all.test
diff --git a/tests/tap-msg0-planskip.test
b/tests/tap-planskip-case-insensitive.test
similarity index 62%
copy from tests/tap-msg0-planskip.test
copy to tests/tap-planskip-case-insensitive.test
index b8c3d6c..2ed2626 100755
--- a/tests/tap-msg0-planskip.test
+++ b/tests/tap-planskip-case-insensitive.test
@@ -15,23 +15,32 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# TAP support:
-# - literal strings "0" and "0.0" as the reason of the skip in a "TAP
-# plan with skip" (i.e., "1..0 # SKIP ...").
+# - "SKIP" keyword in a TAP plan is case-insensitive
parallel_tests=yes
. ./defs || Exit 1
. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
-echo '1..0 # SKIP 0' > a.test
-echo '1..0 # SKIP 0.0' > b.test
+# These nested loops are clearer without indentation.
-TESTS='a.test b.test' $MAKE -e check >stdout || { cat stdout; Exit 1; }
+i=0
+for c1 in s S; do
+for c2 in k K; do
+for c3 in i I; do
+for c4 in p P; do
+ i=`expr $i + 1`
+ case $i in ?) i=0$i; esac
+ echo "1..0 # $c1$c2$c3$c4 foobar" > $i.test
+done; done; done; done
+
+TESTS="`echo *.test`" $MAKE -e check >stdout || { cat stdout; Exit 1; }
cat stdout
-count_test_results total=2 pass=0 fail=0 xpass=0 xfail=0 skip=2 error=0
+count_test_results total=16 pass=0 fail=0 xpass=0 xfail=0 skip=16 error=0
-grep '^SKIP: a.test - 0$' stdout
-grep '^SKIP: b.test - 0\.0$' stdout
+for tst in *.test; do
+ grep "^SKIP: $tst .* foobar$" stdout
+done
:
diff --git a/tests/tap-planskip-unplanned-corner.test
b/tests/tap-planskip-unplanned-corner.test
index cf9f49e..4e825d5 100755
--- a/tests/tap-planskip-unplanned-corner.test
+++ b/tests/tap-planskip-unplanned-corner.test
@@ -38,7 +38,8 @@ cat stdout
count_test_results total=5 pass=0 fail=0 xpass=0 xfail=0 skip=2 error=3
-grep '^SKIP: foo\.test$' stdout
+# Don't be too strict w.r.t. the format of the "skip whole test" message.
+grep '^SKIP: foo\.test$' stdout || grep '^SKIP: foo\.test [^0-9]' stdout
grep '^ERROR: foo\.test 1 # UNPLANNED$' stdout
grep '^ERROR: foo\.test - too many tests run (expected 0, got 1)$' stdout
grep '^SKIP: bar\.test 1 # SKIP$' stdout
diff --git a/tests/tap-planskip.test b/tests/tap-planskip.test
index 2852305..35f7976 100755
--- a/tests/tap-planskip.test
+++ b/tests/tap-planskip.test
@@ -49,12 +49,12 @@ cat > wget.test <<END
END
cat > curl.test <<END
-1..0 # skip: Can't connect to gnu.org!
+1..0 # SKIP: Can't connect to gnu.org!
# See also wget.test
END
cat > mu.test <<END
-1..0 # Skip $weirdchars
+1..0 # SKIP $weirdchars
END
env TESTS='foo.test bar.test baz.test wget.test curl.test mu.test' \
@@ -63,6 +63,10 @@ cat stdout
count_test_results total=6 pass=0 fail=0 xpass=0 xfail=0 skip=6 error=0
+# Look for a regression where the "1..0" wasn't being stripped from the
+# SKIP message.
+$FGREP '1..0' stdout && Exit 1
+
grep '^SKIP: foo\.test$' stdout
grep '^SKIP: bar\.test$' stdout
grep '^SKIP: baz\.test' stdout # Deliberately laxer, see above for why.
hooks/post-receive
--
GNU Automake
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- [Automake-commit] [SCM] GNU Automake branch, test-protocols, updated. v1.11-1046-gc9ca5f6,
Stefano Lattarini <=