[SCM] GNU Autoconf source repository branch, master, updated. v2.63b-25-g7ece247

[Eric Blake]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU Autoconf source repository".

http://git.sv.gnu.org/gitweb/?p=autoconf.git;a=commitdiff;h=7ece247bdd194997f5a79aeaad3e7b02bc7130e2

The branch, master has been updated
       via  7ece247bdd194997f5a79aeaad3e7b02bc7130e2 (commit)
       via  c241057ad4f1a35e764964eeddc8983d07fd9c5e (commit)
       via  e8f34dca8c7477abc5003ba4b5dda5b0873b043d (commit)
      from  bc796433831e23df45f75c9e34e50860e44da90f (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
commit 7ece247bdd194997f5a79aeaad3e7b02bc7130e2
Author: Eric Blake <address@hidden>
Date:   Wed Apr 15 06:38:58 2009 -0600

    Test that autotest handles binary output.
    
    * tests/autotest.at (Binary output): New test.
    Suggested by Ralf Wildenhues.
    
    Signed-off-by: Eric Blake <address@hidden>

commit c241057ad4f1a35e764964eeddc8983d07fd9c5e
Author: Eric Blake <address@hidden>
Date:   Mon Apr 13 07:10:32 2009 -0600

    Add stdout-nolog and ignore-nolog to AT_CHECK.
    
    * lib/autotest/general.m4 (AT_DIFF_STDERR(stderr-nolog))
    (AT_DIFF_STDERR(ignore-nolog), AT_DIFF_STDOUT(stdout-nolog))
    (AT_DIFF_STDOUT(ignore-nolog)): New macros.
    * tests/autotest.at (Logging): New test.
    * doc/autoconf.texi (Writing Testsuites) <AT_CHECK>: Document the
    new logging actions.
    * NEWS: Likewise.
    Reported by Ralf Wildenhues.
    
    Signed-off-by: Eric Blake <address@hidden>

commit e8f34dca8c7477abc5003ba4b5dda5b0873b043d
Author: Eric Blake <address@hidden>
Date:   Mon Apr 13 06:02:42 2009 -0600

    Teach AT_CHECK about hard failures.
    
    * lib/autotest/general.m4 (AT_INIT) <at_fn_check_skip>
    <at_fn_check_status, at_fn_group_postprocess>: Handle hard
    failures.
    * doc/autoconf.texi (Writing Testsuites) <AT_CHECK>: Document
    AT_CHECK_NOESCAPE and exit status 99.
    * NEWS: Likewise.
    * tests/autotest.at (Hard fail, Cleanup): New tests.
    
    Signed-off-by: Eric Blake <address@hidden>

-----------------------------------------------------------------------

Summary of changes:
 ChangeLog               |   25 +++++++++
 NEWS                    |    8 +++
 doc/autoconf.texi       |   93 +++++++++++++++++++++++++++-------
 lib/autotest/general.m4 |   33 +++++++++---
 tests/autotest.at       |  126 +++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 258 insertions(+), 27 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index feaabe6..b7dd8f6 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,28 @@
+2009-04-15  Eric Blake  <address@hidden>
+
+       Test that autotest handles binary output.
+       * tests/autotest.at (Binary output): New test.
+       Suggested by Ralf Wildenhues.
+
+       Add stdout-nolog and ignore-nolog to AT_CHECK.
+       * lib/autotest/general.m4 (AT_DIFF_STDERR(stderr-nolog))
+       (AT_DIFF_STDERR(ignore-nolog), AT_DIFF_STDOUT(stdout-nolog))
+       (AT_DIFF_STDOUT(ignore-nolog)): New macros.
+       * tests/autotest.at (Logging): New test.
+       * doc/autoconf.texi (Writing Testsuites) <AT_CHECK>: Document the
+       new logging actions.
+       * NEWS: Likewise.
+       Reported by Ralf Wildenhues.
+
+       Teach AT_CHECK about hard failures.
+       * lib/autotest/general.m4 (AT_INIT) <at_fn_check_skip>
+       <at_fn_check_status, at_fn_group_postprocess>: Handle hard
+       failures.
+       * doc/autoconf.texi (Writing Testsuites) <AT_CHECK>: Document
+       AT_CHECK_NOESCAPE and exit status 99.
+       * NEWS: Likewise.
+       * tests/autotest.at (Hard fail, Cleanup): New tests.
+
 2009-04-14  Eric Blake  <address@hidden>
 
        Fix yesterday's regression in AS_IF.
diff --git a/NEWS b/NEWS
index 8eb3a06..60a6e20 100644
--- a/NEWS
+++ b/NEWS
@@ -10,6 +10,14 @@ GNU Autoconf NEWS - User visible changes.
    proper m4 quoting.  For shell comments, this is a new feature; for
    non-shell comments, this fixes a regression introduced in 2.63b.
 
+** The macro AT_CHECK now understands the concept of hard failure.  If
+   a test exits with an unexpected status 99, cleanup actions for the
+   test are inhibited and the test is treated as a failure regardless
+   of AT_XFAIL_IF.  It also understands the new directives
+   ignore-nolog, stdout-nolog, and stderr-nolog.
+
+** The autotest macro AT_CHECK_NOESCAPE is now documented.
+
 ** The following documented m4sugar macros are new:
    m4_default_nblank  m4_default_nblank_quoted  m4_ifblank  m4_ifnblank
 
diff --git a/doc/autoconf.texi b/doc/autoconf.texi
index 38b0972..e565dbf 100644
--- a/doc/autoconf.texi
+++ b/doc/autoconf.texi
@@ -22385,34 +22385,89 @@ expansion.  The contents must end with an end of 
line.  @var{file} must
 be a single shell word that expands into a single file name.
 @end defmac
 
address@hidden AT_CHECK (@var{commands}, @dvar{status, 0}, @dvar{stdout, }, @
-  @dvar{stderr, }, @ovar{run-if-fail}, @ovar{run-if-pass})
address@hidden AT_CHECK (@var{commands}, @dvar{status, 0}, @ovar{stdout}, @
+  @ovar{stderr}, @ovar{run-if-fail}, @ovar{run-if-pass})
address@hidden AT_CHECK_NOESCAPE (@var{commands}, @dvar{status, 0}, 
@ovar{stdout}, @
+  @ovar{stderr}, @ovar{run-if-fail}, @ovar{run-if-pass})
 @atindex{CHECK}
address@hidden
 Execute a test by performing given shell @var{commands}.  These commands
 should normally exit with @var{status}, while producing expected
 @var{stdout} and @var{stderr} contents.  If @var{commands} exit with
-status 77, then the whole test group is skipped.  Otherwise, if this test
+unexpected status 77, then the rest of the test group is skipped.  If
address@hidden exit with unexpected status 99, then the test group is
+immediately failed.  Otherwise, if this test
 fails, run shell commands @var{run-if-fail} or, if this test passes, run shell
 commands @var{run-if-pass}.
 
 This macro must be invoked in between @code{AT_SETUP} and @code{AT_CLEANUP}.
 
address@hidden Previously, we had this:
address@hidden  The @var{commands} @emph{must not} redirect the standard 
output, nor the
address@hidden  standard error.
address@hidden to prevent trigerring the double redirect bug on Ultrix, see
address@hidden `File Descriptors'.  This was too restricting, and Ultrix is 
pretty
address@hidden much dead, so we dropped the limitation; the obvious workaround 
on
address@hidden Ultrix is to use a working shell there.
-
-If @var{status}, or @var{stdout}, or @var{stderr} is @samp{ignore}, then
-the corresponding value is not checked.
-
-The special value @samp{expout} for @var{stdout} means the expected
-output of the @var{commands} is the content of the file @file{expout}.
-If @var{stdout} is @samp{stdout}, then the standard output of the
address@hidden is available for further tests in the file @file{stdout}.
-Similarly for @var{stderr} with @samp{experr} and @samp{stderr}.
+If @var{status} is the literal @samp{ignore}, then the corresponding
+exit status is not checked, except for the special cases of 77 (skip)
+and 99 (hard failure).  The existence of hard failures allows one to
+mark a test as an expected failure with @code{AT_XFAIL_IF} because a
+feature has not yet been implemented, but to still distinguish between
+gracefully handling the missing feature and dumping core.  A hard
+failure also inhibits post-test actions in @var{run-if-fail}.
+
+If the value of the @var{stdout} or @var{stderr} parameter is one of the
+literals in the following table, then the test treats the output
+according to the rules of that literal.  Otherwise, the value of the
+parameter is treated as text that must exactly match the output given by
address@hidden on standard out and standard error (including an empty
+parameter for no output); any differences are captured in the testsuite
+log and the test is failed.  The difference between @code{AT_CHECK} and
address@hidden is that only the latter performs shell
+expansions on comparison text given in the @var{stdout} and @var{stderr}
+arguments.
+
address@hidden @samp
address@hidden ignore
+The content of the output is ignored, but still captured in the test
+group log (if the testsuite is run with option @option{-v}, the test
+group log is displayed as the test is run; if the test group later
+fails, the test group log is also copied into the overall testsuite
+log).  This action is valid for both @var{stdout} and @var{stderr}.
+
address@hidden ignore-nolog
+The content of the output is ignored, and nothing is captured in the log
+files.  If @var{commands} are likely to produce binary output (including
+long lines) or large amounts of output, then logging the output can make
+it harder to locate details related to subsequent tests within the
+group, and could potentially corrupt terminal display of a user running
address@hidden -v}.
+
address@hidden stdout
+For the @var{stdout} parameter, capture the content of standard output
+to both the file @file{stdout} and the test group log.  Subsequent
+commands in the test group can then post-process the file.  This action
+is often used when it is desired to use @command{grep} to look for a
+substring in the output, or when the output must be post-processed to
+normalize error messages into a common form.
+
address@hidden stderr
+Like @samp{stdout}, except that it only works for the @var{stderr}
+parameter, and the standard error capture file will be named
address@hidden
+
address@hidden stdout-nolog
address@hidden stderr-nolog
+Like @samp{stdout} or @samp{stderr}, except that the captured output is
+not duplicated into the test group log.  This action is particularly
+useful for an intermediate check that produces large amounts of data,
+which will be followed by another check that filters down to the
+relevant data, as it makes it easier to locate details in the log.
+
address@hidden expout
+For the @var{stdout} parameter, compare standard output contents with
+the previously created file @file{expout}, and list any differences in
+the testsuite log.
+
address@hidden experr
+Like @samp{expout}, except that it only works for the @var{stderr}
+parameter, and the standard error contents are compared with
address@hidden
address@hidden table
 @end defmac
 
 
diff --git a/lib/autotest/general.m4 b/lib/autotest/general.m4
index 906e481..2e622a5 100644
--- a/lib/autotest/general.m4
+++ b/lib/autotest/general.m4
@@ -324,27 +324,33 @@ at_fn_log_failure ()
   exit 1
 }
 
-AS_FUNCTION_DESCRIBE([at_fn_check_skip], [EXIT-CODE],
-[Check whether EXIT-CODE is the special exit code 77, and if so exit the shell
-with that same exit code.])
+AS_FUNCTION_DESCRIBE([at_fn_check_skip], [EXIT-CODE LINE],
+[Check whether EXIT-CODE is a special exit code (77 or 99), and if so exit
+the test group subshell with that same exit code.  Use LINE in any report
+about test failure.])
 at_fn_check_skip ()
 {
   case $[1] in
+    99) echo 99 > "$at_status_file"; at_failed=:
+        AS_ECHO(["$[2]: hard failure"]); exit 99;;
     77) echo 77 > "$at_status_file"; exit 77;;
   esac
 }
 
 AS_FUNCTION_DESCRIBE([at_fn_check_status], [EXPECTED EXIT-CODE LINE],
-[Check whether EXIT-CODE is the expected exit code, and if so do nothing.
-Otherwise, if it is 77 exit the shell with that same exit code; if it is
-anything else print an error message and fail the test.])
+[Check whether EXIT-CODE is the EXPECTED exit code, and if so do nothing.
+Otherwise, if it is 77 or 99, exit the test group subshell with that same
+exit code; if it is anything else print an error message referring to LINE,
+and fail the test.])
 at_fn_check_status ()
 {
 dnl This order ensures that we don't `skip' if we are precisely checking
-dnl $? = 77.
+dnl $? = 77 or $? = 99.
   case $[2] in
     $[1] ) ;;
     77) echo 77 > "$at_status_file"; exit 77;;
+    99) echo 99 > "$at_status_file"; at_failed=:
+        AS_ECHO(["$[3]: hard failure"]); exit 99;;
     *) AS_ECHO(["$[3]: exit code was $[2], expected $[1]"])
       at_failed=:;;
   esac
@@ -1130,11 +1136,16 @@ at_fn_group_postprocess ()
       report this failure to <AT_PACKAGE_BUGREPORT>.
 _ATEOF
     AS_ECHO(["$at_setup_line"]) >"$at_check_line_file"
-    at_xfail=no at_status=99
+    at_status=99
   fi
   $at_verbose AS_ECHO_N(["$at_group. $at_setup_line: "])
   AS_ECHO_N(["$at_group. $at_setup_line: "]) >> "$at_group_log"
   case $at_xfail:$at_status in
+    *:99)
+       at_msg='FAILED ('`cat "$at_check_line_file"`')'
+       at_res=fail
+       at_errexit=$at_errexit_p
+       ;;
     yes:0)
        at_msg="UNEXPECTED PASS"
        at_res=xpass
@@ -2008,8 +2019,11 @@ dnl COMMANDS may contain parameter expansions; expand 
them at runtime.
 # faster than using m4_case, and these are called very frequently.
 m4_define([AT_DIFF_STDERR(stderr)],
          [echo stderr:; tee stderr <"$at_stderr"])
+m4_define([AT_DIFF_STDERR(stderr-nolog)],
+         [echo stderr captured; cp "$at_stderr" stderr])
 m4_define([AT_DIFF_STDERR(ignore)],
          [echo stderr:; cat "$at_stderr"])
+m4_define([AT_DIFF_STDERR(ignore-nolog)])
 m4_define([AT_DIFF_STDERR(experr)],
          [$at_diff experr "$at_stderr" || at_failed=:])
 m4_define([AT_DIFF_STDERR()],
@@ -2017,8 +2031,11 @@ m4_define([AT_DIFF_STDERR()],
 
 m4_define([AT_DIFF_STDOUT(stdout)],
          [echo stdout:; tee stdout <"$at_stdout"])
+m4_define([AT_DIFF_STDOUT(stdout-nolog)],
+         [echo stdout captured; cp "$at_stdout" stdout])
 m4_define([AT_DIFF_STDOUT(ignore)],
          [echo stdout:; cat "$at_stdout"])
+m4_define([AT_DIFF_STDOUT(ignore-nolog)])
 m4_define([AT_DIFF_STDOUT(expout)],
          [$at_diff expout "$at_stdout" || at_failed=:])
 m4_define([AT_DIFF_STDOUT()],
diff --git a/tests/autotest.at b/tests/autotest.at
index d9c6081..721925c 100644
--- a/tests/autotest.at
+++ b/tests/autotest.at
@@ -253,6 +253,16 @@ AT_CHECK_AT_TEST([Skip],
   [], [], [], [], [],
   [AT_CHECK([grep skipped micro-suite.log], [], [ignore])])
 
+AT_CHECK_AT_TEST([Hard fail],
+  [AT_CHECK([exit 99])
+   AT_CLEANUP
+   AT_SETUP([another test])
+   AT_XFAIL_IF([:])
+   AT_CHECK([exit 99])],
+  [], [1], [], [ignore], [],
+  [AT_CHECK([grep '2 failed unexpectedly' micro-suite.log], [], [ignore])
+   AT_CHECK([grep ok micro-suite.log], [1])])
+
 AT_CHECK_AT_TEST([Syntax error],
   [AT_CHECK([:])
    AT_CLEANUP
@@ -277,6 +287,122 @@ AT_CHECK_AT_TEST([errexit],
    AT_CHECK([grep "1 .* inhibited subsequent" stderr], [], [ignore])],
   [--errexit])
 
+AT_CHECK_AT([Logging],
+  [[AT_INIT([artificial test suite])
+    dnl intentionally write failing tests, to see what gets logged
+    AT_SETUP([one])
+    AT_CHECK([echo magicstring01], [1], [ignore])
+    AT_CLEANUP
+    AT_SETUP([two])
+    AT_CHECK([echo magicstring02 >&2], [1], [], [ignore])
+    AT_CLEANUP
+    AT_SETUP([three])
+    AT_CHECK([echo magicstring03], [1], [ignore-nolog])
+    AT_CLEANUP
+    AT_SETUP([four])
+    AT_CHECK([echo magicstring04 >&2], [1], [], [ignore-nolog])
+    AT_CLEANUP
+    AT_SETUP([five])
+    AT_CHECK([echo magicstring05], [1], [stdout])
+    AT_CLEANUP
+    AT_SETUP([six])
+    AT_CHECK([echo magicstring06 >&2], [1], [], [stderr])
+    AT_CLEANUP
+    AT_SETUP([seven])
+    AT_CHECK([echo magicstring07], [1], [stdout-nolog])
+    AT_CLEANUP
+    AT_SETUP([eight])
+    AT_CHECK([echo magicstring08 >&2], [1], [], [stderr-nolog])
+    AT_CLEANUP
+    AT_SETUP([nine])
+    echo magicstring09 > expout
+    AT_CHECK([echo magicstring09], [1], [expout])
+    AT_CLEANUP
+    AT_SETUP([ten])
+    echo magicstring10 > experr
+    AT_CHECK([echo magicstring10 >&2], [1], [], [experr])
+    AT_CLEANUP
+]], [], [1], [], [ignore], [],
+  [AT_CHECK([$CONFIG_SHELL ./micro-suite], [1], [ignore-nolog], [ignore-nolog])
+   AT_CHECK([grep '^magicstring' micro-suite.log], [],
+[[magicstring01
+magicstring02
+magicstring05
+magicstring06
+]])])
+
+
+AT_CHECK_AT([Binary output],
+  [[AT_INIT([artificial test suite])
+    AT_SETUP([pass: no trailing newline])
+    AT_CHECK([printf short], [0], [stdout-nolog])
+    AT_CHECK([cat stdout], [0], [[short]])
+    AT_CLEANUP
+    AT_SETUP([pass: non-printing characters])
+    AT_CHECK([printf '\1\n' >&2], [0], [], [stderr-nolog])
+    printf '\1\n' > expout
+    AT_CHECK([cat stderr], [0], [expout])
+    AT_CLEANUP
+    AT_SETUP([pass: long lines])
+    # 5000 bytes in str
+    str=..........
+    str=$str$str$str$str$str$str$str$str$str$str
+    str=$str$str$str$str$str$str$str$str$str$str
+    str=$str$str$str$str$str
+    AT_CHECK_NOESCAPE([echo $str], [0], [[$str]m4_newline])
+    AT_CLEANUP
+    AT_SETUP([fail: no trailing newline])
+    AT_CHECK([printf short], [0], [stdout-nolog])
+    AT_CHECK([cat stdout], [0], [[long]])
+    AT_CLEANUP
+    AT_SETUP([fail: non-printing characters])
+    AT_CHECK([printf '\1\n' >&2], [0], [], [stderr-nolog])
+    printf '\2\n' > expout
+    AT_CHECK([cat stderr], [0], [expout])
+    AT_CLEANUP
+    AT_SETUP([fail: long lines])
+    # 5000 bytes in str
+    str=..........
+    str=$str$str$str$str$str$str$str$str$str$str
+    str=$str$str$str$str$str$str$str$str$str$str
+    str=$str$str$str$str$str
+    AT_CHECK_NOESCAPE([echo x$str], [0], [[${str}x]m4_newline])
+    AT_CLEANUP
+]], [], [0], [], [], [],
+  [AT_CHECK([$CONFIG_SHELL ./micro-suite 4], [1], [ignore], [ignore])
+   AT_CHECK([$CONFIG_SHELL ./micro-suite 5], [1], [ignore], [ignore])
+   AT_CHECK([$CONFIG_SHELL ./micro-suite 6], [1], [ignore], [ignore])], [1-3])
+
+
+AT_CHECK_AT_TEST([Cleanup],
+  [AT_CHECK([test ! -f cleanup.success && test ! -f cleanup.failure])
+   AT_CHECK_NOESCAPE([exit $value], [ignore], [$output],
+     [], [touch cleanup.failure], [touch cleanup.success])],
+  [], [], [], [],
+  [AT_KEYWORDS([AT@&address@hidden)
+    output=; export output],
+  [AT_CHECK([test -d micro-suite.dir/1])
+   AT_CHECK([test -f micro-suite.dir/1/cleanup.success])
+   AT_CHECK([test ! -f micro-suite.dir/1/cleanup.failure])
+
+   AT_CHECK([$CONFIG_SHELL ./micro-suite -d value=1], [], [ignore])
+   AT_CHECK([test -f micro-suite.dir/1/cleanup.success])
+   AT_CHECK([test ! -f micro-suite.dir/1/cleanup.failure])
+
+   AT_CHECK([$CONFIG_SHELL ./micro-suite -d value=1 output=mismatch],
+     [1], [ignore], [ignore])
+   AT_CHECK([test ! -f micro-suite.dir/1/cleanup.success])
+   AT_CHECK([test -f micro-suite.dir/1/cleanup.failure])
+
+   AT_CHECK([$CONFIG_SHELL ./micro-suite -d value=77], [], [ignore])
+   AT_CHECK([test ! -f micro-suite.dir/1/cleanup.success])
+   AT_CHECK([test ! -f micro-suite.dir/1/cleanup.failure])
+
+   AT_CHECK([$CONFIG_SHELL ./micro-suite -d value=99], [1], [ignore], [ignore])
+   AT_CHECK([test ! -f micro-suite.dir/1/cleanup.success])
+   AT_CHECK([test ! -f micro-suite.dir/1/cleanup.failure])
+  ], [-d value=0])
+
 ## ----------------------------------------------------- ##
 ## Newlines and command substitutions in test commands.  ##
 ## ----------------------------------------------------- ##


hooks/post-receive
--
GNU Autoconf source repository