Re: verbose testsuite output contains 1M newlines

[Eric Blake]

Eric Blake <ebb9 <at> byu.net> writes:

> According to Ralf Wildenhues on 11/23/2008 6:23 AM:

Hi Ralf, and sorry for the long delay,

> >   make check-local TESTSUITEFLAGS=-v
> > 
> > on M4 (git master) produces lots of output on GNU/Linux/bash.  More
> > precisely, the AT_CHECK in builtins.at:470 causes 1024*1024 newlines
> > to be output, and I think there are more instances later.  Can this
> > easily be avoided?
> 
> Hmm, this particular test _has_ to generate more than a megabyte of output
> in order to trigger the code path in m4 being tested (namely, that m4
> starts using temporary files after it collects enough in-memory diversion
> output).  The problem stems from the fact that by default, when you
> specify [stdout] as the third argument to AT_CHECK, autotest emits code
> that both collects the file stdout (good) and copies the resulting file
> into the log (annoying), and when -v is in effect, tees that copy to the
> terminal (bad).  I think the idea behind this decision was that
> testsuite.log should be self-contained, with everything needed to
> reproduce why a test failed.
> 
> The point of specifying [stdout] is that we plan on processing the
> resulting output file later on, so maybe we should quit trying to copy the
> captured file into the logs.  Maybe we should revisit this, and change it
> so that the file is just collected, with nothing output into the log.  Or
> even add a heuristic, where only the first and last 10 lines of the
> resulting file are output into the log.  If the test passed, then the log
> doesn't need megabytes of information; and if the test fails, hopefully
> trimming out the middle of the file from the log still leaves enough
> information in the log to help diagnose what went wrong; the full file
> will still be available in testsuite.dir/nnn/stdout if the log was
> insufficient by itself.

Now that autoconf supports this usage, I've installed the following patch to 
autoconf.  Note that this (once again) means that m4.git master branch requires 
unreleased autotools; oh well.  M4 1.6 is still not out, so it should easily be 
the case that M4 2.0 (the master branch) is not in any rush waiting for the 
autotools to have a stable release in the meantime.


From: Eric Blake <address@hidden>
Date: Mon, 4 May 2009 14:47:25 -0600
Subject: [PATCH] Silence verbose testsuite runs.

* tests/testsuite.at (m4_version_prereq): Bump to recent
autoconf.git version where stdout-nolog was added.
* tests/builtins.at (esyscmd, syscmd): Use new stdout-nolog.
* tests/freeze.at (AT_TEST_FREEZE): Likewise.
* tests/others.at (stderr closed): Likewise.
* HACKING: Mention new requirement.
* bootstrap: Likewise.
Reported by Ralf Wildenhues.

Signed-off-by: Eric Blake <address@hidden>
---
 ChangeLog          |   12 ++++++++++++
 HACKING            |    2 +-
 bootstrap          |    6 +++---
 tests/builtins.at  |    4 ++--
 tests/freeze.at    |    7 ++++---
 tests/others.at    |    2 +-
 tests/testsuite.at |    5 +++--
 7 files changed, 26 insertions(+), 12 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 7e38a6f..88e64e2 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,15 @@
+2009-05-04  Eric Blake  <address@hidden>
+
+       Silence verbose testsuite runs.
+       * tests/testsuite.at (m4_version_prereq): Bump to recent
+       autoconf.git version where stdout-nolog was added.
+       * tests/builtins.at (esyscmd, syscmd): Use new stdout-nolog.
+       * tests/freeze.at (AT_TEST_FREEZE): Likewise.
+       * tests/others.at (stderr closed): Likewise.
+       * HACKING: Mention new requirement.
+       * bootstrap: Likewise.
+       Reported by Ralf Wildenhues.
+
 2009-04-21  Eric Blake  <address@hidden>

        Be more explicit about default quoting characters.
diff --git a/HACKING b/HACKING
index ec22cad..ef04820 100644
--- a/HACKING
+++ b/HACKING
@@ -64,7 +64,7 @@ and is not part of a release distribution.
 * Before you can build from git, you need to bootstrap.  This requires:
   - A pre-installed version of GNU M4 1.4.5 or later, built from a
     package (recommend 1.4.13 or later)
-  - Autoconf 2.62 or later
+  - A git checkout of Autoconf (2.63b-41 or later)
   - Automake 1.10b or later
   - Libtool 2.2 or later
   - Gettext 0.16 or later
diff --git a/bootstrap b/bootstrap
index ae7df5f..628b4bd 100755
--- a/bootstrap
+++ b/bootstrap
@@ -1,6 +1,6 @@
 #! /bin/sh

-# bootstrap (GNU M4) version 2009-03-23
+# bootstrap (GNU M4) version 2009-05-04
 # Copyright (C) 2004, 2005, 2006, 2007, 2008, 2009 Free Software
 # Foundation, Inc.
 # License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
@@ -38,8 +38,8 @@

 # This script bootstraps a git or CVS checkout of GNU M4 by correctly calling
 # out to parts of the GNU Build Platform.  Currently this requires GNU
-# Gettext 0.16 or better, Autoconf 2.62 or better, GNU M4 1.4.5 or better,
-# the latest git Automake 1.10a or better, Libtool 2.2 or better,
+# Gettext 0.16 or better, the latest git Autoconf 2.63b-41 or better,
+# GNU M4 1.4.5 or better, Automake 1.10b or better, Libtool 2.2 or better,
 # and the latest git or CVS checkout of Gnulib.
 # Libtool must be installed; either with the same --prefix as
 # automake, or made accessible to aclocal's search path via
diff --git a/tests/builtins.at b/tests/builtins.at
index 765ebec..4f11927 100644
--- a/tests/builtins.at
+++ b/tests/builtins.at
@@ -485,7 +485,7 @@ world
 esyscmd(`echo hi >&3')divert
 hello
 ]])
-AT_CHECK_M4([3>&-], [0], [stdout], [experr], [in.m4])
+AT_CHECK_M4([3>&-], [0], [stdout-nolog], [experr], [in.m4])
 AT_CHECK([$SED -ne '/./p' stdout], [0], [[hello
 world
 ]])
@@ -1154,7 +1154,7 @@ world
 syscmd(`echo hi >&3')divert
 hello
 ]])
-AT_CHECK_M4([3>&-], [0], [stdout], [experr], [in.m4])
+AT_CHECK_M4([3>&-], [0], [stdout-nolog], [experr], [in.m4])
 AT_CHECK([$SED -ne '/./p' stdout], [0], [[hello
 world
 ]])
diff --git a/tests/freeze.at b/tests/freeze.at
index 338f6f5..c0a3399 100644
--- a/tests/freeze.at
+++ b/tests/freeze.at
@@ -30,18 +30,19 @@ AT_DATA([unfrozen.m4], [$3])

 # First generate the `expout' output by running over the sources before
 # freezing.
-AT_CHECK_M4([frozen.m4 unfrozen.m4], [0], [stdout], [stderr])
+AT_CHECK_M4([frozen.m4 unfrozen.m4], [0], [stdout-nolog], [stderr])

 mv stdout expout
 mv stderr experr

 # Now freeze the first source file.
-AT_CHECK_M4([-F frozen.m4f frozen.m4], [0], [stdout])
+AT_CHECK_M4([-F frozen.m4f frozen.m4], [0], [stdout-nolog])

 mv stdout out1

 # Now rerun the original sequence, but using the frozen file.
-AT_CHECK_M4([-R frozen.m4f unfrozen.m4], [0], [stdout], [experr], [], [ ])
+AT_CHECK_M4([-R frozen.m4f unfrozen.m4],
+            [0], [stdout-nolog], [experr], [], [ ])

 AT_CHECK([cat out1 stdout], [0], [expout])

diff --git a/tests/others.at b/tests/others.at
index e9b7a1b..f95d80f 100644
--- a/tests/others.at
+++ b/tests/others.at
@@ -578,7 +578,7 @@ hello`'dnl(world)
 undivert
 goodbye
 ]])
-AT_CHECK_M4([2>&-], [1], [stdout], [], [in.m4])
+AT_CHECK_M4([2>&-], [1], [stdout-nolog], [], [in.m4])
 AT_CHECK([$SED -ne '/./p' stdout], [0],
 [[hello
 and
diff --git a/tests/testsuite.at b/tests/testsuite.at
index 9372e92..7ba56e5 100644
--- a/tests/testsuite.at
+++ b/tests/testsuite.at
@@ -1,7 +1,8 @@
 # Process with autom4te to create an -*- Autotest -*- test suite.

 # Test suite for GNU M4.
-# Copyright (C) 2001, 2006, 2007, 2008 Free Software Foundation, Inc.
+# Copyright (C) 2001, 2006, 2007, 2008, 2009 Free Software Foundation,
+# Inc.

 # This file is part of GNU M4.
 #
@@ -19,7 +20,7 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.

 # We need a recent Autotest.
-m4_version_prereq([2.62])
+m4_version_prereq([2.63b-41])

 # Default to testing "m4 -b", but allow overrides.
 dnl This uses knowledge of undocumented autotest internals; hopefully
-- 
1.6.2.4