[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: testsuite -C dir
From: |
Eric Blake |
Subject: |
Re: testsuite -C dir |
Date: |
Mon, 28 Jan 2008 07:07:14 -0700 |
User-agent: |
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.9) Gecko/20071031 Thunderbird/2.0.0.9 Mnenhy/0.7.5.666 |
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
According to Paul Eggert on 1/27/2008 11:51 AM:
| Eric Blake <address@hidden> writes:
|
|> +2008-01-27 Eric Blake <address@hidden>
|> +
|> + Add 'testsuite -C dir'.
|
| Thanks for contributing that. It looks good to me.
Applied, after one more round of testing with m4 (there were some
additional impacts, like having to use $at_myself rather than $0 to
reinvoke the --list option of the testsuite to name failed tests in the
log). Also, in the process, I noticed that a standalone test (one where
atconfig is not present), the platform information was not captured in the
log; also the PATH sanitization was taking a long time on cygwin prior to
running the first test. Fixed as follows:
2008-01-28 Eric Blake <address@hidden>
Minor testsuite improvements.
* lib/autotest/general.m4 (AT_INIT) <PREPARE_TESTS>: Use fewer
forks when sanitizing PATH. Always output machine information,
not just when atconfig was located.
- --
Don't work too hard, make some time for fun as well!
Eric Blake address@hidden
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFHneGR84KuGfSFAYARAg/0AJsHZONUB4l2NMDxYgI+hXIznQlorwCeO9P/
HZDs6R1z1NYOpbZZ3//L8yU=
=pFb/
-----END PGP SIGNATURE-----
>From b91a600e7f2d9c72ab9865df88ec8e9a9aa02f1e Mon Sep 17 00:00:00 2001
From: Eric Blake <address@hidden>
Date: Mon, 28 Jan 2008 06:57:46 -0700
Subject: [PATCH] Add 'testsuite -C dir'.
* lib/autotest/general.m4 (_AT_ARG_OPTION): Move missing argument
detection...
(AT_INIT) <PARSE_ARGS_END>: ...here, since -k always takes
argument.
<DEFAULTS>: Delay computation of variables based on $at_dir...
<PREPARE_TESTS>: ...to here, since -C can change $at_dir.
<TESTS>: Re-invoke via absolute name, since -C may be in effect.
<PARSE_ARGS>: Parse new option.
<HELP_TUNING>: Document it.
* tests/autotest.at (Choosing where testsuite is run): New test
for this feature.
(Keywords and ranges): Add test for missing -k argument.
* NEWS: Document this.
* doc/autoconf.texi (testsuite Invocation): Likewise.
Signed-off-by: Eric Blake <address@hidden>
---
ChangeLog | 18 ++++++
NEWS | 3 +
doc/autoconf.texi | 6 ++
lib/autotest/general.m4 | 136 +++++++++++++++++++++++++++--------------------
tests/autotest.at | 28 +++++++++-
5 files changed, 133 insertions(+), 58 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 27fb894..49cfa45 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,21 @@
+2008-01-28 Eric Blake <address@hidden>
+
+ Add 'testsuite -C dir'.
+ * lib/autotest/general.m4 (_AT_ARG_OPTION): Move missing argument
+ detection...
+ (AT_INIT) <PARSE_ARGS_END>: ...here, since -k always takes
+ argument.
+ <DEFAULTS>: Delay computation of variables based on $at_dir...
+ <PREPARE_TESTS>: ...to here, since -C can change $at_dir.
+ <TESTS>: Re-invoke via absolute name, since -C may be in effect.
+ <PARSE_ARGS>: Parse new option.
+ <HELP_TUNING>: Document it.
+ * tests/autotest.at (Choosing where testsuite is run): New test
+ for this feature.
+ (Keywords and ranges): Add test for missing -k argument.
+ * NEWS: Document this.
+ * doc/autoconf.texi (testsuite Invocation): Likewise.
+
2008-01-24 Ralf Wildenhues <address@hidden>
* build-aux/config.guess, build-aux/config.sub,
diff --git a/NEWS b/NEWS
index ba278ad..855ed52 100644
--- a/NEWS
+++ b/NEWS
@@ -81,6 +81,9 @@ GNU Autoconf NEWS - User visible changes.
** Autotest now determines $srcdir correctly.
+** Testsuites built by autotest now accept a -C/--directory=DIR option
+ to adjust the working directory prior to creating files.
+
** Autoconf now requires GNU M4 1.4.5 or later. Earlier versions of M4 have
a bug in macro tracing that interferes with the interaction between
Autoconf and Automake. GNU M4 1.4.8 or later is recommended. The
diff --git a/doc/autoconf.texi b/doc/autoconf.texi
index 78628e5..59e6f97 100644
--- a/doc/autoconf.texi
+++ b/doc/autoconf.texi
@@ -20191,6 +20191,12 @@ Display the list of options and exit successfully.
@itemx -V
Display the version of the test suite and exit successfully.
address@hidden address@hidden
address@hidden -C @var{dir}
+Change the current directory to @var{dir} before creating any files.
+Useful for running the testsuite in a subdirectory from a top-level
+Makefile.
+
@item --clean
@itemx -c
Remove all the files the test suite might have created and exit. Meant
diff --git a/lib/autotest/general.m4 b/lib/autotest/general.m4
index 11ff925..c1101ad 100644
--- a/lib/autotest/general.m4
+++ b/lib/autotest/general.m4
@@ -1,8 +1,8 @@
# This file is part of Autoconf. -*- Autoconf -*-
# M4 macros used in building test suites.
-# Copyright (C) 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007 Free
-# Software Foundation, Inc.
+# Copyright (C) 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008
+# Free Software Foundation, Inc.
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -195,8 +195,8 @@ m4_define([AT_help_all], [])
AS_INIT[]dnl
m4_divert_push([DEFAULTS])dnl
AT_COPYRIGHT(
-[Copyright (C) 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007 Free Software
-Foundation, Inc.
+[Copyright (C) 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008
+Free Software Foundation, Inc.
This test suite is free software; the Free Software Foundation gives
unlimited permission to copy, distribute and modify it.])
AS_PREPARE
@@ -361,31 +361,6 @@ fi
## ---------------------- ##
m4_divert_pop([PREPARE_TESTS])dnl back to DEFAULTS
-# Load the config file.
-for at_file in atconfig atlocal
-do
- test -r $at_file || continue
- . ./$at_file || AS_ERROR([invalid content: $at_file])
-done
-
-# Autoconf <=2.59b set at_top_builddir instead of at_top_build_prefix:
-: ${at_top_build_prefix=$at_top_builddir}
-
-# atconfig delivers names relative to the directory the test suite is
-# in, but the groups themselves are run in testsuite-dir/group-dir.
-if test -n "$at_top_srcdir"; then
- builddir=../..
- for at_dir in srcdir top_srcdir top_build_prefix
- do
- at_val=AS_VAR_GET([at_$at_dir])
- case $at_val in
- [[\\/$]]* | ?:[[\\/]]* ) at_prefix= ;;
- *) at_prefix=../../ ;;
- esac
- AS_VAR_SET([$at_dir], [$at_prefix$at_val])
- done
-fi
-
# Not all shells have the 'times' builtin; the subshell is needed to make
# sure we discard the 'times: not found' message from the shell.
at_times_p=false
@@ -413,7 +388,7 @@ at_groups=
# Whether a write failure occurred
at_write_fail=0
-# The directory we are in.
+# The directory we run the suite in. Default to . if no -C option.
at_dir=`pwd`
# An absolute reference to this testsuite script.
dnl m4-double quote, to preserve []
@@ -421,23 +396,8 @@ dnl m4-double quote, to preserve []
[\\/]* | ?:[\\/]* ) at_myself=$as_myself ;;
* ) at_myself=$at_dir/$as_myself ;;
esac]
-# The directory the whole suite works in.
-# Should be absolute to let the user `cd' at will.
-at_suite_dir=$at_dir/$as_me.dir
-# The file containing the suite.
-at_suite_log=$at_dir/$as_me.log
-# The file containing the location of the last AT_CHECK.
-at_check_line_file=$at_suite_dir/at-check-line
-# The file containing the exit status of the last command.
-at_status_file=$at_suite_dir/at-status
-# The files containing the output of the tested commands.
-at_stdout=$at_suite_dir/at-stdout
-at_stder1=$at_suite_dir/at-stder1
-at_stderr=$at_suite_dir/at-stderr
-# The file containing the function to run a test group.
-at_test_source=$at_suite_dir/at-test-source
-# The file containing dates.
-at_times_file=$at_suite_dir/at-times
+# Whether -C is in effect.
+at_change_dir=false
m4_divert_pop([DEFAULTS])dnl
m4_wrap([m4_divert_text([DEFAULTS],
[
@@ -563,6 +523,15 @@ do
at_groups="$at_groups$at_range "
;;
+ # Directory selection.
+ --directory | -C )
+ at_prev=--directory
+ ;;
+ --directory=* )
+ at_change_dir=:
+ at_dir=$at_optarg
+ ;;
+
# Keywords.
--keywords | -k )
at_prev=--keywords
@@ -616,6 +585,9 @@ m4_divert_push([PARSE_ARGS_END])dnl
esac
done
+# Verify our last option didn't require an argument
+AS_IF([test -n "$at_prev"], [AS_ERROR([`$at_prev' requires an argument.])])
+
# Selected test groups.
if test -z "$at_groups"; then
at_groups=$at_groups_all
@@ -661,6 +633,8 @@ cat <<_ATEOF || at_write_fail=1
dnl extra quoting prevents emacs whitespace mode from putting tabs in output
Execution tuning:
+ -C, --directory=DIR
+[ change to directory DIR before starting]
-k, --keywords=KEYWORDS
[ select the tests matching all the comma-separated KEYWORDS]
[ multiple \`-k' accumulate; prefixed \`!' negates a KEYWORD]
@@ -724,9 +698,63 @@ fi
m4_divert_pop([VERSION_END])dnl
m4_divert_push([PREPARE_TESTS])dnl
+# Take any -C into account.
+if $at_change_dir ; then
+ if test x- = x$"at_dir" ; then
+ at_dir=./-
+ fi
+ test x != x"$at_dir" && cd "$at_dir" \
+ || AS_ERROR([unable to change directory])
+ at_dir=`pwd`
+fi
+
+# Load the config file.
+for at_file in atconfig atlocal
+do
+ test -r $at_file || continue
+ . $at_file || AS_ERROR([invalid content: $at_file])
+done
+
+# Autoconf <=2.59b set at_top_builddir instead of at_top_build_prefix:
+: ${at_top_build_prefix=$at_top_builddir}
+
+# atconfig delivers names relative to the directory the test suite is
+# in, but the groups themselves are run in testsuite-dir/group-dir.
+if test -n "$at_top_srcdir"; then
+ builddir=../..
+ for at_dir_var in srcdir top_srcdir top_build_prefix
+ do
+ at_val=AS_VAR_GET([at_$at_dir_var])
+ case $at_val in
+ [[\\/$]]* | ?:[[\\/]]* ) at_prefix= ;;
+ *) at_prefix=../../ ;;
+ esac
+ AS_VAR_SET([$at_dir_var], [$at_prefix$at_val])
+ done
+fi
+
+# The directory the whole suite works in.
+# Should be absolute to let the user `cd' at will.
+at_suite_dir=$at_dir/$as_me.dir
+# The file containing the suite.
+at_suite_log=$at_dir/$as_me.log
+# The file containing the location of the last AT_CHECK.
+at_check_line_file=$at_suite_dir/at-check-line
+# The file containing the exit status of the last command.
+at_status_file=$at_suite_dir/at-status
+# The files containing the output of the tested commands.
+at_stdout=$at_suite_dir/at-stdout
+at_stder1=$at_suite_dir/at-stder1
+at_stderr=$at_suite_dir/at-stderr
+# The file containing the function to run a test group.
+at_test_source=$at_suite_dir/at-test-source
+# The file containing dates.
+at_times_file=$at_suite_dir/at-times
+
# Don't take risks: use only absolute directories in PATH.
#
-# For stand-alone test suites, AUTOTEST_PATH is relative to `.'.
+# For stand-alone test suites (ie. atconfig was not found),
+# AUTOTEST_PATH is relative to `.'.
#
# For embedded test suites, AUTOTEST_PATH is relative to the top level
# of the package. Then expand it into build/src parts, since users
@@ -1145,17 +1173,17 @@ else
# Summary of failed and skipped tests.
if test $at_fail_count != 0; then
echo "Failed tests:"
- $SHELL "$[0]" $at_fail_list --list
+ $SHELL "$at_myself" $at_fail_list --list
echo
fi
if test $at_skip_count != 0; then
echo "Skipped tests:"
- $SHELL "$[0]" $at_skip_list --list
+ $SHELL "$at_myself" $at_skip_list --list
echo
fi
if test $at_xpass_count != 0; then
echo "Unexpected passes:"
- $SHELL "$[0]" $at_xpass_list --list
+ $SHELL "$at_myself" $at_xpass_list --list
echo
fi
if test $at_fail_count != 0; then
@@ -1249,12 +1277,6 @@ m4_divert_text([PARSE_ARGS_BEGIN],
m4_ifvaln([$3],,[at_arg_[]m4_bpatsubst([AT_first_option], -, _)=false])dnl
at_arg_given_[]m4_bpatsubst([AT_first_option], -, _)=false
])dnl m4_divert_text DEFAULTS
-m4_ifval([$3],[m4_divert_once([PARSE_ARGS_END],
-[
-##
-## Verify our last option didn't require an argument
-##
-AS_IF([test -n "$at_prev"],[AS_ERROR([`$at_prev' requires an argument.])])])])
m4_divert_text([PARSE_ARGS],
[dnl Parse the options and args when necessary.
m4_ifvaln([$3],
diff --git a/tests/autotest.at b/tests/autotest.at
index e043d7a..4fbdba2 100644
--- a/tests/autotest.at
+++ b/tests/autotest.at
@@ -2,7 +2,8 @@
AT_BANNER([Autotest.])
-# Copyright (C) 2004, 2005, 2006, 2007 Free Software Foundation, Inc.
+# Copyright (C) 2004, 2005, 2006, 2007, 2008 Free Software Foundation,
+# Inc.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -395,6 +396,29 @@ AT_CHECK([(cd micro-suite.dir/1 && ./run MY_VAR='one
space')],
])
+# Controlling where the testsuite is run.
+AT_CHECK_AT_TEST([Choosing where testsuite is run],
+ [AT_CHECK([:])], [], [], [], [], [
+dnl AT_CHECK_AT_TEST tests the default of running in `.'.
+AT_CHECK([rm micro-suite.log && mkdir sub1 sub2])
+dnl check specifying a different relative path to run in.
+AT_CHECK([./micro-suite -C sub1], [0], [ignore], [])
+AT_CHECK([test -f micro-suite.log], [1])
+AT_CHECK([test -f sub1/micro-suite.log], [0])
+dnl check specifying an absolute path to run in.
+AT_CHECK([./micro-suite --directory="`pwd`/sub2"], [0], [ignore], [])
+AT_CHECK([test -f micro-suite.log], [1])
+AT_CHECK([test -f sub2/micro-suite.log], [0])
+dnl check for failure detection with bad, missing, or empty directory.
+AT_CHECK([./micro-suite -C nonesuch], [1], [ignore], [ignore])
+AT_CHECK([./micro-suite -C ''], [1], [ignore], [ignore])
+AT_CHECK([./micro-suite -C -], [1], [ignore], [ignore])
+AT_CHECK([./micro-suite -C], [1], [ignore], [ignore])
+dnl check that --help overrides bad directory selection.
+AT_CHECK([./micro-suite -C nonesuch --help], [0], [ignore], [])
+])
+
+
## -------- ##
## Banners. ##
## -------- ##
@@ -491,6 +515,8 @@ AT_CLEANUP
AT_CHECK_AUTOM4TE([--language=autotest -o k k.at])
dnl check that AT_KEYWORDS does not duplicate words
AT_CHECK([grep 'key1.*key1' k], [1])
+dnl check that -k requires an argument
+AT_CHECK([./k -k], [1], [], [ignore])
# AT_CHECK_KEYS(TESTSUITE-OPTIONS, PATTERN1, COUNT1, PATTERN2, COUNT2)
m4_define([AT_CHECK_KEYS],
--
1.5.3.8
>From c6c22af0e79a7f08c150f6dc59841be914dfa66d Mon Sep 17 00:00:00 2001
From: Eric Blake <address@hidden>
Date: Mon, 28 Jan 2008 07:02:43 -0700
Subject: [PATCH] Minor testsuite improvements.
* lib/autotest/general.m4 (AT_INIT) <PREPARE_TESTS>: Use fewer
forks when sanitizing PATH. Always output machine information,
not just when atconfig was located.
Signed-off-by: Eric Blake <address@hidden>
---
ChangeLog | 5 +++++
lib/autotest/general.m4 | 12 ++++++++----
2 files changed, 13 insertions(+), 4 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 49cfa45..b8e7fb0 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,10 @@
2008-01-28 Eric Blake <address@hidden>
+ Minor testsuite improvements.
+ * lib/autotest/general.m4 (AT_INIT) <PREPARE_TESTS>: Use fewer
+ forks when sanitizing PATH. Always output machine information,
+ not just when atconfig was located.
+
Add 'testsuite -C dir'.
* lib/autotest/general.m4 (_AT_ARG_OPTION): Move missing argument
detection...
diff --git a/lib/autotest/general.m4 b/lib/autotest/general.m4
index c1101ad..662f2cb 100644
--- a/lib/autotest/general.m4
+++ b/lib/autotest/general.m4
@@ -785,8 +785,11 @@ esac])
# builtins' (eg., cd) stderr directly: Ultrix's sh hates that.
at_new_path=
_AS_PATH_WALK([$at_path],
-[as_dir=`(cd "$as_dir" && pwd) 2>/dev/null`
-test -d "$as_dir" || continue
+[test -d "$as_dir" || continue
+case $as_dir in
+ [[\\/]]* | ?:[[\\/]]* ) ;;
+ * ) as_dir=`(cd "$as_dir" && pwd) 2>/dev/null` ;;
+esac
case $PATH_SEPARATOR$at_new_path$PATH_SEPARATOR in
*$PATH_SEPARATOR$as_dir$PATH_SEPARATOR*) ;;
$PATH_SEPARATOR$PATH_SEPARATOR) at_new_path=$as_dir ;;
@@ -832,10 +835,11 @@ AS_BOX(m4_defn([AT_TESTSUITE_NAME])[.])
echo
done
- AS_UNAME
- echo
fi
+ AS_UNAME
+ echo
+
# Contents of the config files.
for at_file in atconfig atlocal
do
--
1.5.3.8
- testsuite -C dir, Eric Blake, 2008/01/26
- Re: testsuite -C dir, Paolo Bonzini, 2008/01/27
- Re: testsuite -C dir, Paul Eggert, 2008/01/27
- Re: testsuite -C dir,
Eric Blake <=
- Re: testsuite -C dir, Ralf Wildenhues, 2008/01/28
- Re: testsuite -C dir, Eric Blake, 2008/01/28
- Re: testsuite -C dir, Ralf Wildenhues, 2008/01/28
- Re: testsuite -C dir, Paolo Bonzini, 2008/01/28
- Re: testsuite -C dir, Eric Blake, 2008/01/29
- Re: testsuite -C dir, Paolo Bonzini, 2008/01/29
- Re: testsuite -C dir, Eric Blake, 2008/01/29
- Re: testsuite -C dir, Paolo Bonzini, 2008/01/29
- Re: testsuite -C dir, Eric Blake, 2008/01/29
- Re: testsuite -C dir, Eric Blake, 2008/01/29