Re: [PATCH 2/3] {testsuite-work} tests: can use also $SHELL to check shell scripts from `lib/'

[Peter Rosin]

Den 2011-06-06 21:48 skrev Stefano Lattarini:
> * tests/ar-lib.test: If the variable `$test_prefer_config_shell'
> is set to "yes", run the script under test with configure-time
> determined $SHELL, rather than with /bin/sh.
> The `$test_prefer_config_shell' variable defaults to empty, but
> can be overridden at runtime by the user, thus allowing more
> coverage.
> * tests/compile.test: Likewise.
> * tests/compile2.test: Likewise.
> * tests/compile3.test: Likewise.
> * tests/compile4.test: Likewise.
> * tests/compile5.test: Likewise.
> * tests/compile6.test: Likewise.
> * tests/instsh2.test: Likewise.
> * tests/instsh3.test: Likewise.
> * tests/mkinst3.test: Likewise.
> * tests/missing.test: Likewise.
> * tests/missing2.test: Likewise.
> * tests/missing3.test: Likewise.
> * tests/missing5.test: Likewise.
> * tests/defs (get_shell_script): New subroutine, factoring out
> code common to the tests above.
> (xsi-lib-shell): If `$am_prefer_config_shell' is set to "yes",

$test_prefer_config_shell

> check that $SHELL, not /bin/sh, supports XSI constructs, as we
> expect the test will use $SHELL and not /bin/sh to run the
> script being tested.
> ---
>  ChangeLog           |   29 +++++++++++++++++++++++++++++
>  tests/ar-lib.test   |    2 +-
>  tests/compile.test  |    2 +-
>  tests/compile2.test |    2 +-
>  tests/compile3.test |    2 +-
>  tests/compile4.test |    2 ++
>  tests/compile5.test |    2 +-
>  tests/compile6.test |    2 +-
>  tests/defs          |   24 +++++++++++++++++++++++-
>  tests/instsh2.test  |    3 +++
>  tests/instsh3.test  |    2 ++
>  tests/missing.test  |    2 ++
>  tests/missing2.test |    2 ++
>  tests/missing3.test |    2 ++
>  tests/missing5.test |    2 ++
>  tests/mkinst3.test  |    2 +-
>  16 files changed, 74 insertions(+), 8 deletions(-)
> 
> diff --git a/ChangeLog b/ChangeLog
> index 35e18d0..2254d3b 100644
> --- a/ChangeLog
> +++ b/ChangeLog
> @@ -1,5 +1,34 @@
>  2011-06-06  Stefano Lattarini  <address@hidden>
>  
> +     tests: can use also $SHELL to check shell scripts from `lib/'
> +     * tests/ar-lib.test: If the variable `$test_prefer_config_shell'
> +     is set to "yes", run the script under test with configure-time
> +     determined $SHELL, rather than with /bin/sh.
> +     The `$test_prefer_config_shell' variable defaults to empty, but
> +     can be overridden at runtime by the user, thus allowing more
> +     coverage.
> +     * tests/compile.test: Likewise.
> +     * tests/compile2.test: Likewise.
> +     * tests/compile3.test: Likewise.
> +     * tests/compile4.test: Likewise.
> +     * tests/compile5.test: Likewise.
> +     * tests/compile6.test: Likewise.
> +     * tests/instsh2.test: Likewise.
> +     * tests/instsh3.test: Likewise.
> +     * tests/mkinst3.test: Likewise.
> +     * tests/missing.test: Likewise.
> +     * tests/missing2.test: Likewise.
> +     * tests/missing3.test: Likewise.
> +     * tests/missing5.test: Likewise.
> +     * tests/defs (get_shell_script): New subroutine, factoring out
> +     code common to the tests above.
> +     (xsi-lib-shell): If `$test_prefer_config_shell' is set to "yes",
> +     check that $SHELL, not /bin/sh, supports XSI constructs, as we
> +     expect the test will use $SHELL and not /bin/sh to run the
> +     script being tested.
> +
> +2011-06-06  Stefano Lattarini  <address@hidden>
> +
>       tests defs: better requirements for XSI shells
>       This change avoids potential spurious failures with tests using
>       the requirement 'xsi-shell' to mean that they want */bin/sh* (not
> diff --git a/tests/ar-lib.test b/tests/ar-lib.test
> index 9936c0e..6f6625d 100755
> --- a/tests/ar-lib.test
> +++ b/tests/ar-lib.test
> @@ -19,7 +19,7 @@
>  required=xsi-lib-shell
>  . ./defs || Exit 1
>  
> -cp "$top_testsrcdir/lib/ar-lib" .
> +get_shell_script ar-lib
>  
>  # Use a dummy lib, since lib isn't readily available on all systems.
>  cat >lib <<'END'
> diff --git a/tests/compile.test b/tests/compile.test
> index 8427bad..0ba9ddf 100755
> --- a/tests/compile.test
> +++ b/tests/compile.test
> @@ -18,7 +18,7 @@
>  
>  . ./defs || Exit 1
>  
> -cp "$top_testsrcdir/lib/compile" .
> +get_shell_script compile
>  
>  # -o 'a  c' should not be stripped because 'a  c' is not an object
>  # (it does not matter whether touch creates ./-- or not)
> diff --git a/tests/compile2.test b/tests/compile2.test
> index a466dc4..180fcc3 100755
> --- a/tests/compile2.test
> +++ b/tests/compile2.test
> @@ -18,7 +18,7 @@
>  
>  . ./defs || Exit 1
>  
> -cp "$top_testsrcdir/lib/compile" .
> +get_shell_script compile
>  
>  cat >mycc <<'END'
>  source_seen=no
> diff --git a/tests/compile3.test b/tests/compile3.test
> index 61cde6b..b703f66 100755
> --- a/tests/compile3.test
> +++ b/tests/compile3.test
> @@ -19,7 +19,7 @@
>  required=xsi-lib-shell
>  . ./defs || Exit 1
>  
> -cp "$top_testsrcdir/lib/compile" .
> +get_shell_script compile
>  
>  # Use a dummy cl, since cl isn't readily available on all systems
>  cat >cl <<'END'
> diff --git a/tests/compile4.test b/tests/compile4.test
> index cf8d6cb..c7e8a0e 100755
> --- a/tests/compile4.test
> +++ b/tests/compile4.test
> @@ -20,6 +20,8 @@
>  required='cl'
>  . ./defs || Exit 1
>  
> +get_shell_script compile
> +

This test no longer checks if $AUTOMAKE -a copies over compile, as
that is done manually now. I assume this aspect of $AUTOMAKE -a is
tested elsewhere. Or is it? More below...

>  mkdir sub
>  
>  cat >sub/foo.c <<'EOF'
> diff --git a/tests/compile5.test b/tests/compile5.test
> index cd8baee..2b4eeb0 100755
> --- a/tests/compile5.test
> +++ b/tests/compile5.test
> @@ -19,7 +19,7 @@
>  
>  . ./defs || Exit 1
>  
> -cp "$top_testsrcdir/lib/compile" .
> +get_shell_script compile
>  
>  # Use a dummy cl, since cl isn't readily available on all systems
>  cat >cl <<'END'
> diff --git a/tests/compile6.test b/tests/compile6.test
> index ca1c671..fff1487 100755
> --- a/tests/compile6.test
> +++ b/tests/compile6.test
> @@ -19,7 +19,7 @@
>  required=xsi-lib-shell
>  . ./defs || Exit 1
>  
> -cp "$top_testsrcdir/lib/compile" .
> +get_shell_script compile
>  
>  # Use a dummy cl, since cl isn't readily available on all systems
>  cat >cl <<'END'
> diff --git a/tests/defs b/tests/defs
> index 37b5baa..1d50b1d 100644
> --- a/tests/defs
> +++ b/tests/defs
> @@ -283,6 +283,23 @@ unindent ()
>  }
>  sed_unindent_prog="" # Avoid interferences from the environment.
>  
> +# get_shell_script SCRIPT-NAME
> +# -----------------------------
> +# Fetch an Automake-provided test script from the `lib/' directory into
> +# the current directory, and, if the `$test_prefer_config_shell' variable
> +# is set to "yes", modify its shebang line to use $SHELL instead of
> +# /bin/sh.
> +get_shell_script ()
> +{
> +  if test x"$test_prefer_config_shell" = x"yes"; then
> +    sed "1s|#!.*|#! $SHELL|" "$top_testsrcdir/lib/$1" > "$1"
> +    chmod a+x "$1"
> +  else
> +    cp "$top_testsrcdir/lib/$1" .
> +  fi
> +  sed 10q "$1" # For debugging.
> +}
> +
>  # require_xsi SHELL
>  # -----------------
>  # Skip the test is the given shell fails to support common XSI constructs.
> @@ -337,7 +354,12 @@ do
>      xsi-bin-sh)
>        require_xsi "/bin/sh";;
>      xsi-lib-shell)
> -      require_xsi "/bin/sh";;
> +      if test x"$test_prefer_config_shell" = x"yes"; then
> +        require_xsi "$SHELL"
> +      else
> +        require_xsi "/bin/sh"
> +      fi
> +      ;;
>      bzip2)
>        # Do not use --version, bzip2 still tries to compress stdin.
>        echo "$me: running bzip2 --help"
> diff --git a/tests/instsh2.test b/tests/instsh2.test
> index 2019b35..f86c414 100755
> --- a/tests/instsh2.test
> +++ b/tests/instsh2.test
> @@ -18,6 +18,9 @@
>  # Various install-sh checks
>  
>  . ./defs || Exit 1
> +
> +get_shell_script install-sh
> +
>  # Basic errors
>  ./install-sh && Exit 1
>  ./install-sh -m 644 dest && Exit 1
> diff --git a/tests/instsh3.test b/tests/instsh3.test
> index da50419..2cdf4a9 100755
> --- a/tests/instsh3.test
> +++ b/tests/instsh3.test
> @@ -23,6 +23,8 @@ required=non-root
>  touch -t $old_timestamp foo \
>    || skip_ "touch utility doesn't accept '-t' option"
>  
> +get_shell_script install-sh
> +
>  ./install-sh -d d1
>  
>  # Do not change the timestamps when using -C.
> diff --git a/tests/missing.test b/tests/missing.test
> index 2e6d8d3..8e080eb 100755
> --- a/tests/missing.test
> +++ b/tests/missing.test
> @@ -25,6 +25,8 @@ EOF
>  
>  : > Makefile.am
>  
> +get_shell_script missing
> +
>  $ACLOCAL
>  $AUTOCONF
>  $AUTOMAKE --add-missing
> diff --git a/tests/missing2.test b/tests/missing2.test
> index 88e6b27..9844958 100755
> --- a/tests/missing2.test
> +++ b/tests/missing2.test
> @@ -28,6 +28,8 @@ EOF
>  
>  : > Makefile.am
>  
> +get_shell_script missing
> +
>  $ACLOCAL
>  $AUTOCONF
>  $AUTOMAKE --add-missing
> diff --git a/tests/missing3.test b/tests/missing3.test
> index 21b5e4f..328212c 100755
> --- a/tests/missing3.test
> +++ b/tests/missing3.test
> @@ -18,6 +18,8 @@
>  
>  . ./defs || Exit 1
>  
> +get_shell_script missing
> +
>  # b7cb8259 assumed not to exist.
>  
>  ./missing b7cb8259 --version 2>stderr && { cat stderr >&2; Exit 1; }
> diff --git a/tests/missing5.test b/tests/missing5.test
> index 010b344..b618bd1 100755
> --- a/tests/missing5.test
> +++ b/tests/missing5.test
> @@ -19,6 +19,8 @@
>  
>  . ./defs || Exit 1
>  
> +get_shell_script missing
> +
>  # these programs may be invoked by `missing'
>  needed_tools='chmod find sed test touch'
>  needed_tools_csep=`echo $needed_tools | sed 's/ /, /g'`
> diff --git a/tests/mkinst3.test b/tests/mkinst3.test
> index 0082b53..3572ab2 100755
> --- a/tests/mkinst3.test
> +++ b/tests/mkinst3.test
> @@ -23,7 +23,7 @@ mkdir '~a b' && mkdir '~a b/-x  y' \
>    || skip_ "directory names with spaces and metacharacters not accepted"
>  rm -rf '~a b'
>  
> -cp "$top_testsrcdir/lib/mkinstalldirs" .
> +get_shell_script mkinstalldirs
>  
>  # Test mkinstalldirs with the installed mkdir.
>  

I'm not too fond of rewriting the scripts. Wouldn't it be better
to execute them as they would be executed from Makefiles instead,
and tweak $SHELL for the case where the shebang is desired to
take effect?

I.e.
if "$test_prefer_config_shell" = yes; then
  TEST_SHELL=$SHELL
else
  TEST_SHELL=
endif

and then
 
$TEST_SHELL ./compile ...

or something?

Doing it that way would also not remove the $AUTOMAKE -a tests.

Cheers,
Peter