Re: [PATCH 3/3] {testsuite-work} tests: `lib/' shell scripts transparently tested also with $SHELL

[Stefano Lattarini]

On Tuesday 07 June 2011, Peter Rosin wrote:
> Den 2011-06-06 21:49 skrev Stefano Lattarini:
> > With the previous commit, the user could prefer the use of $SHELL
> > over /bin/sh in some tests checking the Automake-provided shell
> > scripts by manually exporting `test_prefer_config_shell' to "yes"
> > in the environment.  With this commit, we ensure that such tests
> > *always* and *transparently* run using both $SHELL and /bin/sh
> > to execute the checked scripts.  The `test_prefer_config_shell'
> > variable becomes an internal detail, and is no more meant to be
> > manually defined or overridden.
> > 
> > * tests/defs-static.in: Check that `test_prefer_config_shell' is
> > not exported in the environment.  Error out if this is the case.
> > * tests/config-shell-tests.sh: New file, driver script to run
> > checks on the shell scripts in `lib/' using the $SHELL determined
> > at configure time instead of the default system shell /bin/sh.
> > * Makefile.am (TESTS_EXTENSIONS): Add `.shtst'.
> > (SHTST_LOG_COMPILER): Define, it calls `config-shell-tests.sh'.
> > (config_shell_tests): Define to a list of tests that wraps other
> > `*.test' tests using `config-shell-tests.sh'.
> > ($(config_shell_tests)): Dummy dependency declaration required
> > in order to have make actually produce expected log files from
> > the `.shtst.log' suffix rule.
> > (EXTRA_DIST): Distribute `config-shell-tests.sh'.
> > (TESTS): Add `$(config_shell_tests)'.
> > ---
> >  ChangeLog                   |   26 +++++++++++++++++++++
> >  tests/Makefile.am           |   29 +++++++++++++++++++++++
> >  tests/Makefile.in           |   40 +++++++++++++++++++++++++++++---
> >  tests/config-shell-tests.sh |   53 
> > +++++++++++++++++++++++++++++++++++++++++++
> >  tests/defs-static.in        |    4 ++-
> >  5 files changed, 147 insertions(+), 5 deletions(-)
> >  create mode 100755 tests/config-shell-tests.sh
> > 
> > diff --git a/ChangeLog b/ChangeLog
> > index 2254d3b..8fa1cb7 100644
> > --- a/ChangeLog
> > +++ b/ChangeLog
> > @@ -1,5 +1,31 @@
> >  2011-06-06  Stefano Lattarini  <address@hidden>
> >  
> > +   tests: `lib/' shell scripts transparently tested also with $SHELL
> > +   With the previous commit, the user could prefer the use of $SHELL
> > +   over /bin/sh in some tests checking the Automake-provided shell
> > +   scripts by manually exporting `test_prefer_config_shell' to "yes"
> > +   in the environment.  With this commit, we ensure that such tests
> > +   *always* and *transparently* run using both $SHELL and /bin/sh
> > +   to execute the checked scripts.  The `test_prefer_config_shell'
> > +   variable becomes an internal detail, and is no more meant to be
> > +   manually defined or overridden.
> > +   * tests/defs-static.in: Check that `test_prefer_config_shell' is
> > +   not exported in the environment.  Error out if this is the case.
> > +   * tests/config-shell-tests.sh: New file, driver script to run
> > +   checks on the shell scripts in `lib/' using the $SHELL determined
> > +   at configure time instead of the default system shell /bin/sh.
> > +   * Makefile.am (TESTS_EXTENSIONS): Add `.shtst'.
> > +   (SHTST_LOG_COMPILER): Define, it calls `config-shell-tests.sh'.
> > +   (config_shell_tests): Define to a list of tests that wraps other
> > +   `*.test' tests using `config-shell-tests.sh'.
> > +   ($(config_shell_tests)): Dummy dependency declaration required
> > +   in order to have make actually produce expected log files from
> > +   the `.shtst.log' suffix rule.
> > +   (EXTRA_DIST): Distribute `config-shell-tests.sh'.
> > +   (TESTS): Add `$(config_shell_tests)'.
> > +
> > +2011-06-06  Stefano Lattarini  <address@hidden>
> > +
> >     tests: can use also $SHELL to check shell scripts from `lib/'
> >     * tests/ar-lib.test: If the variable `$test_prefer_config_shell'
> >     is set to "yes", run the script under test with configure-time
> > diff --git a/tests/Makefile.am b/tests/Makefile.am
> > index d863b28..339f801 100644
> > --- a/tests/Makefile.am
> > +++ b/tests/Makefile.am
> > @@ -93,6 +93,34 @@ $(depmod_tests:.depmod=.log): depmod-tests.sh 
> > depmod-data.log
> >  $(depmod_tests):
> >  
> >  
> > +TEST_EXTENSIONS += .shtst
> > +SHTST_LOG_COMPILER = $(SHELL) $(srcdir)/config-shell-tests.sh
> > +EXTRA_DIST += config-shell-tests.sh
> > +
> > +# FIXME: this list might probably be made auto-generated, like
> > +# $(parallel_tests) is...  But would that worth the hassle?
> > +config_shell_tests = \
> > +  compile-w.shtst \
> > +  compile2-w.shtst \
> > +  compile3-w.shtst \
> > +  compile4-w.shtst \
> > +  compile5-w.shtst \
> > +  compile6-w.shtst \
> > +  instsh2-w.shtst \
> > +  instsh3-w.shtst \
> > +  mkinst3-w.shtst \
> > +  missing-w.shtst \
> > +  missing2-w.shtst \
> > +  missing3-w.shtst \
> > +  missing5-w.shtst
> 
> I expected to see the ar-lib tests here? Or did you not add those
> since the AM_PROG_AR macro hasn't been merged? But would it hurt
> to test ar-lib with $SHELL prior to that merge?
>
No, this is just a stupid oversight on my part, sorry.  Which probably
shows that we indeed should make this list automatically-generated
after all.  But that's for a follow-up patch IMHO.

Thanks,
  Stefano