Re: [PATCH 1/2] tests: make test runner a script, not a shell function

[Stefano Lattarini]

On Tuesday 21 June 2011, Ralf Wildenhues wrote:
> * Stefano Lattarini wrote on Mon, Jun 20, 2011 at 11:12:23PM CEST:
> > Maybe we should also say that using TESTS_ENVIRONMENT to define a custom
> > test runner is now not only strongly deprecated (as it already was I hope),
> 
> No it wasn't.
>
D'oh.  I'll be more explicit about that in the NEWS entry then.

> "test runner" is not a term I would recognize, btw.
>
Suggestion on how to improve it are welcome.  As usual, I couldn't come
up with a better or clearer term :-(

> > but also unsupported and not working anymore?
> 
> > Also, should I look for TESTS_ENVIRONMENT usages in google code search?
> > I was really hoping to spae myself the pain... ;-)
> 
> Not sure.  If anything, you can use regexes to avoid stuff you're not
> interested in.
> 
> > > and if not, is it possible to have a compatible one (without too much
> > > maintenance effort duplication)?  No need to go the effort right away.
> > >
> > Well, we could add a new option "old-parallel-tests" or something like
> > that, that causes the old code in 'check.am' (with few tweaks in order
> > to support the new parsing of `.log' files) to be used instead of the
> > 'test-driver' script.  By refactoring some code in handle_tests(), we
> > could ensure not to add any real complexity to the automake script
> > (w.r.t. to my patches at least); but the duplication between 'check.am'
> > and 'test-driver' will unavoidable IMHO.
> 
> Well, if we need to use a name, then the name should be for the new one.
> That way you can have full backward compatibility.
>
I'd rather give the user an incentive to move forward, by having him
retain the old semantics only if he really needs to.  BTW, for this to
really work in practice, we should IMHO add new macro(s) AM_IF_VERSION
and/or AM_IF_FEATURE that will allow determination of automake version
and/or support feaures at autoconf time.  It's about time to introduce
such macros, which will help us to make transitions to new semantics
while allowing the user to retain backward compatibility (with a little
more work).

> > > I hope that we do not have to maintain four or more such drivers.
> > >
> > Me too.  And I also think we should start deprecating the old "serial"
> > driver ASAP (i.e., in 1.12, as I think 1.11.2 is sadly  tooearly); but
> > this is for another thread, and another time.
> 
> Erm, there are quite a few packages out there that can only use the
> serial driver.  They require tests run in a specific order, or some
> tests run twice, or output not redirected.
> 
> I "fixed" Libtool a while ago, but the changed code is still quite
> buggy.  I don't think we can expect everyone to migrate.
>
Ouch.  Let's drop these thoughts of deprecation for the moment then.

Thanks,
  Stefano