Re: texi2html tests are huge

[Stefano Lattarini]

Hi Patrice.

On 01/08/2012 08:15 PM, Patrice Dumas wrote:
> On Sat, Jan 07, 2012 at 02:27:35PM -0800, Karl Berry wrote:
>>
>> However, here's my much bigger concern with the tests: it takes too long
>> to run!  I doubt (m)any installers would be patient enough to sit
>> through make check.
> 
> What would be an amount of time you'd find reasonable for tests?  (The
> reference being a 2Ghz second, because it is what I have ;-).  For now
> it is 2m55 (tp only).
> 
>> So I wonder about having  make check  just run a subset of basic tests,
>> that everyone can do, and then have check-all to run them all.  
> 
> I have done something along that in texi2html/test/Makefile.am.
> 
>> Unfortunately, automake doesn't have any support for that.  So maybe
>> it's not feasible.  I haven't really thought about it.  Perhaps I should
>> raise the point with the automakers.
> 
> I simply did a copy and paste of the Makefile portion generated by automake
> relevant to tests...  A hack I am quite satisfied with ;-).  Even though
> it would be nice to have something in automake.
> 
I forward you the answer I've sent to Karl, since somehow it hasn't
make it to the mailing list yet ...

-*-*-*-

Hi Karl.

On 01/07/2012 11:27 PM, Karl Berry wrote:
> >
> > So I wonder about having  make check  just run a subset of basic tests,
> > that everyone can do, and then have check-all to run them all.
> >
> > Unfortunately, automake doesn't have any support for that.
> >
But it's very easy to roll your own by building on the automake-provided
test harness interface; for example:

  $ cat Makefile.am
  basic_TESTS = foo.test bar.test
  more_TESTS = zardoz.test
  # Only the basic tests will be run by "make check" by default.
  TESTS = $(basic_TESTS)
  # But make it easy for the developers to run all the tests with
  # a simple "make check-all".
  all_TESTS = $(basic_TESTS) $(more_TESTS)
  check-all: all
        $(MAKE) $(AM_MAKEFLAGS) TESTS='$(all_TESTS)' check

  $ autoreconf -vi && ./configure && make
  ...

  $ make  check-TESTS
  make[1]: Entering directory `/tmp/foo'
  PASS: foo.test
  PASS: bar.test
  ==================
  All 2 tests passed
  ==================
  make[1]: Leaving directory `/tmp/foo'

  $ make check-all
  make  TESTS='foo.test bar.test zardoz.test' check
  make[1]: Entering directory `/tmp/foo'
  make  check-TESTS
  make[2]: Entering directory `/tmp/foo'
  PASS: foo.test
  PASS: bar.test
  PASS: zardoz.test
  ==================
  All 3 tests passed
  ==================
  make[2]: Leaving directory `/tmp/foo'
  make[1]: Leaving directory `/tmp/foo'

The idiom works both with and without the `parallel-tests' option BTW (tested).

> > So maybe it's not feasible.  I haven't really thought about it.  Perhaps I
> > should raise the point with the automakers.
> >
HTH,
  Stefano