Re: [PPL-devel] Re: Doing something just a bit more complex than `make check'

[Ralf Wildenhues]

* Roberto Bagnara wrote on Sat, Jan 07, 2006 at 06:12:06PM CET:
> Ralf Wildenhues wrote:
> >* Roberto Bagnara wrote on Wed, Oct 19, 2005 at 09:13:16PM CEST:
> >
> >>Instead, what I would like to have is to only say

> >>and then achieve the effect of (sorry for the pseudo-code)
> >>
> >>  for flags in $FLAGS_CHOICES
> >>  do
> >>     make check with CXXFLAGS="$flags"
> >>     force recompilation at the next iteration (e.g., by erasing 
> >>     executables)
> >>  done
> >>
> >>How can I best obtain this effect without giving up (too many of)
> >>the advantages offered by Automake?

> thank you very much for your help (and sorry for the delay, but I wanted
> to make sure we did all our homework before getting back to you).
> Here is the script we are using:
> 
>   
> http://www.cs.unipr.it/cgi-bin/cvsweb.cgi/ppl/tests/BD_Shape/run_tests?rev=1.11

Ah, ok.  A couple of comments.  First, a bug I introduced by giving a
non-complete example: Some `make' implementations will not allow you to
override a macro on the command line iff it is also set in the Makefile.
With `TESTS', that is the case in your script.  Portable would be
   TESTS='...' make -e check

but `make -e' has its share of problems, too, depending on your
environment (same issue with the other variables, of course).  Another
point where Autotest is more flexible.

Furthermore, you write

| check_PROGRAMS=$(MAKEFLAGS='' make -s print_check_PROGRAMS)

which I assume you need to avoid clutter in the output.  I know many
systems where it is very useful to override the `make' command used,
so $MAKE would probably be better here and elsewhere in the script
(unless that interferes with clutter in the output), e.g. to be able
to use a make that allows macro override on the command line.

Then, a comment to the Makefile.am: you don't need the lines
| srcdir = @srcdir@
| VPATH = @VPATH@
| @SET_MAKE@
| SUBDIRS =

automake will take care of that by itself.

> The dirty_marker trick allows us to avoid useless recompilations yet
> addressing the case where one run of tests is interrupted.

Ah, nice.

> >If you want to go much further, you either end up creating more complex
> >shell scripts, or using one of the more advanced test suite creation
> >tools: Autoconf's Autotest, DejaGNU, ...
> 
> Autotest looks attractive.  We may consider switching to it as soon
> as it stabilizes.

The next Autoconf release should have a decently usable version of it.

Cheers,
Ralf