Re: [PATCH] {yacc-work} tests: cover yacc target-specific flags, and `-v' flag handling

[Stefano Lattarini]

On Friday 21 January 2011, Ralf Wildenhues wrote:
> * Stefano Lattarini wrote on Fri, Jan 21, 2011 at 01:13:35PM CET:
> > A small testsuite patch which slightly increases coverage of
> > Automake Yacc support.  OK for the temporary branch yacc-work
> > (which at this point could as well be made public maybe), to
> > be merged into master?
> > 
> > I will push in 72 hours if there are no objections.
> 
> The patch is OK, but still, can you please wait until, say, at least
> Monday before pushing?
>
Yes.  I just wanted to have it approved for the 'yacc-work' branch, 
so that I could move on with the non-testsuite work in there.

> I'm hopeful that I can finish one pass over
> testsuite failures this weekend, and would like to have a "good" rerun
> then that we can use as base for comparison, without introducing more
> than necessary drift in the meantime.
>
A good idea would then be to make the 'yacc-work' branch public, without
merging it to master until testsuite issues settle down.  WDYT?

> The point of introducing new tests is lessened if we don't sometimes go
> around and fix the issues that running them exposes.  And we really
> should strive to have zero FAILures in the test suites by the next
> release on all but the near-museum systems.
>
Ah, but the VPATH issue with FreeBSD (which is long standing BTW) will be
quite tricky to fix.  Maybe your proposed `ensure_distcheck_' hack could
have a good, legitimate use in this issue...  I might prepare a patch
about that in the weekend -- and as a follow-up I might also implement my
proposal of letting the user force a non-VPATH build in "make distcheck",
see:
 <http://lists.gnu.org/archive/html/automake-patches/2011-01/msg00195.html>

For the other failures, I agree with you.

> Things that we cannot fix should be XFAILs instead;
>
Fact is, many failing tests fail only with some make implementations (and
maybe just one).  Declaring them as XFAILing whould just cause XPASSes
with all the other makes :-(

> distros typically won't upgrade upon test
> failures, so abusing that as TODO list is not a good idea in the longer
> term (not even short-term, as having failures for a longer time tends to
> blind oneself for new regressions).
>
Right -- and I'll catch this occasion to ping a patch of mine which
fixed what IMHO amounted to a spurious failure with Solaris make:
 <http://lists.gnu.org/archive/html/automake-patches/2010-11/msg00221.html>
(note that the patch is badly out-of-date, and needs to be rebased -- I'll
do that if you agree it's worth applying).

> Thanks,
> Ralf
> 
> > tests: cover yacc target-specific flags, and `-v' flag handling
> > 
> > * tests/yacc-basic: Try also to build a parser having `-v' as
> 
> Also try
> 
> > target-specific flags.  Add a couple of `ls -l' commands, for
> > debugging.  Update and extend comments.
> 

Thanks for the quick review,
  Stefano