Re: regtest false negative

[James Lowe]

On 05/08/2020 12:04, Dan Eble wrote:
> On Aug 4, 2020, at 07:29, Dan Eble <dan@faithful.be> wrote:
> > On Aug 4, 2020, at 07:03, James Lowe <pkx166h@posteo.net> wrote:
> > > On 04/08/2020 03:25, Dan Eble wrote:
> > > > The regression test framework fails to detect a significant difference (the 
> > > > addition of a whole-measure rest) in the output of 
> > > > input/regression/part-combine-solo-end.ly in this branch:
> > > >
> > > > https://gitlab.com/lilypond/lilypond/-/tree/dev/eble/4599-pc-solo-end
> > > > —
> > > > Dan
> > > I am only seeing a Texidoc string change between the files.
> > I was unclear.  There is no change to the source of the test.  The branch has a LilyPond 
> > bug fix.  Running the test manually on master and on the branch and comparing manually 
> > shows that the fix causes a missing rest to appear in the output; however, "make 
> > check" on the branch fails to detect the difference.
> https://gitlab.com/lilypond/lilypond/-/merge_requests/304

It seems I can reproduce this - albeit there are significant diffs 
between the part-comine-solo-end.eps files on each of the 
test-baselines. I've never looked at what a test-baseline looked like 
before (I assumed there'd be a set of png files generated but I guess 
that is done during the 'check' stage).

Anything more I can provide here?

James