Re: Use of ly:expect-warning in regtests

[Michael Käppler]

Am 02.10.2020 um 14:38 schrieb Dan Eble:
> > Mostly they set it to #f:
> >
> > git grep "(ly:set-option 'warning-as-error #f" input/regression/* | wc -l
> > 39
> I don't know the history behind that, but it doesn't seem useful.  It could be 
> a mistake.
I think I got it.
At first there were neither 'warning-as-error nor ly:expect-warning.

In 2009, 'warning-as-error was introduced in
531042cb79ca69ad0599c77da9e5bfe4edbbde05

IIUC the idea was later to enable 'warning-as-error for all regtests to spot
potential failures earlier.
Since some regtests emitted warnings intentionally, there was a issue
created
for setting 'warning-as-error to #f for these regtests, so that they
would not break
compilation.

https://gitlab.com/lilypond/lilypond/-/issues/814

In 2011, ly:expect-warning was introduced with commit
1ecdd56060e34a00b2be6b38029b286a601ea6f8
by Reinhold Kainhofer.

Reinhold also adapted the regtests in such a way, that all
expected warnings should be suppressed and remaining warnings
unveil bugs.
However, he left the old (ly:set-option 'warning-as-error #f) code in.
(Don't know why)

tl;dr: I would suggest the following policy:
- do not output expected warnings in regtests, instead use ly:expect-warning
- set 'warning-as-error to #t for these regtests

For all existing regtests: remove (ly:set-option 'warning-as-error #f)

In the long run:
- check the regtests for warnings and figure out, whether
they are intended
 - if they are intended, add ly:expect-warning
- run the regtests with 'warning-as-error to #t, if all snipped are fixed

It seems that this topic has been around from time to time.
Maybe James can tell us, why running the regtests with 'warning-as-error #t
was never realized.

Cheers,
Michael