Re: GRUB 2.06~rc1 released

[Glenn Washburn]

On Fri, 12 Mar 2021 17:48:27 -0600
Bruce Dubbs <bruce.dubbs@gmail.com> wrote:

> On 3/12/21 5:25 PM, Glenn Washburn wrote:
> > Hi Bruce,
> > 
> > On Fri, 12 Mar 2021 15:30:01 -0600
> > Bruce Dubbs <bruce.dubbs@gmail.com> wrote:
> > 
> >> On 3/12/21 1:57 PM, Daniel Kiper wrote:
> >>> Hi all,
> >>>
> >>> The GRUB maintainers are proud to announce the GRUB 2.06~rc1 that
> >>> has been just released.
> >>
> >> Congratulations.
> >>
> >> For LFS, we do a very simple build in a sparse environment.  Here
> >> are a few observations.
> >>
> >> We use:
> >>
> >> ./configure --prefix=/usr          \
> >>               --sbindir=/sbin        \
> >>               --sysconfdir=/etc      \
> >>               --disable-efiemu       \
> >>               --disable-werror
> >>
> >> That works cleanly.  However there are two files that generate
> >> warnings:
> >>
> >> ./grub-core/script/parser.y:92.1-12: warning: deprecated directive:
> >> ‘%pure-parser’, use ‘%define api.pure’ [-Wdeprecated]
> >>      92 | %pure-parser
> >>         | ^~~~~~~~~~~~
> >>         | %define api.pure
> >>
> >> ./grub-core/script/parser.y: warning: fix-its can be applied.
> >> Rerun with option '--update'. [-Wother]
> >>
> >> grub-core/lib/libgcrypt-grub/cipher/rijndael.c: In function
> >> ‘do_setkey’: grub-core/lib/libgcrypt-grub/cipher/rijndael.c:229:9:
> >> warning: suggest braces around empty body in an ‘if’ statement
> >> [-Wempty-body] 229 |         ;
> >>         |         ^
> >>
> >> grub-core/lib/libgcrypt-grub/cipher/rijndael.c:352:21: warning:
> >> comparison of integer expressions of different signedness: ‘int’
> >> and ‘unsigned int’ [-Wsign-compare]
> >>     352 |       for (i = 0; i < keylen; i++)
> >>         |                     ^
> >>
> >> It seems that the above is run twice by make, but the fixes are
> >> trivial.
> > 
> > I'm curious, what compiler and version are you using?
> 
> $ gcc --version
> gcc (GCC) 10.2.0
> 
> $ bison --version
> bison (GNU Bison) 3.7.5
> 
> >> For make check, I get 43 failures.  Many appear to be due to our
> >> sparse environment.  In most cases I think these tests should be
> >> SKIP instead of FAIL, but I'll follow up with a more detailed
> >> report later.
> > 
> > I'm looking forward to the report. I think ideally the tests should
> > use a very minimal common set of utilities (with some obvious
> > exceptions like qemu and mkfs.*). The tests should only SKIP if its
> > not an appropriate test for the configured target. More likely,
> > they should ERROR to indicated that there was a failure to run the
> > test, not a failure of what the test was testing. This lets the
> > tester know that there's likely an environmental issue that needs
> > to be fixed to get tests working as they should.
> 
> That's an interesting perspective.  To my mind the failure of a test
> can be due to a problem with the grub code, the grub test, or the 
> environment of the test system.

Sure, and in the later two cases the test should return ERROR because
we want to know that there's a problem somewhere, whereas the first
would be a legitimate FAIL. We don't want the issues with the test
environment to mark the test as SKIP because that would occlude them.
We should be able to ignore tests that return SKIP (for instance
pseries tests on x86_64). Currently the filesystem tests will SKIP if
the tool to create the filesystem being tested is not found. This is a
problem because now we have to distinguish between SKIP tests that we
can ignore and ones we need to look at. Whereas if tests not run due to
environmental issues are marked as ERROR, along with tests that ERROR
for some other reason, it doesn't cause more work or confusion because
we should go through all those tests anyway. And failures of course
should be reserved for tests that fail due to an issue in the GRUB
code. These should be serious high priority failures. Unfortunately,
some tests are currently expected to fail, rendering the testing less
useful that it could be. Alarm bells should go off if there are any
failures on any targets, but we can't do that now because there are
expected failures in some tests. This can lead to a desensitization to
tests failures.

This is one reason I setup the GitLab CI[1] the way I did, which is to
mark some known failing tests as expected and to not fail the CI build
on certain known failing tests.

>  In my case I'm sure that most of the 
> failures, if not all, are due to the test environment.  In my full 
> environment, I only have qemu-system-x86_64, so I'll need to fix
> that. I should have everything else needed, but need to double check.

Glenn

[1] https://gitlab.com/gnu-grub/grub/-/pipelines/258485608