grub-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: GRUB 2.06~rc1 released


From: Glenn Washburn
Subject: Re: GRUB 2.06~rc1 released
Date: Sat, 13 Mar 2021 23:08:44 -0600

On Sat, 13 Mar 2021 16:13:54 -0600
Bruce Dubbs <bruce.dubbs@gmail.com> wrote:

> On 3/12/21 5:48 PM, Bruce Dubbs wrote:
> > On 3/12/21 5:25 PM, Glenn Washburn wrote:
> >> Hi Bruce,
> >>
> >> On Fri, 12 Mar 2021 15:30:01 -0600
> >> Bruce Dubbs <bruce.dubbs@gmail.com> wrote:
> >>
> >>> On 3/12/21 1:57 PM, Daniel Kiper wrote:
> 
> [snip]
> 
> >>> For LFS, we do a very simple build in a sparse environment.  Here
> >>> are a few observations.
> >>>
> >>> We use:
> >>>
> >>> ./configure --prefix=/usr          \
> >>>               --sbindir=/sbin        \
> >>>               --sysconfdir=/etc      \
> >>>               --disable-efiemu       \
> >>>               --disable-werror
> 
> [snip]
> 
> >> I'm curious, what compiler and version are you using?
> > 
> > $ gcc --version
> > gcc (GCC) 10.2.0

That might be why I haven't caught it then. The GitLab CI is using at
most 10.1.0, until the kernel.org prebuilt binaries get added for more
recent versions.

> > $ bison --version
> > bison (GNU Bison) 3.7.5
> > 
> >>> For make check, I get 43 failures.  Many appear to be due to our
> >>> sparse environment.  In most cases I think these tests should be
> >>> SKIP instead of FAIL, but I'll follow up with a more detailed
> >>> report later.
> >>
> >> I'm looking forward to the report. I think ideally the tests
> >> should use a very minimal common set of utilities (with some
> >> obvious exceptions like qemu and mkfs.*). The tests should only
> >> SKIP if its not an appropriate test for the configured target.
> >> More likely, they should ERROR to indicated that there was a
> >> failure to run the test, not a failure of what the test was
> >> testing. This lets the tester know that there's likely an
> >> environmental issue that needs to be fixed to get tests working as
> >> they should.
> 
> I'll note that several tests require being run as the root user.  I 
> cannot find where that is specified.  I only figured that out by 
> directly looking at the test scripts.

I don't think running the test suite is documented anywhere, and it
should be (likely in the developer manual). Is there some documentation
that I'm not aware of?

> When I build in a full LFS/BLFS system I get:
> 
> # TOTAL: 82
> # PASS:  67
> # SKIP:  11
> # XFAIL: 0
> # FAIL:  4
> # XPASS: 0
> # ERROR: 0
> 
> The SKIPs are due to not having the expected executables.
> squashfs_test
> hfsplus_test
> minixfs_test
> f2fs_test
> nilfs2_test
> romfs_test
> exfat_test
> udf_test
> hfs_test
> zfs_test

Yes, and they should be ERRORs so that folks testing understand that
there's an unexpected issue.

> lzocompress_test
>    This test depends on an lzop executable which in not a part of the
>    xz-5.2.5 distribution.  I found it is from a lzop source package
> which is new to us at LFS.
> 
> I doubt many systems will have most of the the software needed for
> these tests, but perhaps a listing of what GRUB expects would be
> useful.

The GitLab integration config that I created has a list of
ubuntu/debian packages that I needed to get these tests running. And
due to some limitations of GitLab shared runners there are a few tests
that don't run (like zfs).

The list of packages start at this line in the config:
https://gitlab.com/gnu-grub/grub/-/blob/a971d40f5d31b8f9ddec3eabfac1c232ca1cf240/.gitlab-ci.yml#L457

> The FAILing tests are a bit more troubling:
> reiserfs_test
>    mount: /tmp/tmp.t6gBOmoH4g/reiserfs_old_rw: can't read superblock
> on /dev/loop0.
>    MOUNT FAILED.
> 
>    I have reiserfsprogs-3.6.27 installed.

Its not easy to instrument the tests currently to provide shell tracing,
which would help here. Though I have some patches in the pipeline that
will help in this regard. Do you have loopback devices working
correctly and available?

>    ----
> fat_test
>    mkfs.vfat: Labels with characters *?.,;:/\|+=<>[]" are not allowed
>    mkfs.fat 4.2 (2021-01-31)
>    FAIL fat_test (exit status: 1)
> 
>    I have dosfstools-4.2 installed.

I wonder if this is due to a newer version dosfstools? The GitLab CI
was setup to test on Ubuntu xenial which is using dosfstools version
3.0.28 and is not hitting this issue. I did this to help make sure
we're maintaining backward compatibility with older systems. 

>    ----
> btrfs_test
>    ERROR: invalid sectorsize 256, expected range is [4K, 64K]
>    btrfs-progs v5.10.1
>    See http://btrfs.wiki.kernel.org for more information.
>    FAIL btrfs_test (exit status: 1)
> 
>    I have btrfs-progs-v5.10.1 installed.

Again, haven't seen this either and perhaps due to a newer version.
GitLab CI is using version 4.4.

>    ----
> grub_func_test
>    FAIL grub_func_test (exit status: 1)
> 
>    I cannot determine from the test what is expected.
>    ----

grub_func_test is a suite of functional tests. All tests should pass.
Currently several do not, and it is known that the tests need to be
updated. So some of these tests are not valid. The GitLab CI
currently ignores the result of videotest_checksum, gfxterm_menu, and
cmdline_cat_test [1].

> 
> It would appear that many of the GRUB regression test results are
> distro specific and not really GRUB tests.

At a minimum, I don't think any of these indicate issues in the GRUB
code. But I think they indicate that some tests will need to be
upgraded in the future to account for changes in test dependencies.

Glenn

[1]
https://gitlab.com/gnu-grub/grub/-/blob/a971d40f5d31b8f9ddec3eabfac1c232ca1cf240/.gitlab-ci.yml#L160



reply via email to

[Prev in Thread] Current Thread [Next in Thread]