Re: Collecting GRUB 2.06 test results

[John Paul Adrian Glaubitz]

Hi Glenn!

On 2/11/21 12:57 AM, Glenn Washburn wrote:
> Ok, I'm glad you're running the tests on native hardware and in that
> case I'm not surprised most of the qemu tests aren't running because I
> assume that qemu is not on most of the non x86 platforms. Perhaps we
> should have the tests show as SKIPPED when the appropriate qemu binary
> can not be found.

OK. I wasn't aware of that.

>>> I'd like to note that it appears that these tests do not perform
>>> most of the various filesystem tests.
>>
>> I ran the tests with "make check" (or "make test"), I did not make
>> any modifications. If some tests were not run that's because GRUB does
>> not enable them on the target in question.
> 
> That's technically true, but not the whole story. GRUB is not enabling
> the filesystem tests because you don't have your target environment
> setup such that the tests will be enabled. The filesystem tests require
> filesystem tools (eg. e2fsprogs for ext* fs tests). You might try
> installing the prerequisites needed by the tests, and then seeing if
> they run.

I can re-run them with the fsprogs installed. Does that just concern
e2fsprogs or other filesystems as well?

>>> Also, I believe mips arch is not being tested here. And it appears
>>> that none of the qemu tests for mipsle are being run due to not
>>> having qemu for that arch installed.
>>
>> What do you mean by "here"?
> 
> I mean that it appears to me that you don't have tests for the mips
> arch, only mipsle. I'm not very familiar with mips, so I may be
> confusing. My understanding is that there are big and little endian
> variants of the mips architecture, and 32 vs 64 bit variants. I
> understand mipsle to indicate the little-endian variant, but GRUB
> supports the big endian variant as well. It seems to me that you only
> have test logs for the 32 and 64 bit little endian variants.

Right. I didn't test big-endian MIPS because it's not part of the
Debian unstable distribution anymore (but Debian Buster). I will test
big-endian in a second test run, I have access to a big-endian MIPS
machine.

>> You can see that by looking at the configure.ac. It's basically all
>> targets that build more than just the utilities.
> 
> Great, that's what I was thinking. I notice that it appears you're not
> testing every supported target, just one target from each architecture.
> For instance, you're not testing and i386 build of the ieee1275 or efi
> platforms. 

Well, if ieee1275 or efi should be tested on i386, I think the testsuite
should enable these tests by default. Although I'm not sure there is x86
hardware which uses OpenFirmware, is there?

> Since you're not using qemu on the non-x86 platforms, I'm not sure if
> testing the various platforms of other architectures would be worth
> while (eg. arm-coreboot and arm-efi). Perhaps someone more
> knowledgeable than I can weight in.

I can also test with QEMU installed if desired for certain architectures.

Maybe you could make a list of target/architecture combinations you want
me to test.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913