Re: roms/efirom, tests/uefi-test-tools: update edk2's own submodules first

[Laszlo Ersek]

On 10/20/20 11:54, Philippe Mathieu-Daudé wrote:
> On 10/20/20 11:44 AM, Daniel P. Berrangé wrote:
>> On Tue, Oct 20, 2020 at 11:29:01AM +0200, Philippe Mathieu-Daudé wrote:
>>> Hi Olaf,
>>>
>>> On 10/20/20 11:16 AM, Olaf Hering wrote:
>>>> This is about qemu.git#ec87b5daca761039bbcf781eedbe4987f790836f
>>>>
>>>> On Mon, Sep 07, Laszlo Ersek wrote:
>>>>
>>>>> In edk2 commit 06033f5abad3 ("BaseTools: Make brotli a submodule",
>>>>> 2020-04-16), part of edk2-stable202005, the Brotli compressor /
>>>>> decompressor source code that edk2 had flattened into BaseTools was
>>>>> replaced with a git submodule.
>>>>>
>>>>> This means we have to initialize edk2's own submodules before building
>>>>> BaseTools not just in "roms/Makefile.edk2", but in "roms/Makefile"
>>>>> (for
>>>>> the sake of the "efirom" target) and
>>>>> "tests/uefi-test-tools/Makefile" as
>>>>> well.
>>>>
>>>>> +++ b/roms/Makefile
>>>>>    edk2-basetools:
>>>>> +    cd edk2/BaseTools && git submodule update --init --force
>>>>>    build-edk2-tools:
>>>>> +    cd $(edk2_dir)/BaseTools && git submodule update --init --force
>>>>
>>>>
>>>> This change can not possibly be correct.
>>>>
>>>> With current qemu.git#master one is forced to have network access to
>>>> build the roms. This fails with exported (and complete) sources in an
>>>> offline environment.
>>>
>>> The EDK2 roms are only used for testing, we certainly don't want them
>>> to be used by distributions. I suppose the question is "why is this
>>> rule called if tests are not built?".
>>
>> I don't believe that is correct - the pc-bios/edk*  ROMs and the
>> corresponding  pc-bios/descriptor files are there for real world
>> end user consumption.   roms/edk2 should (must) match / reflect
>> the content used to build the pci-bios/edk* blobs.
>>
>> Many distros have a policy requiring them to build everything
>> from source, so they will ignore the pre-built edk2 ROMs, but
>> regular end users taking QEMU directly from upstream can certainly
>> use our edk2 ROMs.
> 
> Well I'm lost (and I don't think mainstream QEMU have the
> bandwidth to follow mainstream EDK2 security fixes) so I'm
> giving up, waiting for clarification from Laszlo.

I definitely don't have time for keeping the edk2 blobs bundled with
QEMU fresh wrt. security fixes in upstream edk2, so anyone expecting
that is in for a bad surprise. The blobs are provided, from my
perspective, (a) for some tests in the test suite (such as
bios-tables-test for the aarch64 target), (b) as a convenience for
end-users that desire to build QEMU from source, without wanting to
build OVMF from source.

I don't understand the particular problem (or rather: use case) that
Olaf is reporting (and this is not the first time). I see four classes
of people here:

(1) end-users described above, in point (b) -- then, there is no need
for rebuilding the bundled edk2 binaries using the QEMU build infrastructure

(2) end-users building everything from source (genuine standalone clones
/ checkouts), for themselves

(3) distributors building everything from source (genuine, standalone
clones / checkouts), for their users

(4) QEMU co-maintainers that sometimes refresh the binaries -- this is
the only group that the build infra *needs* to work for (in the future,
the edk2 build infra should actually target a remote build system, but
we're not there yet -- and even in that case, the edk2 build scripts
inside the QEMU tree will only have to work for *that* environment)

Olaf: if you build QEMU from source, why don't you build SeaBIOS, iPXE,
edk2 etc *also* from their corresponding pristine upstream clones /
checkouts, using your own dedicated build scripts / packagings?

... On the technical side, I guess the problem is that edk2, unlike some
other submodules of QEMU, has its own submodules (meaning that, from the
QEMU superproject's perspective, edk2 creates recusrive submodules). I
have really zero idea how to deal with that (or more precisely, what the
grander impact of that would be); but importantly, it does not *matter*,
in my opinion. If you don't co-maintain the edk2 binaries bundled with
QEMU, then the edk2 build stuff present in QEMU is not *required* to
work for you.

If you don't like that, feel free to post patches, or I can quit even
this level of maintenance for the bundled edk2 binaries. I will
absolutely not consider downstream packaging needs with *how* the
bundled edk2 binaries are built.

Thanks
Laszlo