Re: [PATCH 1/5] tests/boot_linux_console: Add initrd test for the Exynos4210

[Peter Maydell]

On Tue, 8 Oct 2019 at 22:49, Cleber Rosa <address@hidden> wrote:
> On Mon, Oct 07, 2019 at 05:28:49PM +0100, Peter Maydell wrote:
> > Do our other acceptance tests download random third-party
> > (ie "not a well-known distro") binaries for the tests ?
> > It seems a bit hazardous for reproducability and trustability
> > reasons...

> A quick and dirty grep shows (excluding comments and docs):
>
>    boot_linux_console.py:        kernel_url = 
> ('https://archives.fedoraproject.org/pub/archive/fedora'
>    boot_linux_console.py:        deb_url = 
> ('http://snapshot.debian.org/archive/debian/'
>    boot_linux_console.py:        deb_url = 
> ('http://snapshot.debian.org/archive/debian/'
>    boot_linux_console.py:        deb_url = 
> ('http://snapshot.debian.org/archive/debian/'
>    boot_linux_console.py:        initrd_url = 
> ('https://github.com/groeck/linux-build-test/raw/'
>    boot_linux_console.py:        kernel_url = 
> ('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
>    boot_linux_console.py:        kernel_url = 
> ('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
>    boot_linux_console.py:        kernel_url = 
> ('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
>    boot_linux_console.py:        kernel_url = 
> ('https://archives.fedoraproject.org/pub/archive/fedora'
>    boot_linux_console.py:        kernel_url = 
> ('https://archives.fedoraproject.org/pub/archive/fedora'
>    boot_linux_console.py:        uboot_url = 
> ('https://raw.githubusercontent.com/'
>    boot_linux_console.py:        spi_url = 
> ('https://raw.githubusercontent.com/'
>    boot_linux_console.py:        kernel_url = 
> ('https://archives.fedoraproject.org/pub/archive'
>    boot_linux_console.py:        kernel_url = 
> ('http://archive.debian.org/debian/dists/lenny/main/'
>    boot_linux_console.py:        kernel_url = 
> ('https://archives.fedoraproject.org/pub/archive'
>    linux_initrd.py:        kernel_url = 
> ('https://archives.fedoraproject.org/pub/archive/fedora/li'
>    linux_initrd.py:        kernel_url = 
> ('https://archives.fedoraproject.org/pub/archive/fedora'
>    linux_ssh_mips_malta.py:        'be': {'image_url': 
> ('https://people.debian.org/~aurel32/qemu/mips/'
>    linux_ssh_mips_malta.py:        'le': {'image_url': 
> ('https://people.debian.org/~aurel32/qemu/mipsel/'
>    linux_ssh_mips_malta.py:        kernel_url = 
> ('https://people.debian.org/~aurel32/qemu/mips/'
>    linux_ssh_mips_malta.py:        kernel_url = 
> ('https://people.debian.org/~aurel32/qemu/mipsel/'
>    linux_ssh_mips_malta.py:        kernel_url = 
> ('https://people.debian.org/~aurel32/qemu/mips/'
>    linux_ssh_mips_malta.py:        kernel_url = 
> ('https://people.debian.org/~aurel32/qemu/mipsel/'
>    machine_m68k_nextcube.py:        rom_url = 
> ('http://www.nextcomputers.org/NeXTfiles/Software/ROM_Files/'
>
> I find it hard to judge precisely how much of a third-party some of
> these are.  I remember Philippe mentioning that one of them, I guess
> the images used on linux_ssh_mips_malta.py, were "as official as it
> gets" (my words, from my often misleading memory).
>
> Reproducibility is definitely an issue, in the sense given that some
> of these can indeed go away, but as long as they're available the hash
> recorded in the test should guarantee that we're running the same
> images.
>
> Do you think we should do something different here?

I'm not sure, which is why I asked whether this new test
was in line with what we've done previously. Since these
are just test cases and we don't redistribute them to
other people there's less of a traceability/reproducibility
worry, and if we check hashes on download that cuts off
a lot of "fail to notice if the image changes for some
reason" possible problems.

thanks
-- PMM