Re: [PATCH 1/5] tests/boot_linux_console: Add initrd test for the Exynos4210

[Cleber Rosa]

On Mon, Oct 07, 2019 at 05:28:49PM +0100, Peter Maydell wrote:
> On Sat, 5 Oct 2019 at 16:47, Philippe Mathieu-Daudé <address@hidden> wrote:
> >
> > This test boots a Linux kernel on a smdkc210 board and verify
> > the serial output is working.
> >
> > The cpio image used comes from the linux-build-test project:
> > https://github.com/groeck/linux-build-test
> 
> Thanks for putting this test case together, very helpful.
> 
> > +        initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
> > +                      '2eb0a73b5d5a28df3170c546ddaaa9757e1e0848/rootfs/'
> > +                      'arm/rootfs-armv5.cpio.gz')
> 
> Do our other acceptance tests download random third-party
> (ie "not a well-known distro") binaries for the tests ?
> It seems a bit hazardous for reproducability and trustability
> reasons...
> 
> thanks
> -- PMM

A quick and dirty grep shows (excluding comments and docs):

   boot_linux_console.py:        kernel_url = 
('https://archives.fedoraproject.org/pub/archive/fedora'
   boot_linux_console.py:        deb_url = 
('http://snapshot.debian.org/archive/debian/'
   boot_linux_console.py:        deb_url = 
('http://snapshot.debian.org/archive/debian/'
   boot_linux_console.py:        deb_url = 
('http://snapshot.debian.org/archive/debian/'
   boot_linux_console.py:        initrd_url = 
('https://github.com/groeck/linux-build-test/raw/'
   boot_linux_console.py:        kernel_url = 
('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
   boot_linux_console.py:        kernel_url = 
('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
   boot_linux_console.py:        kernel_url = 
('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
   boot_linux_console.py:        kernel_url = 
('https://archives.fedoraproject.org/pub/archive/fedora'
   boot_linux_console.py:        kernel_url = 
('https://archives.fedoraproject.org/pub/archive/fedora'
   boot_linux_console.py:        uboot_url = 
('https://raw.githubusercontent.com/'
   boot_linux_console.py:        spi_url = ('https://raw.githubusercontent.com/'
   boot_linux_console.py:        kernel_url = 
('https://archives.fedoraproject.org/pub/archive'
   boot_linux_console.py:        kernel_url = 
('http://archive.debian.org/debian/dists/lenny/main/'
   boot_linux_console.py:        kernel_url = 
('https://archives.fedoraproject.org/pub/archive'
   linux_initrd.py:        kernel_url = 
('https://archives.fedoraproject.org/pub/archive/fedora/li'
   linux_initrd.py:        kernel_url = 
('https://archives.fedoraproject.org/pub/archive/fedora'
   linux_ssh_mips_malta.py:        'be': {'image_url': 
('https://people.debian.org/~aurel32/qemu/mips/'
   linux_ssh_mips_malta.py:        'le': {'image_url': 
('https://people.debian.org/~aurel32/qemu/mipsel/'
   linux_ssh_mips_malta.py:        kernel_url = 
('https://people.debian.org/~aurel32/qemu/mips/'
   linux_ssh_mips_malta.py:        kernel_url = 
('https://people.debian.org/~aurel32/qemu/mipsel/'
   linux_ssh_mips_malta.py:        kernel_url = 
('https://people.debian.org/~aurel32/qemu/mips/'
   linux_ssh_mips_malta.py:        kernel_url = 
('https://people.debian.org/~aurel32/qemu/mipsel/'
   machine_m68k_nextcube.py:        rom_url = 
('http://www.nextcomputers.org/NeXTfiles/Software/ROM_Files/'

I find it hard to judge precisely how much of a third-party some of
these are.  I remember Philippe mentioning that one of them, I guess
the images used on linux_ssh_mips_malta.py, were "as official as it
gets" (my words, from my often misleading memory).

Reproducibility is definitely an issue, in the sense given that some
of these can indeed go away, but as long as they're available the hash
recorded in the test should guarantee that we're running the same
images.

Do you think we should do something different here?

Thanks!
- Cleber.