Re: [PATCH v7 8/8] Acceptance test: add "boot_linux" tests

[Cleber Rosa]

On Fri, Nov 08, 2019 at 05:42:14PM -0200, Wainer dos Santos Moschetta wrote:
> 
> On 11/4/19 1:13 PM, Cleber Rosa wrote:
> > This acceptance test, validates that a full blown Linux guest can
> > successfully boot in QEMU.  In this specific case, the guest chosen is
> > Fedora version 31.
> > 
> >   * x86_64, pc and q35 machine types, with and without kvm as an
> >     accellerator
> > 
> >   * aarch64 and virt machine type, with and without kvm as an
> >     accellerator
> > 
> >   * ppc64 and pseries machine type
> > 
> >   * s390x and s390-ccw-virtio machine type
> > 
> > The method for checking the successful boot is based on "cloudinit"
> > and its "phone home" feature.  The guest is given an ISO image
> > with the location of the phone home server, and the information to
> > post (the instance ID).  Upon receiving the correct information,
> > from the guest, the test is considered to have PASSed.
> > 
> > This test is currently limited to user mode networking only, and
> > instructs the guest to connect to the "router" address that is hard
> > coded in QEMU.
> > 
> > To create the cloudinit ISO image that will be used to configure the
> > guest, the pycdlib library is also required and has been added as
> > requirement to the virtual environment created by "check-venv".
> > 
> > The console output is read by a separate thread, by means of the
> > Avocado datadrainer utility module.
> > 
> > Signed-off-by: Cleber Rosa <address@hidden>
> > ---
> >   tests/acceptance/boot_linux.py | 175 +++++++++++++++++++++++++++++++++
> >   tests/requirements.txt         |   1 +
> >   2 files changed, 176 insertions(+)
> >   create mode 100644 tests/acceptance/boot_linux.py
> > 
> > diff --git a/tests/acceptance/boot_linux.py b/tests/acceptance/boot_linux.py
> > new file mode 100644
> > index 0000000000..882f7dc5df
> > --- /dev/null
> > +++ b/tests/acceptance/boot_linux.py
> > @@ -0,0 +1,175 @@
> > +# Functional test that boots a complete Linux system via a cloud image
> > +#
> > +# Copyright (c) 2018-2019 Red Hat, Inc.
> > +#
> > +# Author:
> > +#  Cleber Rosa <address@hidden>
> > +#
> > +# This work is licensed under the terms of the GNU GPL, version 2 or
> > +# later.  See the COPYING file in the top-level directory.
> > +
> > +import os
> > +
> > +from avocado_qemu import Test, SRC_ROOT_DIR
> > +
> > +from qemu import kvm_available
> > +
> > +from avocado.utils import cloudinit
> > +from avocado.utils import network
> > +from avocado.utils import vmimage
> > +from avocado.utils import datadrainer
> > +
> > +
> > +KVM_NOT_AVAILABLE = "KVM accelerator does not seem to be available"
> > +
> > +
> > +class BootLinux(Test):
> > +    """
> > +    Boots a Linux system, checking for a successful initialization
> > +    """
> > +
> > +    timeout = 600
> > +    chksum = None
> > +
> > +    def setUp(self):
> > +        super(BootLinux, self).setUp()
> > +        self.prepare_boot()
> > +        self.vm.add_args('-smp', '2')
> > +        self.vm.add_args('-m', '2048')
> > +        self.vm.add_args('-drive', 'file=%s' % self.boot.path)
> > +        self.prepare_cloudinit()
> > +
> > +    def prepare_boot(self):
> > +        self.log.info('Downloading/preparing boot image')
> Cosmetic: replace 'Downloading/preparing' with 'Downloading' since this
> function does only download the boot image.

Actually the "preparing" is intentional, because this will also
prepare the snapshot image (the original vmimage is never touched).

> > +        # Fedora 31 only provides ppc64le images
> > +        image_arch = self.arch
> > +        if image_arch == 'ppc64':
> > +            image_arch = 'ppc64le'
> > +        try:
> > +            self.boot = vmimage.get(
> > +                'fedora', arch=image_arch, version='31',
> > +                checksum=self.chksum,
> > +                algorithm='sha256',
> > +                cache_dir=self.cache_dirs[0],
> > +                snapshot_dir=self.workdir)
> > +        except:
> > +            self.cancel('Failed to download/prepare boot image')
> 
> Likewise.
>

Likewise too :)

> > +
> > +    def prepare_cloudinit(self):
> > +        self.log.info('Preparing cloudinit image')
> > +        try:
> > +            cloudinit_iso = os.path.join(self.workdir, 'cloudinit.iso')
> > +            self.phone_home_port = network.find_free_port()
> > +            cloudinit.iso(cloudinit_iso, self.name,
> > +                          username='root',
> > +                          password='password',
> > +                          # QEMU's hard coded usermode router address
> > +                          phone_home_host='10.0.2.2',
> > +                          phone_home_port=self.phone_home_port)
> > +            self.vm.add_args('-drive', 'file=%s,format=raw' % 
> > cloudinit_iso)
> > +        except Exception:
> > +            self.cancel('Failed to prepared cloudinit image')
> > +
> > +    def launch_and_wait(self):
> > +        self.vm.set_console()
> > +        self.vm.launch()
> > +        console_drainer = 
> > datadrainer.LineLogger(self.vm.console_socket.fileno(),
> > +                                                 
> > logger=self.log.getChild('console'))
> > +        console_drainer.start()
> > +        self.log.info('VM launched, waiting for boot confirmation from 
> > guest')
> > +        cloudinit.wait_for_phone_home(('0.0.0.0', self.phone_home_port), 
> > self.name)
> > +
> > +
> > +class BootLinuxX8664(BootLinux):
> > +    """
> > +    :avocado: tags=arch:x86_64
> > +    """
> > +
> > +    chksum = 
> > 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'
> > +
> > +    def test_pc(self):
> > +        """
> > +        :avocado: tags=machine:pc
> > +        """
> > +        self.launch_and_wait()
> > +
> > +    def test_kvm_pc(self):
> > +        """
> > +        :avocado: tags=machine:pc
> > +        :avocado: tags=accel:kvm
> > +        """
> > +        if not kvm_available(self.arch):
> > +            self.cancel(KVM_NOT_AVAILABLE)
> 
> kvm_available() solely checks for kvm presence on host, not if enabled in
> the target binary. That said, as a follow up we could adapt this test to use
> a strengthen approach as I proposed in patch 02 of '[PATCH 0/3] Acceptance
> tests: boot Linux with KVM test' [1] series.
> 
> [1] https://www.mail-archive.com/address@hidden/msg627498.html
>

Yes, I'll revisit that.  Thanks for the reminder.

> > +        self.vm.add_args("-accel", "kvm")
> > +        self.launch_and_wait()
> > +
> > +    def test_q35(self):
> > +        """
> > +        :avocado: tags=machine:q35
> > +        """
> > +        self.launch_and_wait()
> > +
> > +    def test_kvm_q35(self):
> > +        """
> > +        :avocado: tags=machine:q35
> > +        :avocado: tags=accel:kvm
> > +        """
> > +        if not kvm_available(self.arch):
> > +            self.cancel(KVM_NOT_AVAILABLE)
> > +        self.vm.add_args("-accel", "kvm")
> > +        self.launch_and_wait()
> > +
> > +
> > +class BootLinuxAarch64(BootLinux):
> > +    """
> > +    :avocado: tags=arch:aarch64
> > +    :avocado: tags=machine:virt
> > +    """
> > +
> > +    chksum = 
> > '1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49'
> > +
> > +    def test_virt(self):
> > +        self.vm.add_args('-cpu', 'cortex-a53')
> > +        self.vm.add_args('-bios',
> > +                         os.path.join(SRC_ROOT_DIR, 'pc-bios',
> > +                                      'edk2-aarch64-code.fd'))
> > +        self.vm.add_args('-device', 'virtio-rng-pci,rng=rng0')
> > +        self.vm.add_args('-object', 
> > 'rng-random,id=rng0,filename=/dev/urandom')
> > +        self.launch_and_wait()
> > +
> > +    def test_kvm_virt(self):
> > +        """
> > +        :avocado: tags=accel:kvm
> > +        """
> > +        if not kvm_available(self.arch):
> > +            self.cancel(KVM_NOT_AVAILABLE)
> > +        self.vm.add_args("-accel", "kvm")
> > +        self.test_virt()
> > +
> > +
> > +class BootLinuxPPC64(BootLinux):
> > +    """
> > +    :avocado: tags=arch:ppc64
> > +    """
> > +
> > +    chksum = 
> > '7c3528b85a3df4b2306e892199a9e1e43f991c506f2cc390dc4efa2026ad2f58'
> > +
> > +    def test_pseries(self):
> > +        """
> > +        :avocado: tags=machine:pseries
> > +        """
> > +        self.launch_and_wait()
> > +
> > +
> > +class BootLinuxS390X(BootLinux):
> > +    """
> > +    :avocado: tags=arch:s390x
> > +    """
> > +
> > +    chksum = 
> > '4caaab5a434fd4d1079149a072fdc7891e354f834d355069ca982fdcaf5a122d'
> > +
> > +    def test_s390_ccw_virtio(self):
> > +        """
> > +        :avocado: tags=machine:s390-ccw-virtio
> > +        """
> > +        self.launch_and_wait()
> > diff --git a/tests/requirements.txt b/tests/requirements.txt
> > index a2a587223a..3893361e0c 100644
> > --- a/tests/requirements.txt
> > +++ b/tests/requirements.txt
> > @@ -2,3 +2,4 @@
> >   # in the tests/venv Python virtual environment. For more info,
> >   # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
> >   avocado-framework==72.0
> > +pycdlib==1.8.0
> 
> Besides the cosmetics suggestions, this test is ready to go.
> 
> In other to write powerful tests there will be needed a mechanism to boot a
> full blown guest which can be interacted with. I see the proposed BootLinux
> class here as the first milestone towards that goal. Another reason to merge
> this as soon as possible.
>

Yes, agreed.

> Ran the tests on my x86_68 machine with RHEL 8.1, and all pass
> (BootLinuxAarch64.test_kvm_virt correctly skipped).
> 
> Reviewed-by: Wainer dos Santos Moschetta <address@hidden>
> 
> Thanks,
> 
> Wainer
> 

Thanks!
- Cleber.