Re: [PATCH v2 03/11] tests/acceptance: add base class record/replay kernel tests

[Philippe Mathieu-Daudé]

On 5/27/20 12:30 PM, Pavel Dovgalyuk wrote:
> This patch adds a base for testing kernel boot recording and replaying.
> Each test has the phase of recording and phase of replaying.
> Virtual machines just boot the kernel and do not interact with
> the network.
> Structure and image links for the tests are borrowed from 
> boot_linux_console.py
> Testing controls the message pattern at the end of the kernel
> boot for both record and replay modes. In replay mode QEMU is also
> intended to finish the execution automatically.
> 
> Signed-off-by: Pavel Dovgalyuk <address@hidden>
> 
> --
> 
> v2:
>  - changed default value of args to None (suggested by Willian Rampazzo)
>  - inherited common functions from boot_linux_console (suggested by Willian 
> Rampazzo)
> ---
>  0 files changed
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 47ef3139e6..e9a9ce4f66 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -2497,6 +2497,7 @@ F: net/filter-replay.c
>  F: include/sysemu/replay.h
>  F: docs/replay.txt
>  F: stubs/replay.c
> +F: tests/acceptance/replay_kernel.py
>  
>  IOVA Tree
>  M: Peter Xu <address@hidden>
> diff --git a/tests/acceptance/replay_kernel.py 
> b/tests/acceptance/replay_kernel.py
> new file mode 100644
> index 0000000000..b8b277ad2f
> --- /dev/null
> +++ b/tests/acceptance/replay_kernel.py
> @@ -0,0 +1,57 @@
> +# Record/replay test that boots a Linux kernel
> +#
> +# Copyright (c) 2020 ISP RAS
> +#
> +# Author:
> +#  Pavel Dovgalyuk <address@hidden>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or
> +# later.  See the COPYING file in the top-level directory.
> +
> +import os
> +import gzip
> +
> +from avocado_qemu import wait_for_console_pattern
> +from avocado.utils import process
> +from avocado.utils import archive
> +from boot_linux_console import LinuxKernelUtils
> +
> +class ReplayKernel(LinuxKernelUtils):
> +    """
> +    Boots a Linux kernel in record mode and checks that the console
> +    is operational and the kernel command line is properly passed
> +    from QEMU to the kernel.
> +    Then replays the same scenario and verifies, that QEMU correctly
> +    terminates.
> +    """
> +
> +    timeout = 90
> +
> +    def run_vm(self, kernel_path, kernel_command_line, console_pattern,
> +               record, shift, args):
> +        vm = self.get_vm()
> +        vm.set_console()
> +        if record:
> +            mode = 'record'
> +        else:
> +            mode = 'replay'
> +        vm.add_args('-icount', 'shift=%s,rr=%s,rrfile=%s' %
> +                    (shift, mode, os.path.join(self.workdir, 'replay.bin')),

Please make replay_filename an argument, so we can add tests replaying
without having to record, and or do verification on the replay_filename
(check size, check header, run with incorrect format...).

> +                    '-kernel', kernel_path,
> +                    '-append', kernel_command_line,
> +                    '-net', 'none')
> +        if args:
> +            vm.add_args(*args)
> +        vm.launch()
> +        self.wait_for_console_pattern(console_pattern, vm)
> +        if record:
> +            vm.shutdown()
> +        else:
> +            vm.wait()
> +
> +    def run_rr(self, kernel_path, kernel_command_line, console_pattern,
> +        shift=7, args=None):
> +        self.run_vm(kernel_path, kernel_command_line, console_pattern,
> +                    True, shift, args)
> +        self.run_vm(kernel_path, kernel_command_line, console_pattern,
> +                    False, shift, args)
> 

Also please consider this snippet for logging:

-- >8 --
--- a/tests/acceptance/replay_kernel.py
+++ b/tests/acceptance/replay_kernel.py
@@ -10,6 +10,7 @@

 import os
 import gzip
+import logging

 from avocado_qemu import wait_for_console_pattern
 from avocado.utils import process
@@ -51,8 +52,11 @@ def run_vm(self, kernel_path, kernel_command_line,
console_pattern,

     def run_rr(self, kernel_path, kernel_command_line, console_pattern,
         shift=7, args=None):
+        logger = logging.getLogger('replay')
+        logger.info('recording...')
         self.run_vm(kernel_path, kernel_command_line, console_pattern,
                     True, shift, args)
+        logger.info('replaying...')
         self.run_vm(kernel_path, kernel_command_line, console_pattern,
                     False, shift, args)
---

Then you can use the logger using:

avocado --show=app,replay run -t machine:vexpress-a9 \
  tests/acceptance/replay_kernel.py

Ideally I'd like to see timestamp (Avocado surely provide the option)
and record.bin size between.