[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PATCH v5 09/10] acpi/tests/avocado/bits: add a README file to describe
From: |
Ani Sinha |
Subject: |
[PATCH v5 09/10] acpi/tests/avocado/bits: add a README file to describe the test |
Date: |
Wed, 19 Oct 2022 08:28:27 +0530 |
Add a README file that describes the purpose of the various test files and gives
guidance to developers on where and how to make changes.
Cc: Daniel P. Berrange" <berrange@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Maydell Peter <peter.maydell@linaro.org>
Cc: John Snow <jsnow@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Michael Tsirkin <mst@redhat.com>
Signed-off-by: Ani Sinha <ani@anisinha.ca>
---
tests/avocado/acpi-bits/README | 133 +++++++++++++++++++++++++++++++++
1 file changed, 133 insertions(+)
create mode 100644 tests/avocado/acpi-bits/README
diff --git a/tests/avocado/acpi-bits/README b/tests/avocado/acpi-bits/README
new file mode 100644
index 0000000000..4945dfc1f2
--- /dev/null
+++ b/tests/avocado/acpi-bits/README
@@ -0,0 +1,133 @@
+=============================================================================
+ACPI/SMBIOS AVOCADO TESTS USING BIOSBITS
+=============================================================================
+
+Biosbits is a software written by Josh Triplett that can be downloaded
+from https://biosbits.org/. The github codebase can be found here:
+https://github.com/biosbits/bits/tree/master. It is a software that executes
+the bios components such as acpi and smbios tables directly through acpica
+bios interpreter (a freely available C based library written by Intel,
+downloadable from https://acpica.org/ and is included with biosbits) without an
+operating system getting involved in between.
+There are several advantages to directly testing the bios in a real physical
+machine or VM as opposed to indirectly discovering bios issues through the
+operating system. For one thing, the OSes tend to hide bios problems from the
+end user. The other is that we have more control of what we wanted to test
+and how by directly using acpica interpreter on top of the bios on a running
+system. More details on the inspiration for developing biosbits and its real
+life uses can be found in (a) and (b).
+This directory contains tests written in python using avocado framework that
+exercizes the QEMU bios components using biosbits and reports test failures.
+For QEMU, we maintain a fork of bios bits in gitlab along with all the
+dependent submodules:
+https://gitlab.com/qemu-project/biosbits-bits
+This fork contains numerous fixes, a newer acpica and changes specific to
+running this avocado QEMU tests using bits. The author of this document
+is the sole maintainer of the QEMU fork of bios bits repo.
+
+Under the directory tests/avocado/, acpi-bits.py is a QEMU avocado test that
+drives all this.
+
+A brief description of the various test files follows.
+
+Under tests/avocado/ as the root we have:
+
+├── acpi-bits
+│ ├── bits-config
+│ │ └── bits-cfg.txt
+│ ├── bits-tests
+│ │ ├── smbios.py2
+│ │ ├── smilatency.py2
+│ │ ├── testacpi.py2
+│ │ └── testcpuid.py2
+│ └── README
+├── acpi-bits.py
+
+tests/avocado:
+ - acpi-bits.py: This is the main python avocado test script that generates a
+ biosbits iso. It then spawns a QEMU VM with it, collects the log and reports
+ test failures. This is the script one would be interested in if they wanted
+ to add or change some component of the log parsing, add a new command line
+ to alter how QEMU is spawned etc. Test writers typically would not need to
+ modify this script unless they wanted to enhance or change the log parsing
+ for their tests. Following environment variables are used in this test:
+ - V=1 : This enables verbose mode for the test. It dumps the entire log
+ from bios bits and also more details in case failure happens. It is
+ useful for debugging the test failures or tests themselves.
+
+ In order to run this test, please perform the following steps from the QEMU
+ build directory:
+
+ $ make check-venv (needed only the first time to create the venv)
+ $ ./tests/venv/bin/avocado run -t acpi tests/avocado
+
+ The above will run all acpi avocado tests including this one.
+ In order to run the individual tests, perform the following:
+
+ $ ./tests/venv/bin/avocado run tests/avocado/acpi-bits.py --tap -
+
+ The above will produce output in tap format. You can omit "--tap -" in the
+ end and it will produce output like the following:
+
+ $ ./tests/venv/bin/avocado run tests/avocado/acpi-bits.py
+ Fetching asset from
tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
+ JOB ID : eab225724da7b64c012c65705dc2fa14ab1defef
+ JOB LOG :
/home/anisinha/avocado/job-results/job-2022-10-10T17.58-eab2257/job.log
+ (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: PASS
(33.09 s)
+ RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 |
CANCEL 0
+ JOB TIME : 39.22 s
+
+ You can inspect the log file for more information about the run or in order
+ to diagnoze issues. If you pass V=1 in the environment, more diagnostic logs
+ would be found in the test log.
+
+tests/avocado/acpi-bits:
+ - README: This text file.
+
+tests/avocado/acpi-bits/bits-config:
+ This location contains biosbits config files that determine how the software
+ runs the tests.
+ - bits-config.txt: this is the biosbits config file that determines what tests
+ or actions are performed by bits. The description of the config options are
+ provided in the file itself.
+
+tests/avocado/acpi-bits/bits-tests:
+ This directory contains biosbits python based tests that are run from within
+ the biosbits environment in the spawned VM. New additions of test cases can
+ be made in the appropriate test file. For example, new acpi tests can go
+ into testacpi.py2 and one would call testsuite.add_test() to register the
new
+ test so that it gets executed as a part of the ACPI tests.
+ It might be occasionally necessary to disable some subtests or add a new
+ test that belongs to a test suite not already present in this directory. To
+ do this, please clone the bits source from
+ https://gitlab.com/qemu-project/biosbits-bits/-/tree/qemu-bits.
+ Note that this is the "qemu-bits" branch and not the "bits" branch of the
+ repository. "qemu-bits" is the branch where we have made all the QEMU
+ specific enhancements and we must use the source from this branch only.
+ Copy the test suite/script that needs modification (addition of new tests
+ or disabling them) from python directory into this directory. For
+ example, in order to change cpuid related tests, copy the following
+ file into this directory and rename it with .py2 extension:
+
https://gitlab.com/qemu-project/biosbits-bits/-/blob/qemu-bits/python/testcpuid.py
+ Then make your additions and changes here. Hence, the steps are:
+
+ step (a): copy unmodified test script to this directory from bits source.
+ step (b): perform modifications to the test.
+ Commit (a) and (b) should go under separate commits so that the original
+ test script and the changes we have made are separated and clear.
+
+ The test framework will then use your modified test script to run the test.
+ No further changes would be needed. Please check the logs to make sure that
+ appropriate changes have taken effect.
+
+ The tests have an extension .py2 in order to indicate that
+ (a) they are python2.7 based scripts and not python 3 scripts.
+ (b) they are run from within the bios bits VM and is not subjected to QEMU
+ build/test python script maintainance and dependency resolutions.
+ (c) They need not be loaded by avocado framework when running tests.
+
+Author: Ani Sinha <ani@anisinha.ca>
+
+References:
+(a)
https://blog.linuxplumbersconf.org/2011/ocw/system/presentations/867/original/bits.pdf
+(b) https://www.youtube.com/watch?v=36QIepyUuhg
--
2.34.1
- [PATCH v5 06/10] acpi/tests/avocado/bits: disable smilatency test since it does not pass everytime, (continued)
[PATCH v5 09/10] acpi/tests/avocado/bits: add a README file to describe the test,
Ani Sinha <=
[PATCH v5 10/10] MAINTAINERS: add myself as the maintainer for acpi biosbits avocado tests, Ani Sinha, 2022/10/18