[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH] tests/avocado: Disable the test_sbsaref_edk2_firmware in the
|
From: |
Peter Maydell |
|
Subject: |
Re: [PATCH] tests/avocado: Disable the test_sbsaref_edk2_firmware in the gitlab-CI |
|
Date: |
Mon, 10 Jul 2023 15:54:47 +0100 |
On Mon, 10 Jul 2023 at 15:43, Thomas Huth <thuth@redhat.com> wrote:
>
> The test fails occasionally, see e.g.:
>
> https://gitlab.com/thuth/qemu/-/jobs/4196177756#L489
> https://gitlab.com/thuth/qemu/-/jobs/4623296271#L290
>
> Disable it at least in the gitlab-CI to avoid failing CI
> pipelines due to this problem.
I'm not really a fan of disabling flaky tests only in the
gitlab CI. It means that plain "make check-avocado"
is useless to the average developer because it can
have all manner of broken or flaky tests in it that
never get tested by our CI. In fact I've had to develop
the habit of running "GITLAB_CI=1 make check-avocado"
so I can run the CI tests that we think work and not
get distracted by intermittent failures.
If we want "disable this test except for if you're
trying to investigate why it's flaky" then we should
have the test be opt-in, not opt-out, the same way
we do with QEMU_TEST_FLAKY_TESTS in the qtests.
thanks
-- PMM