Re: [PATCH v4 05/18] ci: explicitly skip I/O tests on alpine

[Philippe Mathieu-Daudé]

On 11/24/21 14:46, Daniel P. Berrangé wrote:
> On Wed, Nov 24, 2021 at 02:36:59PM +0100, Thomas Huth wrote:
>> On 24/11/2021 14.01, Daniel P. Berrangé wrote:
>>> The block I/O tests don't work on Alpine because their alternative libc
>>> impl emits different strings for errnos, which breaks the expected
>>> output matching. e.g.
>>>
>>> === IO: pattern 102
>>>   wrote 512/512 bytes at offset 512
>>>   512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>>> -qemu-img: Error while reading offset 0 of 
>>> blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: Input/output error
>>> +qemu-img: Error while reading offset 0 of 
>>> blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: I/O error
>>>   4
>>>   Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
>>>   Formatting 'TEST_DIR/t.IMGFMT.2', fmt=IMGFMT size=0
>>>
>>> Currently the I/O tests are skipped as a side effect of the Alpine image
>>> containing a minimal busybox 'sed' binary, rather than GNU Sed. This is
>>> a fragile assumption that will be invalidated when the dockerfile is
>>> changed to be autogenerated from a standardized package list that
>>> includes GNU Sed.

"GNU sed" in lowercase? ("stream editor", 2 occurrences in description).

>>>
>>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>>> ---
>>>   .gitlab-ci.d/buildtest.yml | 2 +-
>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
>>> index 71d0f407ad..e1fe37e563 100644
>>> --- a/.gitlab-ci.d/buildtest.yml
>>> +++ b/.gitlab-ci.d/buildtest.yml
>>> @@ -24,7 +24,7 @@ check-system-alpine:
>>>         artifacts: true
>>>     variables:
>>>       IMAGE: alpine
>>> -    MAKE_CHECK_ARGS: check
>>> +    MAKE_CHECK_ARGS: check-unit check-qtest
>>
>> Hmm, that's just a work-around ... what if some user wants to run "make
>> check" on an Alpine installation that has the real GNU sed installed? ... I
>> think this rather requires some fixing in the iotests instead - or maybe the
>> related tests should simply not be in the "auto" group anymore?
> 
> Of course, the I/o tests should be fixed, but that's a big job that on
> one has volunteered for.
> 
> Taking the tests in question out of the "auto" group would be very
> detrimental for test coverage on other platforms. There are many
> affected tests so that is not desirable.
> 
> As explained above, the CI job is already skipping the I/O tests today
> as a side effect of only having Busybox sed. This change just makes
> that explicit in the CI config so it is clear that we're missing this
> coverage and won't accidentally break when GNU Sed appears in the
> dockerfile.

This sounds reasonable.

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>