[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name
From: |
Cleber Rosa |
Subject: |
Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping |
Date: |
Wed, 17 Oct 2018 16:59:25 -0400 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.0 |
On 10/17/18 4:46 PM, Murilo Opsfelder Araujo wrote:
> On Wed, Oct 17, 2018 at 04:09:51PM -0300, Eduardo Habkost wrote:
>> On Wed, Oct 17, 2018 at 07:40:51PM +0100, Peter Maydell wrote:
>>> On 17 October 2018 at 18:38, Cleber Rosa <address@hidden> wrote:
>>>>
>>>>
>>>> On 10/17/18 12:29 PM, Eduardo Habkost wrote:
>>>>> On Wed, Oct 17, 2018 at 01:34:41PM +0100, Peter Maydell wrote:
>>>>>> So, why does the test code need to care? It's not clear
>>>>>> from the patch... My expectation would be that you'd
>>>>>> just test all the testable target architectures,
>>>>>> regardless of what the host architecture is.
>>>>>
>>>>> I tend to agree. Maybe the right solution is to get rid of the
>>>>> os.uname(). I think the default should be testing all QEMU
>>>>> binaries that were built, and the host architecture shouldn't
>>>>> matter.
>>>
>>> Yes, looking at os.uname() also seems like an odd thing
>>> for the tests to be doing here. The test framework
>>> should be as far as possible host-architecture agnostic.
>>> (For some of the KVM cases there probably is a need to
>>> care, but those are exceptions, not the rule.)
>>>
>>>> I'm in favor of exercising all built targets, but that seems to me to be
>>>> on another layer, above the test themselves. This change is about the
>>>> behavior of a test when not told about the target arch (and thus binary)
>>>> it should use.
>>>
>>> At that level, I think the right answer is "tell the user
>>> they need to specify the qemu executable they are trying to test".
>>> In particular, there is no guarantee that the user has actually
>>> built the executable for the target that corresponds to the
>>> host, so it doesn't work to try to default to that anyway.
>>
>> Agreed. However, I don't see when exactly this message would be
>> triggered. Cleber, on which use cases do you expect
>> pick_default_qemu_bin() to be called?
>>
>> In an ideal world, any testing runner/tool should be able to
>> automatically test all binaries by default. Can Avocado help us
>> do that? (If not, we could just do it inside a
>> ./tests/acceptance/run script).
>>
>> --
>> Eduardo
>>
>
> Why don't we add a variants file to QEMU's tree listing all known possible
> paths
> for QEMU binary that can result from a build? For example:
>
> $ cat tests/acceptance/variants
> qemu_bin:
> - ppc-softmmu/qemu-system-ppc
> - ppc64-softmmu/qemu-system-ppc64
> - x86_64-softmmu/qemu-system-x86_64
>
That's possible, and even proposed on the "basic arch support" series.
But this is a higher level question IMO (it exceeds the test boundaries).
> Then avocado could multiplex these variants file and call
> ./tests/acceptance/run
> for each value of qemu_bin. `run` script could skip and return if $qemu_bin
> doesn't exist. This approach also allows user forcing a value of qemu_bin
> when
> calling `run` manually, for example:
>
> ./tests/acceptance/run --qemu_bin=/path/to/your/qemu-system-blah ...
>
> This ./tests/acceptance/run can serve as an entry point to run all the tests.
> If more parameters are considered mandatory in the future, the logic can be
> placed there.
>
I'm completely favorable to having extra scripts or make rules that
define a standard "job" behavior. In fact, I think we'll end up having
many of those. "make check-acceptance" is just the first one.
But being able to running individual tests should still be possible, and
easy IMO.
Regards,
- Cleber.
> --
> Murilo
>
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, (continued)
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Eduardo Habkost, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Cleber Rosa, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Eduardo Habkost, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Cleber Rosa, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Eduardo Habkost, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Cleber Rosa, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Murilo Opsfelder Araujo, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping,
Cleber Rosa <=
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Eduardo Habkost, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Cleber Rosa, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Eduardo Habkost, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Murilo Opsfelder Araujo, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Eduardo Habkost, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Wainer dos Santos Moschetta, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Cleber Rosa, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Eduardo Habkost, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Eduardo Habkost, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Cleber Rosa, 2018/10/17