[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name
From: |
Eduardo Habkost |
Subject: |
Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping |
Date: |
Wed, 17 Oct 2018 16:48:33 -0300 |
User-agent: |
Mutt/1.9.2 (2017-12-15) |
On Wed, Oct 17, 2018 at 03:25:34PM -0400, Cleber Rosa wrote:
>
>
> On 10/17/18 3:09 PM, Eduardo Habkost wrote:
> > On Wed, Oct 17, 2018 at 07:40:51PM +0100, Peter Maydell wrote:
> >> On 17 October 2018 at 18:38, Cleber Rosa <address@hidden> wrote:
> >>>
> >>>
> >>> On 10/17/18 12:29 PM, Eduardo Habkost wrote:
> >>>> On Wed, Oct 17, 2018 at 01:34:41PM +0100, Peter Maydell wrote:
> >>>>> So, why does the test code need to care? It's not clear
> >>>>> from the patch... My expectation would be that you'd
> >>>>> just test all the testable target architectures,
> >>>>> regardless of what the host architecture is.
> >>>>
> >>>> I tend to agree. Maybe the right solution is to get rid of the
> >>>> os.uname(). I think the default should be testing all QEMU
> >>>> binaries that were built, and the host architecture shouldn't
> >>>> matter.
> >>
> >> Yes, looking at os.uname() also seems like an odd thing
> >> for the tests to be doing here. The test framework
> >> should be as far as possible host-architecture agnostic.
> >> (For some of the KVM cases there probably is a need to
> >> care, but those are exceptions, not the rule.)
> >>
> >>> I'm in favor of exercising all built targets, but that seems to me to be
> >>> on another layer, above the test themselves. This change is about the
> >>> behavior of a test when not told about the target arch (and thus binary)
> >>> it should use.
> >>
> >> At that level, I think the right answer is "tell the user
> >> they need to specify the qemu executable they are trying to test".
> >> In particular, there is no guarantee that the user has actually
> >> built the executable for the target that corresponds to the
> >> host, so it doesn't work to try to default to that anyway.
> >
> > Agreed. However, I don't see when exactly this message would be
> > triggered. Cleber, on which use cases do you expect
> > pick_default_qemu_bin() to be called?
> >
>
> When a test is run ad-hoc. You even suggested that tests could/should
> be executable.
>
> > In an ideal world, any testing runner/tool should be able to
> > automatically test all binaries by default. Can Avocado help us
> > do that? (If not, we could just do it inside a
> > ./tests/acceptance/run script).
> >
>
> Avocado can do that indeed. But I'm afraid that's not the main issue.
> Think of the qemu-iotests: do we want a "check" command to run all
> tests with all binaries?
Good question. That would be my first expectation, but I'm not
sure.
Pro: testing all binaries by default would cause less confusion
than picking a random QEMU binary.
Con: testing all binaries may be inconvenient for quickly
checking if a test case works. (I'm not convinced this is a
problem. If you don't want to test everything, you probably
already have a short target list in your ./configure line?)
Pro: testing a single binary using uname() is already
implemented.
Con: making `avocado run` automatically generate variants of a
test case may take some effort?
--
Eduardo
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, (continued)
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Philippe Mathieu-Daudé, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Peter Maydell, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Eduardo Habkost, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Eduardo Habkost, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Cleber Rosa, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping,
Eduardo Habkost <=
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Cleber Rosa, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Eduardo Habkost, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Cleber Rosa, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Eduardo Habkost, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Murilo Opsfelder Araujo, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Cleber Rosa, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Eduardo Habkost, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Cleber Rosa, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Eduardo Habkost, 2018/10/17
- Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping, Murilo Opsfelder Araujo, 2018/10/17