Re: [RFC PATCH 0/9] tests: run python tests under the build/tests/venv environment

[Paolo Bonzini]

On 5/13/22 17:39, John Snow wrote:
> On Fri, May 13, 2022, 6:21 AM Paolo Bonzini <pbonzini@redhat.com 
> <mailto:pbonzini@redhat.com>> wrote:
>
>     On 5/13/22 02:06, John Snow wrote:
>      > The only downside I am really frowning at is that I will have to
>      > replicate some "update the venv if it's outdated" logic that is
>     usually
>      > handled by the Make system in the venv bootstrapper. Still, I
>     think it's
>      > probably the only way to hit all of the requirements here without
>     trying
>      > to concoct a fairly complex Makefile.
>      >
>      > any thoughts? If not, I'll just move on to trying to hack up that
>      > version next instead.
>
>     I would not even bother with keeping the venv up to date.  Just
>     initialize
>
> I'm worried about this idea being very inconvenient for iterative 
> development of the python code.

Wouldn't "-e" mostly avoid the inconvenience?

> I'm not sure this makes sense. python/qemu will continue to exist 
> in-tree and will only ever be "internal" in that sense. It won't be 
> something you can wholesale install from pip.
>
> i.e. I plan to continue to break off pieces and upstream them, but I 
> intend to leave several modules as internal only.

Oh, that's what I was missing.  I thought long term all of it would come 
from pip.  But anyway...

> So I'm not sure "internal" vs "pip" makes sense config-wise, it's 
> intended to be a mixture of both, really.

... then neither "system" nor "pip" would not cover the parts of 
"python-qemu" that are always internal, i.e. currently only 
"python-qemu-qmp".  And after

 $python -m venv venv/

you would have

 $python -m pip install -e python/

(That's probably a better way to invoke a pip that's related to $python, 
at least until the venv exists).

By the way, where would you put the python-qemu-qmp submodule?

> But, I suppose this is how you'd like to address different venv setup 
> behaviors to accommodate spec builds vs dev builds - with a configure 
> flag of some kind.

Yes.

> (I suppose you'd then like to see configure error out if it doesn't have 
> the necessary requisites given the venv-style chosen?)
>
>     - use CONFIG_PYTHON_QEMU to enable/disable iotests in
>     tests/qemu-iotests/meson.build
>
> So it's just skipped if you don't have the reqs to make the venv? (Not 
> an error?)

That's usually what we do with missing bits yes; you can use 
--enable-python-qemu to force an error in that case.

>     - add a configure option --enable-avocado=
>     {system,pip,auto,enabled}/--disable-avocado and matching
>     CONFIG_AVOCADO=y to config-host.mak
>
>     - use it to enable/disable acceptance tests in tests/Makefile.include
>
> And this is how you propose eliminating the need for an always-present 
> avocado builddep.

Yep.

>     rm -rf venv/
>     $python -m venv venv/
>     do_pip venv/ enable_python_qemu qemu.qmp python/qemu -- qemu.qmp
>     do_pip venv/ enable_avocado avocado -- -r tests/requirements.txt
>
> Won't this rebuild the venv like *all of the time*, basically whenever 
> you see the "configuration is stale" message?

Yes, it would.  I think that's going to happen less and less since 
configure is on a serious diet; but it might still be annoying.  OTOH 
installing system packages (or also "pip install --user") will speed up 
creating the virtual env, because then pip will not be run in the venv.

> That both seems like way too often, *and* it wouldn't cover cases when 
> it really ought to be refreshed.

Which cases are those?

Paolo