Re: iotests and python dependencies

[Kevin Wolf]

Am 04.05.2022 um 21:38 hat John Snow geschrieben:
> Howdy!
> 
> So, I want to finally delete python/qemu/qmp from qemu.git, and this
> creates a small problem -- namely, iotests needs access to it in order
> to run the python-based tests.
> 
> What I think needs to happen is that we create a virtual environment
> that installs python/qemu/. The reason this cannot be done with
> PYTHONPATH alone anymore is because the qmp package itself won't be
> there anymore, we need an installer like `pip` to actually fetch it
> for us and put it somewhere. (i.e., we need to process the
> dependencies of python/qemu now and can't treat it as a pre-installed
> location.)
> 
> Avocado tests are already creating a venv for the purposes of
> installing and running Avocado. We can amend e.g. "../../python" to
> tests/requirements.txt and the Avocado environment is A-OK good-to-go.
> The Makefile magic for avocado tests creates a venv-per-build.

"per build" sounded pretty bad, because I thought it meant building a
new venv every time I run either 'make' or the tests. This would
obviously be very noticable for short-running tests like some iotests.
But fortunately this is not what it does, it keeps the venv around in
the build directory. So it's only per build directory really, which I
guess is fine.

> It seems to work well enough. One thing to note here is that the
> supported invocation for avocado tests is only through the Makefile,
> which handles creating and entering the venv to make the command
> seamless.
> 
> iotests, however, manages its own execution environment with
> testenv.py, and we support running iotests from outside of the
> Makefile, for example by going to $build/tests/qemu-iotests and
> running ./check.

Yes, and I agree that this is important.

> Now ... I could update testenv.py to be smart enough to create and
> enter a python venv, and have even prototyped this. It seems to work
> pretty well! This approach seemed like the least invasive to how
> iotests are expected to be run and used. But a downside with this
> approach is that now avocado tests and iotests are each managing their
> own python venv. Worse, vm-tests and device-crash-test are still
> unhandled entirely.

Is there a reason why testenv.py couldn't enter just the shared venv in
build/tests/venv?

If not, I guess it would be enough if iotests just checks that the venv
exists and all of the dependencies are there in the right version and
error out if not, telling the user to run 'make check-venv'.

Or actually, it could just unconditionally run 'make check-venv' by
itself, which is probably easier to implement than checking the
dependencies and more convenient for the user, too.

(One more detail: 'make check-venv GIT_SUBMODULES_ACTION=ignore' seems
to make it pretty much instantaneous if the venv is current, but leaving
the submodule mechanism enabled adds a noticable delay.)

> I'd like to find a solution where I create a unified python testing
> venv tied to the build shared by avocado, iotests, vm-tests and
> device-crash-test. I'm not completely sure how exactly I'll manage
> that right now, but I wanted to throw this out there in case there are
> some requirements I might be overlooking.
> 
> I think vm-tests and avocado-tests can both have a venv created for
> them and activated before the test runs. device-crash-test I believe
> will need a script change in the gitlab ci yaml. iotests is somewhat
> unique in that it needs to run both by manual invocation and from
> makefile invocations. If I want a shared VM between all of these, I'll
> need to isolate the create-and-enter-venv logic somewhere where it can
> be shared both inside and outside of a Makefile.

If just calling 'make' isn't an option, then moving that part out into a
separate script probably wouldn't be too hard either. But 'make' has the
advantage that it already contains the check if the venv is already
there and requirements.txt hasn't changed, which you would have to
replicate otherwise.

Kevin