iotests and python dependencies

[John Snow]

Howdy!

So, I want to finally delete python/qemu/qmp from qemu.git, and this
creates a small problem -- namely, iotests needs access to it in order
to run the python-based tests.

What I think needs to happen is that we create a virtual environment
that installs python/qemu/. The reason this cannot be done with
PYTHONPATH alone anymore is because the qmp package itself won't be
there anymore, we need an installer like `pip` to actually fetch it
for us and put it somewhere. (i.e., we need to process the
dependencies of python/qemu now and can't treat it as a pre-installed
location.)

Avocado tests are already creating a venv for the purposes of
installing and running Avocado. We can amend e.g. "../../python" to
tests/requirements.txt and the Avocado environment is A-OK good-to-go.
The Makefile magic for avocado tests creates a venv-per-build. It
seems to work well enough. One thing to note here is that the
supported invocation for avocado tests is only through the Makefile,
which handles creating and entering the venv to make the command
seamless.

iotests, however, manages its own execution environment with
testenv.py, and we support running iotests from outside of the
Makefile, for example by going to $build/tests/qemu-iotests and
running ./check.

Now ... I could update testenv.py to be smart enough to create and
enter a python venv, and have even prototyped this. It seems to work
pretty well! This approach seemed like the least invasive to how
iotests are expected to be run and used. But a downside with this
approach is that now avocado tests and iotests are each managing their
own python venv. Worse, vm-tests and device-crash-test are still
unhandled entirely.

I'd like to find a solution where I create a unified python testing
venv tied to the build shared by avocado, iotests, vm-tests and
device-crash-test. I'm not completely sure how exactly I'll manage
that right now, but I wanted to throw this out there in case there are
some requirements I might be overlooking.

I think vm-tests and avocado-tests can both have a venv created for
them and activated before the test runs. device-crash-test I believe
will need a script change in the gitlab ci yaml. iotests is somewhat
unique in that it needs to run both by manual invocation and from
makefile invocations. If I want a shared VM between all of these, I'll
need to isolate the create-and-enter-venv logic somewhere where it can
be shared both inside and outside of a Makefile.

I'll see what I can cook up, but if you have any concerns or Cool
Ideas, lemme know. I want to make sure this is as painless as I can
think to make it.

Thanks,
--js