Re: [PATCH v3 06/10] iotests: add testfinder.py

[Kevin Wolf]

Am 07.05.2020 um 19:43 hat Vladimir Sementsov-Ogievskiy geschrieben:
> 21.04.2020 19:56, Kevin Wolf wrote:
> > Am 21.04.2020 um 09:35 hat Vladimir Sementsov-Ogievskiy geschrieben:
> > > Add python script with new logic of searching for tests:
> > > 
> > > Current ./check behavior:
> > >   - tests are named [0-9][0-9][0-9]
> > >   - tests must be registered in group file (even if test doesn't belong
> > >     to any group, like 142)
> > > 
> > > Behavior of new test:
> > >   - group file is dropped
> > >   - tests are searched by file-name instead of group file, so it's not
> > >     needed more to "register the test", just create it with name
> > >     *-test. Old names like [0-9][0-9][0-9] are supported too, but not
> > >     recommended for new tests
> > 
> > I wonder if a tests/ subdirectory instead of the -test suffix would
> > organise things a bit better.
> > 
> 
> It will make more difficult to import iotests.py.. Calling common.rc from
> bash tests will need to be modified too.
> 
> So, we'll need additional line in all python tests:
> 
> sys.path.append(os.path.join(os.path.dirname(__file__), '..'))

Hm, yes, this is nasty.

Would it be any better have a wrapper script that just imports the
actual test so that the import path would still contain the main
qemu-iotests/ directory?

> which doesn't seem to be good practice.. So, instead we'd better call
> tests with PYTHONPATH set appropriately..

This is another option, especially if we do want to create a lib/.

> and modify bash tests to do
> . ../common.rc
> . ../common.filter
> <etc>
> 
> or again, better to export BASH_TEST_LIB directory.

I don't think this is necessary because the working directory wouldn't
change, so bash scripts should just keep working as before.

> Is it worth doing?
> 
> I think, there are two variants:
> 
> 1) keep as is: all in one directory, add *-test notation

I think it would make it rather hard to find the files that belong to
the test harness implementation between all the tests. Currently, 'ls'
in the qemu-iotests directory is still kind of usable because all the
test cases are at the start and everything that comes later is not a
test.

> 2) go further and restructure to something like:
> 
> iotests/<only check script and its dependencies here>
> iotests/tests/<all-tests-here>
> iotests/lib/python/iotests.py
> iotests/lib/bash/<common.rc and friends here>
> 
> And then, check script will export PYTHONPATH and BASH_TEST_LIB
> variables.

I think leaving everything except named tests where it is, but setting
PYTHONPATH or having a wrapper script, is still a third option that's
worth considering. It sounds like the most attrative option to me.

Kevin