Re: [PATCH v3 4/5] qemu-iotests: let "check" spawn an arbitrary test command

[Max Reitz]

On 30.03.21 12:44, Max Reitz wrote:
> On 30.03.21 12:38, Max Reitz wrote:
> > On 26.03.21 16:05, Max Reitz wrote:
> > > On 26.03.21 15:23, Paolo Bonzini wrote:
> > > > Right now there is no easy way for "check" to print a reproducer 
> > > > command.
> > > > Because such a reproducer command line would be huge, we can instead 
> > > > teach
> > > > check to start a command of our choice.  This can be for example a 
> > > > Python
> > > > unit test with arguments to only run a specific subtest.
> > > >
> > > > Move the trailing empty line to print_env(), since it always looks 
> > > > better
> > > > and one caller was not adding it.
> > > >
> > > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> > > > Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> > > > Tested-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
> > > > Message-Id: <20210323181928.311862-5-pbonzini@redhat.com>
> > > > ---
> > > >   tests/qemu-iotests/check         | 18 +++++++++++++++++-
> > > >   tests/qemu-iotests/testenv.py    |  3 ++-
> > > >   tests/qemu-iotests/testrunner.py |  1 -
> > > >   3 files changed, 19 insertions(+), 3 deletions(-)
> > > >
> > > > diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
> > > > index d1c87ceaf1..df9fd733ff 100755
> > > > --- a/tests/qemu-iotests/check
> > > > +++ b/tests/qemu-iotests/check
> > > > @@ -19,6 +19,9 @@
> > > >   import os
> > > >   import sys
> > > >   import argparse
> > > > +import shutil
> > > > +from pathlib import Path
> > > > +
> > > >   from findtests import TestFinder
> > > >   from testenv import TestEnv
> > > >   from testrunner import TestRunner
> > > > @@ -101,7 +104,7 @@ def make_argparser() -> argparse.ArgumentParser:
> > > >                          'rerun failed ./check command, starting 
> > > > from the '
> > > >                          'middle of the process.')
> > > >       g_sel.add_argument('tests', metavar='TEST_FILES', nargs='*',
> > > > -                       help='tests to run')
> > > > +                       help='tests to run, or "--" followed by a 
> > > > command')
> > > >       return p
> > > > @@ -114,6 +117,19 @@ if __name__ == '__main__':
> > > >                     imgopts=args.imgopts, misalign=args.misalign,
> > > >                     debug=args.debug, valgrind=args.valgrind)
> > > > +    if len(sys.argv) > 1 and sys.argv[-len(args.tests)-1] == '--':
> > > > +        if not args.tests:
> > > > +            sys.exit("missing command after '--'")
> > > > +        cmd = args.tests
> > > > +        env.print_env()
> > > > +        exec_path = Path(shutil.which(cmd[0]))
> > >
> > > 297 says:
> > >
> > > check:125: error: Argument 1 to "Path" has incompatible type 
> > > "Optional[str]"; expected "Union[str, _PathLike[str]]"
> > > Found 1 error in 1 file (checked 1 source file)
> > >
> > > Normally I’d assert this away, but actually I think the returned 
> > > value should be checked and we should print an error if it’s None.  
> > > (Seems like shutil.which() doesn’t raise an exception if there is no 
> > > such command, it just returns None.)
> > >
> > > Max
> > >
> > > > +        if exec_path is None:
> > > > +            sys.exit('command not found: ' + cmd[0])
> >
> > Oh, I see, the intent to print an error is actually there.  The 
> > problem is just that Path(None) throws an exception, so we must check 
> > shutil.which()’s return value.
> >
> > I’ll squash this in if you don’t mind:
> >
> > diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
> > index df9fd733ff..e2230f5612 100755
> > --- a/tests/qemu-iotests/check
> > +++ b/tests/qemu-iotests/check
> > @@ -122,9 +122,10 @@ if __name__ == '__main__':
> >               sys.exit("missing command after '--'")
> >           cmd = args.tests
> >           env.print_env()
> > -        exec_path = Path(shutil.which(cmd[0]))
> > -        if exec_path is None:
> > +        exec_pathstr = shutil.which(cmd[0])
> > +        if exec_pathstr is None:
> >               sys.exit('command not found: ' + cmd[0])
> > +        exec_path = Path(exec_pathstr)
> >           cmd[0] = exec_path.resolve()
> >           full_env = env.prepare_subprocess(cmd)
> >           os.chdir(Path(exec_path).parent)
> >
> > > > +        cmd[0] = exec_path.resolve()
> > > > +        full_env = env.prepare_subprocess(cmd)
> > > > +        os.chdir(Path(exec_path).parent)
>
> Oh, and this Path() does nothing, I presume, so I’m going to replace it 
> with just “exec_path”.

On third thought, the pathlib doc says:

> If you want to walk an arbitrary filesystem path upwards, it is
> recommended to first call Path.resolve() so as to resolve symlinks and
> eliminate “..” components.

So I guess the best would be to make it “exec_path = 
Path(exec_pathstr).resolve()”.

I’d also prefer if cmd[0] was a string and not a Path object 
(Path.resolve returns a Path).  os.execve() can work with Path objects 
as of 3.6 (which is the minimum version we require), but 
prepare_subprocess() expects a list of strings.  (I don’t know why mypy 
doesn’t complain.  I presume it just can’t resolve cmd's type.)

So here’s the full diff:

diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
index df9fd733ff..7c9d3a0852 100755
--- a/tests/qemu-iotests/check
+++ b/tests/qemu-iotests/check
@@ -122,12 +122,13 @@ if __name__ == '__main__':
             sys.exit("missing command after '--'")
         cmd = args.tests
         env.print_env()
-        exec_path = Path(shutil.which(cmd[0]))
-        if exec_path is None:
+        exec_pathstr = shutil.which(cmd[0])
+        if exec_pathstr is None:
             sys.exit('command not found: ' + cmd[0])
-        cmd[0] = exec_path.resolve()
+        exec_path = Path(exec_pathstr).resolve()
+        cmd[0] = str(exec_path)
         full_env = env.prepare_subprocess(cmd)
-        os.chdir(Path(exec_path).parent)
+        os.chdir(exec_path.parent)
         os.execve(cmd[0], cmd, full_env)

     testfinder = TestFinder(test_dir=env.source_iotests)


But now these are so many changes that I feel uncomfortable making this 
change myself.  This series only affects the iotests, so AFAIU we are in 
no hurry to get this into rc1, and it can still go into rc2.

Max