Re: [PATCH 0/3] iotests: multiprocessing!!

[Vladimir Sementsov-Ogievskiy]

06.12.2021 21:37, John Snow wrote:
> On Fri, Dec 3, 2021 at 7:22 AM Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com 
> <mailto:vsementsov@virtuozzo.com>> wrote:
>
>     Hi all!
>
>     Finally, I can not stand it any longer. So, I'm happy to present
>     multiprocessing support for iotests test runner.
>
>     testing on tmpfs:
>
>     Before:
>
>     time check -qcow2
>     ...
>     real    12m28.095s
>     user    9m53.398s
>     sys     2m55.548s
>
>     After:
>
>     time check -qcow2 -j 12
>     ...
>     real    2m12.136s
>     user    12m40.948s
>     sys     4m7.449s
>
>
> VERY excellent. And this will probably flush a lot more bugs loose, too. (Which 
> I consider a good thing!)

Thanks!)

> We could look into utilizing it for 'make check', but we'll have to be prepared 
> for a greater risk of race conditions on the CI if we do. But... it's seriously 
> hard to argue with this kind of optimization, very well done!

I thought about this too. I think, we can at least passthrought -j flag of "make -j9 
check" to ./check

I think, CIs mostly call make check without -j flag. But I always call make -j9 check. And it 
always upset me that all tests run in parallel except for iotests. So if it possible to detect that 
we are called through "make -j9 check" and use "-j 9" for iotests/check in this 
case, it would be good.

>     Hmm, seems -j 6 should be enough. I have 6 cores, 2 threads per core.
>     Anyway, that's so fast!
>
>     Vladimir Sementsov-Ogievskiy (3):
>        iotests/testrunner.py: add doc string for run_test()
>        iotests/testrunner.py: move updating last_elapsed to run_tests
>        iotests: check: multiprocessing support
>
>       tests/qemu-iotests/check         |  4 +-
>       tests/qemu-iotests/testrunner.py | 86 ++++++++++++++++++++++++++++----
>       2 files changed, 80 insertions(+), 10 deletions(-)
>
>     -- 
>     2.31.1


--
Best regards,
Vladimir