06.12.2021 21:37, John Snow wrote:
>
>
> On Fri, Dec 3, 2021 at 7:22 AM Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com <mailto:vsementsov@virtuozzo.com>> wrote:
>
> Hi all!
>
> Finally, I can not stand it any longer. So, I'm happy to present
> multiprocessing support for iotests test runner.
>
> testing on tmpfs:
>
> Before:
>
> time check -qcow2
> ...
> real 12m28.095s
> user 9m53.398s
> sys 2m55.548s
>
> After:
>
> time check -qcow2 -j 12
> ...
> real 2m12.136s
> user 12m40.948s
> sys 4m7.449s
>
>
> VERY excellent. And this will probably flush a lot more bugs loose, too. (Which I consider a good thing!)
Thanks!)
> We could look into utilizing it for 'make check', but we'll have to be prepared for a greater risk of race conditions on the CI if we do. But... it's seriously hard to argue with this kind of optimization, very well done!
I thought about this too. I think, we can at least passthrought -j flag of "make -j9 check" to ./check
I think, CIs mostly call make check without -j flag. But I always call make -j9 check. And it always upset me that all tests run in parallel except for iotests. So if it possible to detect that we are called through "make -j9 check" and use "-j 9" for iotests/check in this case, it would be good.
>
>
> Hmm, seems -j 6 should be enough. I have 6 cores, 2 threads per core.
> Anyway, that's so fast!
>
> Vladimir Sementsov-Ogievskiy (3):
> iotests/testrunner.py: add doc string for run_test()
> iotests/testrunner.py: move updating last_elapsed to run_tests
> iotests: check: multiprocessing support
>
> tests/qemu-iotests/check | 4 +-
> tests/qemu-iotests/testrunner.py | 86 ++++++++++++++++++++++++++++----
> 2 files changed, 80 insertions(+), 10 deletions(-)
>
> --
> 2.31.1
>
I'll also now add:
I tried to find a different workaround in just a few minutes, but that just made it clear that your solution was right. While I had it checked out, I ran it a few times and it looks good to me!
(And no new problems from the Python CI stuff, so it looks good to me.)