Am 18.05.2022 um 01:28 hat John Snow geschrieben:
> Hi Kevin,
>
> I remember that you wanted some minimum Niceness threshold in order to
> agree to me removing iotest 297.
>
> I've already moved it onto GitLab CI in the form of the
> check-python-pipenv job, but I recall you wanted to be able to run it
> locally as well before agreeing to axe 297. I remember that you didn't
> think that running "make check-pipenv" from the python directory was
> sufficiently Nice enough.
>
> Do you need it to be part of "make check", or are you OK with
> something like "make check-python" from the build directory?
>
> I have a bit more work to do if you want it to be part of 'make check'
> (if you happen to have the right packages installed), but it's pretty
> easy right now to give you a 'make check-python' (where I just
> forcibly install those packages to the testing venv.)
Hm, what is the reason for 'make check-python' not being part of 'make
check'?
Oh, it just needs more logic so that it performs correctly in RPM building environments. As a manual test, I'm free to just grab stuff from PyPI and build a venv to some precise specification and automate it. This is how "check-pipenv" and "check-tox" work. The RPM environment can't dial out to PyPI, so it shouldn't try any venv-based tests by default.
To wire it up to "make check" by *default*, I believe I need to expand the configure script to poll for certain requisites and then create some wrapper script of some kind that only engages the python tests if the requisites were met ... and I lose some control over the mypy/pylint versioning windows. I have to tolerate a wider versioning, or it'll never get run in practice.
I have some reluctance to doing this, because pylint and mypy change so frequently that I don't want "make check" to fail spuriously in the future.
(In practice, these failures occur 100% of the time when I am on vacation.)
The gitlab ci job check-python-tox pulls whatever the latest and greatest are, and these jobs fail so constantly we had to mark the job as optional. The check-pipenv job by contrast is extremely stable (its still must-pass) because it can concoct its own lil' universe.
So, I can add something to make check by default but it needs some scaffolding to skip the test based on environment, and I have some reliability concerns.
Ultimately, I don't believe tolerating a wide matrix for mypy/pylint really adds any value to 297; it only really matters if a specific environment comes up green, and that a developer like you or I can replicate that test locally and quickly.
That said ... maybe I can add a controlled venv version of "check-python" and just have a --disable-check-python or something that spec files can opt into. Maybe that will work well enough?
i.e. maybe configure can check for the presence of pip, the python venv module (debian doesnt ship it standard...), and PyPI connectivity and if so, enables the test. Otherwise, we skip it.
Something like that.
I'm currently running two things locally, 'make check' (which is the
generic one that everyone should run) and iotests (for which it is
reasonable enough that I need to run it separately because it's the
special thing for my own subsystem).
Pretty much exactly what I do. (Except I run the python tests these days, too.)
Now adding a third one 'make check-python' certainly isn't the end of
the world, but it's not really something that is tied to my subsystem
any more. Having to run test cases separately for other subsystems
doesn't really scale for me, so I would prefer not to start doing that.
I can usually get away with not running the more special tests of other
subsystems before the pull request because I'm unlikely to break things
in other subsystems, but Python style warnings are easy to get.
Reasonable. I already forget to run things like avocado and vm tests, and I am sympathetic to not wanting to expand the list of manually run tests.
(What avocado and vm tests have in common is that they need to fetch stuff from the internet, which I am learning makes them unsuitable for make check, which must work without internet. """Coincidentally""", tests that require internet seem to break an awful lot more often because they are getting run a lot less and in fewer places.)
If we're going to have 'make check-python' separate, but CI checks it,
we'll get pull requests that don't pass it and would then only be caught
by CI after a long test run, requiring a v2 pull request. I feel for
something that checks things like style (and will fail frequently on
code where nobody ran the check before), that's a bit too unfriendly.
Kevin
Got it. I'll see what I can come up with that checks the boxes for everyone, thanks for clarifying yours.
I want to make everything "just work" but I'm also afraid of writing too much magic crap that could break and frustrate people who have no desire to understand python packaging junk, so I'm trying to balance that.
(Parting thought: the python ecosystem mantra of "just use a venv!" unfortunately influences a lot of upstream developer attitude which is then increasingly hard to square with environments in which I cannot count on internet or the ability to spin up a venv, and thus my headache. Overwhelmingly, it seems to be the expectation that you'd just pin or vendor things like pylint/mypy, because they're not usually runtime deps for Python packages. Urgh, blegh.)
--js