bug#63622: lisp/progmodes/python.el: performance regression introduced by multiline font-lock

[Stefan Monnier]

> I agree.  It seems to me that it is not python-font-lock-extend-region
> itself that is slow, but rather font-lock's processing of the area
> extended by it.  So one workaround would be to limit the number of
> lines to be extended, as in the attached patch.  If this limit is
> exceeded, the area or the entire buffer must be font-locked manually
> later.  What do you think of this idea?

FWIW, I recommend against using
`font-lock-extend-after-change-region-function`.

E.g. in a case like `python-font-lock-assignment-statement-multiline-1`,
the current code may misfontify code because jit-lock may decide to first
call font-lock on a chunk that goes until:

    [
        a,
        b

and will call again font-lock later to fontify the rest:

    ] # (
        1,
        2
    )

and this can happen with no buffer modification at all (e.g. on the
initial fontification of a buffer).

You can use `font-lock-extend-region-functions` instead (which performs
the region-extension right before fontifying a chunk) to avoid this problem.
[ It won't help with the current performance issue, tho.  ]

`font-lock-extend-after-change-region-function` can also be costly when
a command makes many changes (since
`font-lock-extend-after-change-region-function` is called for every such
change rather than once at the end).
`font-lock-extend-region-functions` tends to be better behaved in this
respect (it's called once per chunk, and there's usually only
a single chunk (re)fontified per command, even after a command that makes
many changes),

One more thing: Tom mentioned a suspicion that the performance issue may
have to do with interaction with `syntax-ppss`.  This is odd, because
`syntax-ppss` and `syntax-propertize` should not be affected by
font-lock.


        Stefan