bug#63622: lisp/progmodes/python.el: performance regression introduced by multiline font-lock

[Eli Zaretskii]

> From: Tom Gillespie <tgbugs@gmail.com>
> Date: Sat, 20 May 2023 20:14:19 -0700
> 
> The changes in 4915ca5dd4245a909c046e6691e8d4a1919890c8 have
> introduced a significant performance regression when editing python
> code that contains dictionary structures across multiple lines.
> 
> The current behavior makes Emacs unusable when editing python
> dictionaries with more than 20 or so lines. I would suggest reverting
> the commit until the performance issue can be addressed.

If the problem is so severe, I wonder how come this comes up only now,
9 months after those changes were installed.  It probably means these
cases are quite rare in practice.  Nevertheless, it would be good to
solve them, of course.

FWIW, python-ts-mode doesn't show performance issues in the examples
you posted.

> The literal dictionary below is sufficient to demonstrate the issue
> and if you bisect and compare the behavior to the immediately prior
> commit 31e32212670f5774a6dbc0debac8854fa01d8f92 the difference is
> clear. Open the file and hit enter a couple of times and the lag is
> obvious (if you can't detect the issue try doubling the number of
> lines at the deepest nesting level from 25 to 50).
> 
> By profiling and varying the number of repeated lines (e.g. by
> doubling them) it appears that the issue is some lurking quadratic
> behavior in syntax-ppss as a result of the changes in 4914ca. In my
> testing 25, 50, and 100 lines take 100ms, 800ms, and 5 seconds
> respectively to insert a new line while the cursor is inside the outer
> most paren.
> 
> Collapsing all the structures into one line hides the issue. The
> longer each individual line the more rapid the slowdown.
> 
> The example below is not syntactically correct python, however it
> highlights the issue in a way that is clearer than it would be
> otherwise.

Any chance of your posting some real-life Python code where the issue
rears its head?  I mean, real-life code that makes sense, not just
syntactically correct code invented to make a point?

> Despite my previous speculation about the regexp,
> the issue is in python-font-lock-extend-region. When
> replaced with a no-op the performance returns to normal.

kobarity, could you please look into this ASAP?  Emacs 29.1 is in late
stages of pretest, and I'd like to have this resolved, one way or
another, soon enough.  TIA.

P.S.  Tom, please don't change the Subject when posting followups,
please use the same Subject for all your messages that discuss this
bug.

Also, for the record, please state which Emacs version are you using.
You didn't use "M-x report-emacs-bug" to submit the bug report, so
this and other important information is missing from your OP.