bug#69385: 30.0.50; Long lines with bidi text slow down Emacs

[Stephen Berman]

On Sun, 25 Feb 2024 19:06:55 +0200 Eli Zaretskii <eliz@gnu.org> wrote:

>> Date: Sun, 25 Feb 2024 17:23:11 +0100
>> From:  Stephen Berman via "Bug reports for GNU Emacs,
>>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
>>
>> Displaying a buffer that contains a long line with bidirectional text
>> greatly slows down Emacs.  A simple reproduction is to copy the the
>> Arabic example from etc/HELLO (`C-h h'), yank it into a buffer
>> (fundamental-mode suffices), add " Hello ", and then create a single
>> line consisting of a large number of copies of these strings; on my
>> machine 500 copies clearly shows the slowdown, and with 800 copies it is
>> much worse.
>
> It is not clear what exactly constitutes a slow-down in this recipe.
> You describe how to create a test buffer, but not what you do
> afterwards to demonstrate slow-down.  Please fill the gaps.
>
> FTR, I tried C-f/C-b (no perceptible slow-down), C-v/M-v (likewise),
> and C-n/C-p (no slow-down at the beginning of buffer, considerable
> slow-down near the end: about 0.5 to 0.8 sec response time -- but this
> is a non-optimized build; your optimized build should see about 0.1
> sec to 0.2 sec).  Is this what you see?  If not, please tell what did
> you do and what did you see, and please describe it in detail and with
> timings if possible.

The first thing I tried, with both 500 and 800 copies, is M-> from
point-min: with 500 it took ~4 seconds, with 800 ~10 seconds, and each
time Emacs used 100% of one core for the duration (my machine has an
i7-8700 processor with 6 cores/12 threads).  After that even just
invoking commands like `C-x 1' or `M-x' and entering something in the
minibuffer shows a noticeable delay (several seconds) and 100% CPU.
When I typed `C-n' at point-min in the 500 copy buffer and held down the
key till the cursor froze, it took about a minute until it moved again
(to point-max), during which the core used by Emacs stayed at 100%.

>> There is no slowdown with a line of the same length consisting only of
>> RTL or only of LTR text, nor with the above test line when
>> bidi-display-reordering is set to nil in the buffer (but then, of
>> course, the Arabic is not displayed correctly).
>
> That's not what I see: setting bidi-display-reordering to nil doesn't
> affect the slow-down on my system in any perceptible way.

Very strange: here, after setting bidi-display-reordering to nil, M->
and M-< are practically instantaneous and even holding down C-n or C-p
traverses the entire buffer in a couple of seconds, with just a bit of
stuttering.

>> It seems that the long line optimizations added to Emacs 29 do not
>> work with bidirectional text.
>
> Long line optimizations don't kick in until the lines are longer than
> the value of long-line-threshold, by default 50000 characters.  Since
> 800 copies of the Arabic greeting don't reach that threshold, the
> optimizations are not in effect at all in this case.

Ah, ok.  Then this is a different long-line issue.

>> (FTR, I encountered this issue with a program of mine that generates
>> Emacs Lisp files containing such long lines with bidirectional text.
>> These files are not intended for display but I was examining one and
>> experienced the slowdown.)
>
> If your real-life cases are all with Arabic text, then the reason is
> likely not bidirectional reordering of RTL text by itself (although it
> does slow down redisplay to some degree), but the fact that all of the
> Arabic text gets shaped by HarfBuzz via composition-function-table,
> which involves calls from the C code of the display engine into Lisp,
> which then turns and calls back into C.  This is relatively slow, but
> we have no choice: Arabic shaping requires that all Arabic characters
> be handed to the shaping engine for rendering, otherwise the display
> will either be illegible for Arabic speakers or at least look ugly in
> their eyes.

I created a test buffer with 800 copies of the Hebrew HELLO text
concatenated with " Hello " and found similar slowdowns with M-> and C-n
as with the Arabic text.  IIUC displaying the Hebrew script does not
involve text shaping, or does it?

> For now, I see no bugs here.  Maybe if you tell more we will arrive at
> something that is a real problem, but what I've read and saw until now
> is what I would expect in these cases.

I can try other tests if you can suggest any you think will help.

Steve Berman