bug#69385: 30.0.50; Long lines with bidi text slow down Emacs

[Stephen Berman]

On Sun, 25 Feb 2024 21:31:19 +0200 Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Stephen Berman <stephen.berman@gmx.net>
>> Cc: 69385@debbugs.gnu.org
>> Date: Sun, 25 Feb 2024 19:03:24 +0100
>> 
>> The first thing I tried, with both 500 and 800 copies, is M-> from
>> point-min: with 500 it took ~4 seconds, with 800 ~10 seconds, and each
>> time Emacs used 100% of one core for the duration (my machine has an
>> i7-8700 processor with 6 cores/12 threads).  After that even just
>> invoking commands like `C-x 1' or `M-x' and entering something in the
>> minibuffer shows a noticeable delay (several seconds) and 100% CPU.
>> When I typed `C-n' at point-min in the 500 copy buffer and held down the
>> key till the cursor froze, it took about a minute until it moved again
>> (to point-max), during which the core used by Emacs stayed at 100%.
>
> To make sure we are testing the same: what is the value of point-max
> in that buffer?  I see something around 30k characters after making
> 800 copies.

With 800 repetitions of the Arabic text from etc/HELLO ("السّلام عليكم")
concatenated with " Hello " with no newlines, point-max is 16001.  So
the length of the whole line is 16000 characters, and for the initial
string " Hello السّلام عليكم", M-= returns 20, so for 800 copies
point-max is what I would expect.  Do the Arabic characters in your font
occupy two columns each?  If the string I've tested is indeed that much
shorter than what you've test, it's even more mysterious that I see such
a slowdown and you don't...

>> >> There is no slowdown with a line of the same length consisting only of
>> >> RTL or only of LTR text, nor with the above test line when
>> >> bidi-display-reordering is set to nil in the buffer (but then, of
>> >> course, the Arabic is not displayed correctly).
>> >
>> > That's not what I see: setting bidi-display-reordering to nil doesn't
>> > affect the slow-down on my system in any perceptible way.
>> 
>> Very strange: here, after setting bidi-display-reordering to nil, M->
>> and M-< are practically instantaneous and even holding down C-n or C-p
>> traverses the entire buffer in a couple of seconds, with just a bit of
>> stuttering.
>
> Then I don't know what is going on.  Maybe you have a very weak CPU or
> something?  Or maybe your version of HarfBuzz is somehow much slower?

My CPU is Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz, certainly not
superfast but not superslow either, e.g. bootstrapping Emacs (without
native compilation) takes just a few minutes (with native compilation
it's a couple of minutes longer).  My HarfBuzz is version 8.3.0.

> Just to be sure: this is in "emacs -Q", and you see the same with many
> different fonts used for Arabic, not just with a few of them?

Yes, this is with -Q.  The only fonts I've tried are Noto Naskh Arabic,
DejaVu Sans, and Amiri.  I'm not sure whether any other fonts I have
installed support Arabic (I only have 294 fonts installed).

>> > If your real-life cases are all with Arabic text, then the reason is
>> > likely not bidirectional reordering of RTL text by itself (although it
>> > does slow down redisplay to some degree), but the fact that all of the
>> > Arabic text gets shaped by HarfBuzz via composition-function-table,
>> > which involves calls from the C code of the display engine into Lisp,
>> > which then turns and calls back into C.  This is relatively slow, but
>> > we have no choice: Arabic shaping requires that all Arabic characters
>> > be handed to the shaping engine for rendering, otherwise the display
>> > will either be illegible for Arabic speakers or at least look ugly in
>> > their eyes.
>> 
>> I created a test buffer with 800 copies of the Hebrew HELLO text
>> concatenated with " Hello " and found similar slowdowns with M-> and C-n
>> as with the Arabic text.  IIUC displaying the Hebrew script does not
>> involve text shaping, or does it?
>
> For the text in HELLO yes, it does: there are diacriticals (so-called
> "niqqud") there.  But not all of the Hebrew text goes through the
> shaper, only the combinations with diacriticals, so it should be
> somewhat faster.  And if you remove the diacriticals, Emacs will not
> use HarfBuzz at all for Hebrew.

Ok, I repeated the test with Hebrew without niqqud ("שלום") and indeed,
M-> and M-< are much quicker and even holding down C-n and C-p is
quicker (it does freeze, but unfreezes more quickly).  So it seems the
bottleneck really is the shaping.

>> > For now, I see no bugs here.  Maybe if you tell more we will arrive at
>> > something that is a real problem, but what I've read and saw until now
>> > is what I would expect in these cases.
>> 
>> I can try other tests if you can suggest any you think will help.
>
> I don't know what to suggest, because I see something very different,
> and have never seen what you describe.  The only idea I have is to run
> this under perf, but interpreting the results is not easy.

I've never used perf, and don't even have it installed (I build the
kernel myself but I haven't built perf, but I could do that -- though
probably not in the very short term -- if it might be helpful).

> Can someone else reproduce Stephen's results?  I'm quite surprised by
> what he describes, TBH.  That's not the performance of the Emacs
> bidirectional display I'm used to see.

I also hope others chime in.

Steve Berman