bug#69385: 30.0.50; Long lines with bidi text slow down Emacs

[Stephen Berman]

On Tue, 27 Feb 2024 10:52:23 +0200 Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Stephen Berman <stephen.berman@gmx.net>
>> Cc: 69385@debbugs.gnu.org
>> Date: Mon, 26 Feb 2024 22:47:21 +0100
>> 
>> I created the test buffer now by first typing "Hello" and the inserting
>> " السّلام عليكم " on the right, so the base paragraph direction was LTR.
>> Then I ran the benchmarks, and there is indeed a big difference, but one
>> that I think will surprise you.
>
> It doesn't, see below.
>
>> First, the timing for the LTR run with bidi-display-reordering set
>> to nil was:
>> 
>> (0.035104007 0 0.0)
>> 
>> This is, unsurprisingly, almost the same as the corresponding test run
>> with RTL base paragraph direction, which was (0.034058467 0 0.0).
>> 
>> In contrast, recall that with bidi-display-reordering at the default
>> value t, the RTL test run was (5.249231941 1 0.014300497000000023), but
>> now the LTR test run gave this:
>> 
>> (10.613699099 1 0.012965359999999981)
>
> Yes, that's the result of the fix I installed yesterday: it makes the
> display correct, but it costs more CPU, sometimes much more.

My recollection is that I ran the benchmarks before updating master with
your fix, but perhaps I misremember.  So I just now ran the benchmarks
for the default value of bidi-display-reordering again on emacs-29,
which does not have your fix.  Here is the result for the buffer with
RTL base paragraph direction:

(5.334990188 0 0.0)

And for the buffer with LTR base paragraph direction:

(10.537410334999999 1 0.011070665000000035)

And I ran them again on master now, to be sure I'm testing with your
fix; here are the respective results:

(5.350407252 1 0.013722977000000025)
(10.55348192 1 0.014691314999999983)

So practically the same difference without and with your fix.

> To see why this is so costly, disable auto-composition-mode in the
> buffer, and compare the times with bidi-display-reordering nil and
> non-nil.  With auto-composition-mode disabled, you see the effect of
> the reordering alone (because characters are still reordered for
> display, just not shaped correctly, as C-f will show you), and it
> should be quite small; at least, that's what I see here.  The

This I can confirm (on master with your fix); in the RTL buffer with
bidi-display-reordering first t and then nil:

(0.049796204999999996 0 0.0)
(0.033475448 0 0.0)

and likewise in the LRT buffer:

(0.049654598 0 0.0)
(0.033401513 0 0.0)

> (mandatory) shaping of Arabic text is what takes most of the redisplay
> time in these cases.  In particular, M-> does a recenter to show EOB
> nicely, and that is very expensive in this case; disable that
> recentering by using "(goto-char (point-max))" instead to see the
> difference.

I guess by "does a recenter to show EOB nicely" you mean the call of
(recenter -3) at the end of the code in end-of-buffer, because after
calling (goto-char (point-max)) the last line is vertically centered in
the window?

>> The buffer size here is considerably larger than the buffers for my test
>> runs; I couldn't remember it exactly, so I typed `M-: (point)' in the
>> buffer, with the result 43901.  It took a number of seconds for the
>> result to be displayed, during which Emacs was unresponsive, so I then
>> ran (benchmark-run nil (point)) in that buffer, with this result:
>> 
>> (9.030000000000001e-07 0 0.0)
>> 
>> I don't know what this result means, but the reported execution time
>> bears no relation to actual elapsed time until the value of point was
>> displayed.  So I reran the benchmark and also manually timed it with a
>> stopwatch.  This is the result of the second benchmark:
>> 
>> (3.1120000000000004e-06 0 0.0)
>> 
>> However, the stopwatch showed fully 15 seconds.  Maybe benchmark-run
>> doesn't work for point for some reason.
>
> A Lisp program that just calls '(point)' doesn't include the redisplay
> caused by "M-:", which enters the minibuffer, and therefore triggers a
> fairly thorough redisplay of the window, and by display of the result.
> That's Emacs's MVC design in action for you.

Ah, ok; thanks for the explanation.

>> This sounds plausible, but my results seem to indicate there something
>> else (or more) going on.
>
> Yes: that something is character composition, which in the case of
> Arabic text is quite expensive, because every chunk of Arabic text
> between SPACEs goes through the shaping engine.
>
> (This explains well why Emacs tries so hard to call the shaper only
> when absolutely necessary, contrary to HarfBuzz folks' opinions that
> we should pass all the text through the shaping engine, to get correct
> display with ligatures, kerning, and other niceties.  They are right,
> of course, but with the way character composition was designed in
> Emacs, doing so would be prohibitive from the performance POV.)
>
>> > I will see if we can do better in this matter.
>> 
>> I appreciate whatever you can do.
>
>>From what I've seen so far, it won't be easy, because it requires
> changes to some key logic in the display code.  Hmm...

I'm afraid I can't help with the display code logic, but I'm happy to
test any ideas you have.

Steve Berman