Re: Ligatures (was: Unify the Platforms: Cairo+FreeType+Harfbuzz Everywhere (except TTY))

[Pip Cet]

On Sat, May 23, 2020 at 4:34 PM Eli Zaretskii <address@hidden> wrote:
> > From: Pip Cet <address@hidden>
> > Date: Sat, 23 May 2020 15:13:38 +0000
> > Cc: address@hidden, address@hidden, address@hidden
> > Because what the current layout code does by default is to break
> > along any glyph boundary, and I don't see how that's broken in any
> > way.
>
> The code assumes that breaking on some glyph leaves the buffer
> iterator ('struct it') in a state that we can simply continue to the
> next buffer position.

Yes. I see no reason to change that.

> But if you already picked up several characters
> via look-ahead, that is not true, and you will have to return back
> several character positions, in order to continue on the next screen
> line.

You're describing why look-ahead is difficult: a while ago, you
appeared to be saying it wasn't. This confuses me.

Obviously, when I say "look-ahead", I mean receiving the next display
elements an iterator would produce if it were actually advanced,
without advancing it.

An easy, but potentially slow, way of doing that is to copy the
iterator to a new one, advance that, retrieve the display elements,
then throw away the copied iterator and return.

> The whole convoluted logic of display_line (and a similar one
> in move_it_in_display_line_to) is based on the assumption that this
> line-wrap decisions are made as soon as a single glyph is produced;
> that code will need to be rewritten if this assumption breaks.

I see no reason to break that assumption.

> And
> since the code is already hairy, to say the least, I cannot even
> imagine what it will look like after such rewriting.

Good thing I'm not planning to do that, then.

> This is just a small example of how deep are the current design
> assumptions entrenched in the code.

One I don't understand, because those fundamental design assumptions
aren't something I'm willing to break at this point.

> > > In short, I really don't see how this could ever work, except in a
> > > very limited set of simple use cases.  E.g., what do you do with
> > > bidirectional text? ignore it?
> >
> > A bidi boundary is a hard boundary for HarfBuzz, and no shaping
> > happens across it. Is that what you mean by "ignore it"?
>
> I don't mean the boundary, I meant the fact that clusters need to be
> reordered.

I see no fundamental problem there, certainly not of the "I don't see
how this could ever work" variety.

> > > I don't see how it could work in production while supporting all the
> > > use cases we already do.
> >
> > It only comes in for use cases not handled otherwise, i.e. those where
> > the iterator is at an IT_CHARACTER. All other use cases are
> > unaffected, because they mean we're overriding the font decision
> > anyway.
>
> I see no reason to add such patches just to handle some simple enough
> use cases.

If it's so simple to get ligatures and kerning right, please tell me
how to do it.

> If we want the shaper to handle all the text we display,

Do we? A while back you said Lisp control over compositions was an
important feature, and I'm inclined to think we shouldn't break the
existing composition code.

> we should go all the way and do it for any text, ASCII, non-ASCII,
> symbols, emoji, everything.

Are you suggesting I'm somehow limiting myself to ASCII? Let me assure
you that's not the case.

> The current codebase is already very
> difficult to understand and modify;

I agree with that.

> you seem to suggest to make it
> even more so,

Well, yes, it's not going to be a free feature. The changes are
comparatively tiny compared to what else has been done to xdisp.c.

> and on top of that solve only a small part of the
> underlying problem.

Ligatures and kerning (right now, for LTR text). Is that a small
problem because of the lack of RTL support?