Re: Unicode support

[Werner LEMBERG]

>  WL> What you are talking about is Unicode level 1.  But to properly
>  WL> support Indic languages more things have to be done.  Even simple
>  WL> combining characters will have to be represented differently,
>  WL> AFAIK.
> 
> For info, what needs to be done essentially differently for these
> languages compared with existing support in Emacs?
> 
> For what it's worth, I did try a trivial proof of concept for
> Unicode support (as I understand it) of Thai, based on thai-util.el
> and post-read conversion.  As far as I could tell, it treated the
> level 2 example in Kühn's demo text correctly, but Mule-UCS is the
> only thing against which I can compare.

Well, as the support for Devenagari shows, everything is already
possible in Emacs.  The question is rather how to do that in a
systematic way, avoiding ad-hoc solutions for a specific language.
Thai support is special since no new additional glyphs not already
available in the TIS encoding are necessary.

I think a text layout engine is necessary between the Emacs Lisp layer
and the glyph engine to make Emacs work with characters, not glyphs.
I don't know which language would be suited best.  Probably as with
the rest of Emacs, some core C functions should be accompanied with
Lisp code (probably mostly in CCL).

The following tasks should be performed:

  . For Arabic scripts, adding initial, medial, final, and isolate
    properties to the characters.

  . For Indic scripts, reordering of glyphs according to the Unicode
    standard.

  . For Right-To-Left scripts, reeordering from logical to visual
    order, taking care of nested levels etc. as described in the
    Unicode standard.

  . For everything else, applying data from the various Unicode tables
    to make the glyph engine aware of non-spacing accents, combining
    characters, etc.

Note that these items are completely independent from the used
fonts.

BTW, I suggest to extend the glyph engine with OpenType features,
probably using the FreeType library: glyph substitution and glyph
positioning (the GSUB and GPOS tables in OpenType fonts).
Alternatively, the ICU library (from IBM) can be used -- it now comes
with an X11 license so that it can be used with Emacs.  Additionally,
a Lisp solution for glyph substition is necessary to make the already
available BDF fonts work; it should be straightforward to emulate GSUB
features with code already available in the devanagari module.

This means the following modules:


                   Emacs Lisp
                       |
                       | characters
                       |
               Text Layout Engine
                       |
                       | characters
                       |
                GPOS/GSUB Engine
                       |
                       | glyphs
                       |
                 Display Engine


   Werner