Re: [Groff] grops and Unicode, Unicode in general.

[Alejandro López-Valencia]

On 5/21/05, Werner LEMBERG <address@hidden> wrote:
> 
> > Wartan guesses correctly that the best way would be to start from
> > the original URW++ fonts and add the Cyrillic glyphs to them, but
> > with some caveats:
> 
> Do you mean starting again?  Uh, oh, this is a *large* project, and
> normally it will stop unfinished...  Cf. the `freefont' project which
> has no contributions since two years.  I don't say that the Cyrillic
> extension to URW are bad in general, I say that they aren't stable
> yet.

No. Not starting again, but reusing the existing work *properly*· That
is, don't touch the original outlines, but rather *add* to them the
new outlines and hints to the postscript container *by hand* (or using
tools such as those developed by the Polish TeX Users Group,
BachoTeX?). A Type 1 font is just a large PostScript program and
should be treated as such. This doesn't mean to draw the new out lines
by hand, use whatever tool you want, but when doing the addition do it
manually.

And yes, it is lots of work. And you need to be a lot more familiar
with PostScript than most of us are to do it properly (I wouldn't dare
to do more than the occasional small hack for my own private purposes,
though I did it for certain Classic MacOS fonts somewhere in CTAN when
I was younger and more self-absorbed...).

> > 1. The addition *must* be done by hand to guarantee that the
> > outline, hint and subroutine programs are not damaged by a graphical
> > font editor (believe me, *all* graphical editing be it with a high
> > end tool (Fontlab, Asia Font Studio, Ikarus or DTL FontMaster), or
> > with a half-baked tool such as Fontforge *will* destroy the original
> > digital font program) and each iteration will have lower quality.
> 
> I don't believe you. :-) First, PS fonts don't have outline, hint, or
> subroutine programs.  You are apparently thinking of TrueType fonts.
> Second, PS hints can be autogenerated by tools like FontForge, but I'm
> sure that *all* editors allow preservation and manual alteration of
> hints.

See my answer above :-)  And but of course Type 1 fonts have outline,
hint and subroutine programs! Most are embedded in the outline
description themeselves but there are hint substitution routines,
outline segment subroutines, flex hints, subrs, composites and
whatnot. You can do amazing things if coding type 1 fonts by hand. For
example Erik van Blokland and Just van Rossum have  some amazing
fonts: http://letterror.com/content/nypels/randomfont.html

> > 2. The fonts must have different names.
> 
> I disagree.  Fonts must have updated version numbers, together with
> proper ChangeLogs.  This is one of the fields where the people working
> on `improving' the URW fonts have done big mistakes in the past.

I agre, using version numbers would be enough, but still the problem
with lower quality remains. That's why I think that a fork with new
names would have been wiser from the start.

> > 3. You don't need to have all glyphs in single font container!!!
> 
> Hmmm.  There are a big number of glyphs which are shared by, say, the
> Latin and Cyrillic script: numbers, accents, punctuation marks, etc.
> Those glyphs must be available in both a Cyrillic and a Latin font --
> while creating virtual fontsets is easy, virtual kerning for them is
> not, nor is it supported in X, AFAIK.  With other words, having a
> single glyph container for Latin and Cyrillic makes sense.
> 
> > X is very happy to use virtual fontsets, so you can place your
> > Cyrillic glyphs in physically different containers that appear as
> > one to the graphics display engine; in fact, that's how CID fonts
> > work, sort of.
> 
> Indeed, sort of!  CIDs have been invented to do exactly the opposite
> of what you are suggesting, namely to have big glyph containers
> (mainly for CJK fonts).  For non-CJK fonts, non-CID-keyed CFFs are
> *much* better since you have proper glyph names.

Yes, but CID fonts are really metacontainers. Inside glyph programs
are stll organized in 8-bit chunks. So, I think my point is still
valid.

> For the readers who are not familiar with the many acronyms: CFF
> (Compact File Format) is the new default font format from Adobe.  The
> glyphs itself are in Type1 format (but with a compressed
> representation), either with glyph names or with CIDs.
> 
> A CID (Character Identifier) is a number, representing a glyph from a
> repository.  It is mainly used for CJK fonts where glyph names are not
> meaningful.  Adobe has defined various repositories -- which are
> extended if needed -- to cover a script with most typographical
> features.  Since PS fonts are organized as a collection of various
> dictionaries, the term `CID-keyed' is used: The key is the CID, and
> the value is the glyph data.
> 
> Here some citations from the Adobe Technical note 5094 (in file
> 5094.CJK_CID.pdf).  This is a bit lengthy, but it shows how such glyph
> repositories evolve and how large they are.

[snip]

> > And you can recode them on-the-fly so that the glyphs are delivered
> > as Unicode characters to the client applications.
> 
> Neither kerning nor other higher-level font features like OpenType
> support will work.  Virtual fontsets are only useful to combine
> scripts which don't overlap like, say, Thai and Latin.  But Latin and
> Cyrillic (and Greek) *do* overlap!
> 

But only insofar as you have duplicate glyphs in different containers.
If you want to add cyrillic support to the original fonts, you can
create font containers that only have the cyrillic glyphs properly
named using the AGL (as you pointed out below).

> > Therefore you could use the original URW++ fonts untouched and add
> > the Cyrillic glyphs from different physical sources.
> 
> Now we are coming back from discussing the pro and cons of virtual
> fontsets and glyph containers to the reality how to use Cyrillic fonts
> in groff.  Except the URW extensions for Cyrillic I'm not aware of
> other freely available fonts which match the appearance of the
> standard PS fonts.  Do you know something else?

Unfortunately not. I'm not aware of anything of that level of quality
for the base LW35; so we need to live with what we have. In fact, I
don't think that the Cyrillic additions to the URW fonts are bad, it
is the implementation details that bother me, a lot :-)

On the groff side, I think we shouldn't have problems creating proper
virtual fonts for postscript output if the glyph containers were
separate as outlines above. I agree that kerning is a problem when
trying to meld separate fonts, but it could be solved by using
modifying the existing metrics generatiion tools.

>     Werner
> 

Alejo

-- 
Alejandro López-Valencia