Re: [Groff] The future redux

[Pierre-Jean]

Hello Peter,
Hello groffers,

Peter Schaffter <address@hidden> wrote:

> For example, groff's line-at-a-time approach to
> formatting, if unchanged, will remain an impediment to high quality
> typesetting and ensure groff's demise for anything other than
> writing manpages.

It already is an impediment to high quality typesetting.
Dealing with what we call in french the "gris typographique"
is the basis of typography. Paragraph at once adjustment
was done manually long before computers exists. The same is
for arbitrary ligatures, real small capitals fonts, etc.

These of us which are not convinced that they need these
features should read _Der Detail in der Typografie_, by Jost
Hochuli.

> Since the point of implementing page-at-once
> formatting (or, as Werner dreamed, document-at-once) would be
> to improve the quality of typeset output, not to change the
> fundamentals of groff usage, resisting such a change seems like
> misplaced Luddism.


For sure !
Let's look how Heirloom Troff handle that:

- It has the -x option, which set the extension level of
  troff, from 0 (historical behaviour), to 3 (use all
  extensions). The .xflag macro give the ability to define
  that inside document or macro.
- It has the .padj macro which enable or disable the
  paragraph at once adjustment.

With these macros, it's easy to switch between the
historical and modern behaviour of troff, and to format old
documents and moderns ones. Backward compatibility is
strictly preserved (and is the default behaviour of heirloom
troff).

But, to effectively use the paragraph at once adjustment, one
need to patch the macro sets (or create some new ones).
Because of the architecture of troff, things are done this
way:
- when troff format the paragraph at once, it also
  deal with diversion (and footnotes) inside it. 
- then it output the formated paragraph, and deal with
  break page and traps.
- The problem is that the diversion containing all the
  footnotes of the paragraph has been populated when troff
  encounters the traps and break page. So, footnotes might
  not be on the correct page.

To handle these kind of situation, Heirloom Troff provide
the \P[macro] escape sequence, which will execute 'macro'
when the line is printed. So, to deal correctly with
footnotes, the note macro *must* divert the content of each
note inside a macro 'noteX', add the \P[noteX] to the
content of the paragraph, and let the noteX macro do the
usual job of a note macro.

I believe that Groff will face this kind of problems.


> * Backward compatibility

The question about compatibility, in my opinion, does not
concern backward compatibility, but the horizontal one: To
do a correct typographic job, Heirloom troff define a lot of
new macros:

        .fzoom (zoom font by some factor)
        .fp (mount a font)
        .fps (mount a font with a special character map)
        .feature (control opentype feature)
        .fallback (fallback font in case a glyph is not found)
        .hidechar (hide some characters from font)
        .fspacewidth (define space width for a font)
        .minss (minimum word space)
        .letadj (micro-typography adjustment of letter size)
        .sentchar (sentence ending character)
        .transchar (transparent character for sentence ending)
        .track (letter space tracking)
        .kern (pairwise kerning)
        .fkern (control the use of kerning table of font)
        .kernpair (define a kerning pair for glyphs from different fonts)
        .kernbefore (add space before character)
        .lhang (hanging characters in left margin)
        .rhang (hanging characters in right margin)
        .flig (define ligature)
        .fdeferlig (remove ligature)
        .pshape (define the shape of a paragraph)
        ... And I probably forget some.


I can't believe that groffers simply ignore this. It could
be, at least, a good starting point to decide what's needed
for groff. But I hope that the groff community will also try
to follow the same convention when possible.



> keep presentation and formatting separate

With this idea in mind, one could create different macros
for different mediums, in the manner that web graphists
actually create different css files for differents screen
resolutions. Formatting the document for different mediums
should be as simply as changing the macro name in the groff
command line.

In fact, that was my project for the Utmac macros: a
strictly semantic troff syntax, used by different macros
to produce differents layouts. I've also created special
macro for Nroff to produce plain text files formatted as
READMEs, markdown files and... xml files.

> In short, groff has always had the "architecture" to support source
> files that are both human- and xml-friendly, it's just that it's
> rarely been taken advantage of.

I'm agree with that.

Nroff itself is able to produce nice xml documents. One
simply have to write a nroff macro to do that, and let nroff
itself deal with all the macro created by the end user and
inserted in the final document.

What's difficult, thought, is to deal with newlines. Nroff
could probably do this alone, but it's hard to implement.
Much more simpler is to let him insert some tags that inform
a post-processor to delete the not needed newlines. This
way, the command line to convert a troff document to xml is:

nroff -mux f.tr | postxml > f.xml

It's so easy !

> * Summing up

In one hand, I'm often feeling that our recents discussions
are pointless since Groff does not have any developper to
implement what we're discussing about.

But, in the other hand, an overview of the state of the art
in the troff land  (and probably also latex), a precise
roadmap, and an organized community is probably needed to
find a developper.

Maybe that we could continue this discussion in the aim of
producing a public document that would encourage people to
come and work on groff.

> Again, sorry for the long post.

So do I.

Cheers,

Pierre-Jean.