[Groff] Back to the future

[Peter Schaffter]

I thought I'd leave Future Redux alone for a while to see where
the cards fell.

List members, apparently, suffer somewhat from ADD because--no real
surprise--it forked into a discussion about manpages.  I've no
quibble with that, but if we're to figure out what to do about groff
in the future, we're going to have to stick to the big picture.
Discussions about presentational vs semantic, manpages vs info,
terminal vs browser, etc are muddying the waters.  If I may mix
metaphors, it's preventing us from seeing the forest for the trees.

I propose applying not so much Occam's Razor as Occam's Hatchet for
the time being.  As Eric realised in his bacon-and-eggs epiphany, a
lot of this stuff is red herring, and it's causing us not to think
clearly.

That said, and only because it makes my point, I'd like to report my
own bacon-and-eggs (cornbread-and-cappuccino, actually) epiphany:
we've been looking at the whole xml/semantic issue backwards.  Groff
is a rendering engine that produces typeset copy, the same way that
a browser is a rendering engine that produces screen copy.  It
isn't groff's place to produce presentationally-neutral output, but
rather to receive presentationally-neutral output and interpret
it for typesetting.  The logical flow isn't groff => XSL-FO, it's
XSL-FO => groff.

In the simplest of terms, groff is a typesetting backend.

  Deri James:
    "I use groff as a typesetting engine called from a front end...all
     I want is a fast engine which handles the typography."

  Tethys:
    "I too use groff as a typesetting back end."
   
  Chad Roseburg:
    "This is the primary use case for us as well [ typesetting backend]"

  Ingo Schwarze:
    "...groff is really the tool for producing quality typeset output
     even of manuals."

We have a bit of a terminology problem in that typesetting, which
used to mean *setting* type, increasing means *rendering* type.  Thus,
to speak of groff as a typesetting engine is a tad confusing,
since, more accurately, it renders type, primarily for two modes of
presentation: terminal and PostScript/PDF.

It does a good job, too.  And it's been doing it for decades
"with with considerable grace under fire." [cstr54, 2nd ed.]
I see no reason why it shouldn't continue to do so in the future.
It's proven itself flexible enough to accommodate a whole whack of
changes without ever losing touch with its origins.  It's a mighty
accomplishment, and one not ready to be declared moribund.

Corollary to acknowledging that groff's primary role is as a
typesetting backend is keeping debates about manpages, semantically
useful macros, and well-formed input files distinct from discussions
about code development.  What goes on in macroland should stay
in macroland.  Whether, or how, well-formed files and macrosets
should/could be xml/stylesheet-parsable in a semantic world has no
bearing on groff's future.  It's a front-end debate when back-end
development is what's needed.

I use the term front-end for macros because that's what they,
whether user-written or from a distributed set.  It's how we all see
them; it's how we all use them.  When I started work on mom, that
was my goal: to create a macro set that formed a complete front end
to groff, analogous to LaTeX.

I field enquiries about mom off list, and I notice that users
tend to view mom as an application in and of itself, with groff
merely taking care of the final typesetting.  I posit that since
groff is perceived and used this way, its future rests on how
well it performs as a typesetting back end.  All other concerns
are secondary, including groff's role as a Unix documentation
formatter.  The manpages debate is largely about groff usage, not
groff development.  (I much to say about manpages, but shall forgo
it here in order to follow my own advice.)

A frequent comment I see is: "And here I thought groff was just
for formatting manpages."  To be honest, way back when, I might
have thought the same thing were it not for happening upon the
cstr54 when I was researching typesetting with GNU/Linux.

Groff has a terrible reputation outside the cognoscenti: archaic, of
limited use, poor typesetting compared to TeX, legendarily difficult
to master.  None of it is true--except, of course, the last bit.  (A
joke-list that circulated for a while entitled _The Ultimate Geek_
had as one of its items, "Writes own groff macros.")

The "old-school, for geeks only, manpage formatter" aura surrounding
groff is unwarranted, especially after Keith and Deri made
full-featured, direct-to-PDF a reality.  Yet it persists.  As
recently as this past fall, Robin Haberkorn, a contributor to
mom, made the comment, "How many prose writers are geeky enough to
use groff?"  (Hmm...let's see...there's Alejandro, and I, and...?)

For preservation of content, ease of modification, and efficiency in
typesetting, the batch processing model is superior to all others.
One shouldn't have to be a geek to benefit from the advantages,
which leads me to think that, ancillary to backend improvement,
there's a pressing need for groff advocacy.  Put another way, folks
need to know that using groff to format a novel isn't novel.  It's
pretty routine, and you shouldn't have to be a hacker to do it.
Mind you, that entails robust and easily-mastered front-ends--but
where are these front-ends to come from if we don't attract new
users who have the knowledge, drive, and skill to write them?  And why
would this new crop of groffers come to groff at all when there's
TeX, with its stellar reputation for typographic excellence?

In the Redux post, I asked the question: Do we need groff at all
when there's TeX?  That no one took the bait means either that no
one can answer the question, or that the answer is self-evident.
For me, it's the latter.  Groff is leaner, meaner, and faster.  Its
programming syntax is beyond simple.  Its request set is extensive
and supplies users and macro programmers with an addictively good
typographical Meccano set.  With groff, if I can dream it, I can
build it.  As Clarke Echols says: "groff has incredible flexibility
for the imaginative, even far away from typography."

If groff's formatting engine, the backend, were to be brought
up to snuff with TeX's, then the answer to my question would be
even easier: we need both groff and TeX because diversity fosters
evolution.  For example, Steffen Nurpmes observed:

  "...when i look at TeX output once in a while, i indeed feel
   uncomfortable, especially when i look from a bit of a distance: This
   appearance of «painted uniformity» all over the paper..."

I'm glad someone else has noticed this.  It's like the Gutenberg
Bible: the grey is so uniform I'm more inclined to admire the
typography than read the words.  Which indicates that TeX is not yet
the _ne plus ultra_ of Unix typography since it does not evince a
satisfactory page rhythm to go along with its excellent _gris_.

(I do wish "page rhythm" weren't to typography what "swing" is to
music: impossible to quantify, undeniable when it's present.)

I'm not suggesting this should be groff's goal, rather that
there's room for improvement in both TeX and groff when it comes
to typesetting.  Groff's backend could benefit enormously by
adopting/adapting elements of the TeX approach, but who's to say
that TeX could not benefit equally from developments within groff?
It's a cross-pollination that won't happen if groff stays stuck
where it is.

Why do I care?  Well, with the greatest of respect to Eric, I
feel that "the page", as it has been recognized for centuries,
is far, far from being irrelevant to the future of material
intended to be read at the screen.  The rectangular, portrait
orientation of the majority of printed material since Gutenberg
isn't an accident.  The host of psychological and cognitive factors
responsible for its overwhelming adoption won't go away simply
because browsers hugely favour landscape orientation with natively
long line-lengths and an absence of fixed, proportionally-balanced
whitespace framing the text.  Eric's reservations about PDF and
linking aside, given a choice between "Read HTML" and "Download
PDF", I will always select PDF because the content is more easily
grasped when the typography adheres to the printed-page paradigm.
If we abandon the paradigm, if we dispense with it because black
squiggles on dead trees just ain't the way of the future, we become
victims--short-sighted victims--of fashion and expedience.

I realize all this blather is useless if groff doesn't have a
technical lead, or leads.  Had I Eric's "wizardly coding chops", I'd
begin exploring ways to implement what I've discussed here myself,
very likely by studying Gunnar Ritter's work (Heirloom Troff) and
folding it into groff.  I do not, however, have said chops, not by a
country mile.  (I can't even get ssh and git to cooperate so I can
clone the repo, despite 12 years of error-free ssh/cvs-ing to and
from Savannah.)  Which is by way of saying that the most pressing
need for groff right now is a technical lead, and whatever we can do
to attract one--preferably one with Werner's sensitivity to the big
picture--is our first priority.

To that end, I propose, as someone else did (sorry, can't seem to
find the post quickly), that we draft a sort of mission statement or
set of guidelines.  Chief on my list would be:

1.  Backward compatibility remains a top-level priority; not
    exactly sacrosanct, but very close to it.  Mike still needs
    those 70s documents. :)

2.  Primary development geared toward overhauling and updating the
    backend, both its approach to formatting and the requests that
    drive it.

3.  No changes to the syntax of existing requests.

4.  Where existing requests would benefit from being made more sane
    for the purposes of macro writing *in the future*, either add a
    compatibility flag or create new requests in a vein similar to
    de1, am1, etc.

5.  Implement new requests freely (as, for example, when Werner
    added .fzoom) provided they do not break backward compatibility.

If others on the list are prepared to make--and debate--"big
picture" suggestions for a statement of this sort, I will see to
the compiling and writing of it.  I stress "big picture" because,
while we all have our pet peeves and favourite soapboxes, we need to
figure out where we want groff to go before we can address them.

As before, apologies for the long post.  In future, if members would
prefer, I can post to my blog and send links to the list instead.

-- 
Peter Schaffter
http://www.schaffter.ca