[Groff] XML and groff again

[Steve Izma]

Sorry, folks, but I'm three weeks behind reading the groff
mailing list, so I've only this morning come across the
discussion about XML and groff and it's of extreme importance to
me.

We started used troff and variants to produce books sometime
around 1988. From the beginning, we used a simple tag system
based on SGML for input and used shell and awk scripts to produce
the troff input on a pipeline to get our PostScript files. Around
1996 or 1997 we moved to groff (from SoftQuad troff) and around
1999 we started using SGML that was parseable by nsgmls, although
we used XML rules (no shortcuts) for the tagging. Then, about
three years ago we ran out of people (and management) willing to
take the time to learn the skills (including vim-style editing
and commandline practices) to do the work efficiently and now
most publications are produced by Quark (which I won't have
anything to do with). We still produce one large series (The
Collected Works of Florence Nightingale, in 15 volumes) using XML
and groff.

Around 1999 I developed some useful but crude python scripts to
convert RTF files to a level of XML tagging that allowed first
draft of typeset pages to be produced in about 7 or 8 hours
(usually the authors provide almost structureless word
processing documents). Once the files become parsable SGML or
XML, they can be run through a python script that converts tags
to groff requests and outputs PS. On a 2.4Ghz pc, this takes
place at about 40 pages per second. So if you're working on a
chapter, you can run gv in "watch" mode, have a memory key in vi
loaded with the command line, and updates are instantaneous.

During this period of time we produced hundreds of books and
journals with this method. I still use these methods for
producing books for Between The Lines, another publisher I'm
associated with.

The main problems:

- the python scripts are rickety and I've been working on
  cleaning them up
- XML input and structured text in general operate best with a
  very different philosophy of design for tmacs; I have written
  modular tmac files (one for page transition, one for
  footnotes/endnotes, one for lists, one for subheads, etc.) that
  are common or can be used for all projects, then a different
  main tmac file directly related to the particular DTD used that
  calls these "library" files; these are also at the point that
  they require some redesigning
- most authors have been spoiled by word processors and don't
  have a clue about structured text and don't want to learn

When I'm writing my own documents for this process, I use plain
text with "restructured text" rules, more or less, and run a
python or awk script that adds tags afterwards. I then clean up
the file with gvim, using a menu that allows me to add tags in
visual mode (all by keyboard shortcuts -- no mouse). I do the
same thing when I'm cleaning up html files. Here's an example for
HTML (XML has many more lines, but you'll get the idea from
this):

-----------------------------------------------------------------

" XML tagging menu

function! AddTag(tagname) range
        execute "normal `<my`>a</" . a:tagname . ">\<Esc>`yi<" . a:tagname . 
">\<Esc>"
endfunction

vmenu 80.50 Ta&gs.&Front.-sep1- <Nop>
vmenu 80.50.100 Ta&gs.&Front.&Title<tab>\<title\> :call AddTag("title")<CR>
vmenu 80.50.102 Ta&gs.&Front.&Head<tab>\<head\> :call AddTag("head")<CR>
vmenu 80.50.104 Ta&gs.&Front.&Body<tab>\<body\> :call AddTag("body")<CR>
vmenu 80.60 Ta&gs.&Subheads.-sep1-      <Nop>
vmenu 80.60.11 Ta&gs.&Subheads.H&1<tab>\<h1\> :call AddTag("h1")<CR>
vmenu 80.60.12 Ta&gs.&Subheads.H&2<tab>\<h2\> :call AddTag("h2")<CR>
vmenu 80.60.13 Ta&gs.&Subheads.H&3<tab>\<h3\> :call AddTag("h3")<CR>
vmenu 80.60.14 Ta&gs.&Subheads.H&4<tab>\<h4\> :call AddTag("h4")<CR>
vmenu 80.70 Ta&gs.&Tables.-sep1-        <Nop>
vmenu 80.70.1 Ta&gs.&Tables.&Table<tab>\<table\> :call AddTag("table")<CR>
vmenu 80.70.3 Ta&gs.&Tables.&Caption<tab>\<caption\> :call AddTag("caption")<CR>
vmenu 80.70.5 Ta&gs.&Tables.&Row<tab>\<tr\> :call AddTag("tr")<CR>
vmenu 80.70.7 Ta&gs.&Tables.&Data<tab>\<td\> :call AddTag("td")<CR>
vmenu 80.70.9 Ta&gs.&Tables.&Head<tab>\<th\> :call AddTag("th")<CR>
vmenu 80.80 Ta&gs.&Lists.-sep1- <Nop>
vmenu 80.80.1 Ta&gs.&Lists.&Ol<tab>Ordered\ list :call AddTag("ol")<CR>
vmenu 80.80.3 Ta&gs.&Lists.&Ul<tab>Unordered\ list :call AddTag("ul")<CR>
vmenu 80.80.5 Ta&gs.&Lists.&List\ Item<tab>\<li\> :call AddTag("li")<CR>
vmenu 80.90 Ta&gs.-sep1-        <Nop>
vmenu 80.100 Ta&gs.&B<tab>bold :call AddTag("b")<CR>
vmenu 80.100 Ta&gs.&P<tab>\<p\> :call AddTag("p")<CR>
vmenu 80.101 Ta&gs.&I<tab>italics :call AddTag("i")<CR>
vmenu 80.102 Ta&gs.B&q<tab>\<bq\> :call AddTag("blockquote")<CR>

" end of XML tagging menu
"

-----------------------------------------------------------------

You can get this menu in gvim by using :so and the menu filename.

In order to get it to work, you use vim's visual commands to
select text then Alt-g to pull down the menu, followed by the
keyboard shortcut for the tag you want. The highlighted text
becomes surrounded by that tag.

In other words, I think vim + XML + groff is the easiest way of
producing typeset text, at least, once you've got the macros set
up.

If others are interested in this, let's collaborate.

        -- Steve

-- 
Steve Izma
    Computing Systems Administrator       (519) 884-0710 ext. 6125
    Wilfrid Laurier University Press      FAX: (519) 725-1399
    Waterloo, Ont., Canada N2L 3C5        address@hidden