[Groff] mdoc problem

[Werner LEMBERG]

Ruslan,


I've CCed the groff list since this a quite generic problem not only
related to mdoc, and I'm not sure how to solve it gracefully.  Any
hints or ideas are highly welcome.

Consider the following mdoc code:

  .Oo
  text longer than a single output line
  .Oc

`.Oo' produces an opening bracket with no space afterwards.  `.Oc'
should do the opposite, i.e., inserting a closing white bracket
without a space before.  The current solution in mdoc is to start a
diversion with .Oo which is closed with .Oc; the diversion is then
copied to a string:

  .ds foo \*[bar]\c

The final `\c' ensures that the break inserted by .Oc to finish the
diversion doesn't produce a space.  As you can easily imagine, this
fails for more than a single output line because a troff string can't
contain newlines.

While preparing mdocNG (a new and improved mdoc version which you can
find in the *.new files in the tmac subdirectory of the current
snapshot), I've temporarily removed the use of diversions which made
the code very complicated and error-prone.  Now I'm investigating
alternative solutions.

First of all, diversions always include the started partial line:

  abc
  .di foo
  def
  .br
  .di
  ghi
  .br

The diversion `foo' now contains `abc def' which is not very useful
for the mdoc problem (I doubt that this behaviour is useful in any
situation).
  
My idea was to introduce a new request `.box' which doesn't have this
property, i.e., the above code will produce a diversion which contains
`def' instead.  Together with the `.chop' request (which removes the
effects of `.br' in the diversion) I thought that I've found a working
solution.

Unfortunately, after some testing and peeking around in the code I
found out that a `.box' command alone isn't sufficient.

Reason:

The idea of a diversion is to save *formatted* text.  Usually, this
works fine:

  .di xxx
  .ev foo
  ... prepare font size, line length, etc. ...
  .fi
  text text text
  .br
  .ev
  .di

To insert such a diversion, do this:

  .nf
  .xxx

The `.nf' command assures that troff doesn't try to reformat
(i.e. inserting more whitespace) the already formatted diversion.

GNU troff strictly separates between input and output characters.  A
formatted space is no longer a space character but a space token of a
given length.  For example,

  .di xxx
  .ev 1
  .ll 1i
  abc def ghi
  .br
  .ev
  .di
  .
  x
  .xxx
  y

produces

  x abc    def ghi y

and there is no way to undo the gap between `abc' and `def'.

The ideal solution which completely works at input level is this:

  .de Oo
  [\c
  .de box Oc
  ..
  .
  .de Oc
  .chop box
  \\*[box]]
  ..

The `.Oo' defines the `box' macro which body is closed as soon as
`.Oc' is encountered.  Then, .Oc is immediately executed.  Maybe
you've ever wondered why troff provides this strange alternative
syntax of `.de' -- I'll add this example to the groff reference
eventually...

Alas, it can't be used within mdoc due to the syntax of the mdoc
macros (please consult groff_mdoc.samples.man for more details);
additionally, it can't be nested.

Now enters the `.asciify' request.  This command can `undo' (to a
certain extent) the formatting of a diversion.  For example, glyph
nodes are converted back to input characters, ligatures are split into
the its two input characters, etc.  However, for some reasons not
known to me, horizontal space nodes (except `\ ') are not converted
back.

The question is whether I shall extend/modify the .asciify request to
handle that.  Opinions?  Has someone ever used .asciify?  The example
given in the manual page must be based on some real example IMHO...

Let's assume for the moment that .asciify can convert horizontal space
nodes of all kinds back to a normalized input form.  Then, the
following two (simplified) versions of the .Oo and .Oc macros would
work with multiple lines:

  .de Oo
  [\c              \" insert `[' and prevent space
  .box xxx         \" start box
  ..
  .
  .de Oc
  .br              \" flush partial line to box
  .box             \" end box
  .chop xxx        \" remove final newline (from previous `.br')
  .asciify xxx     \" undo formatting
  \\*[xxx]]        \" insert box and closing `]'
  ..


    Werner