graphical manuals

[James K. Lowden]

I have a public groff demonstration project.  Its purpose is to show
the world what groff + mandoc can produce in terms of technical
documentation, and to show ourselves what it cannot do.  I'd like to
ask first for suggestions on how to bring it up to the state of the
art.  Then see what we can do to improve on that.  

http://www.schemamania.org/groff/gcobol-demo/
https://gitlab.cobolworx.com/COBOLworx/gcc-cobol/-/tree/master+cobol/gcc/cobol/man

As I'm sure no one on this is aware, I'm part of a project to add COBOL
to gcc.  Like any self-respecting compiler, ours needs a reference
manual.  And, like any self-respecting troff fan, I want to use groff
and, because HTML, mandoc.

But I really can't.  Not yet, not for what I want and what the project
demands. Mostly because we want "railroad diagrams", but tables, too.  

Perhaps IBM's best known and most verbose programming languages are SQL
and COBOL. Our COBOL parser at present recognizes 1593 grammar rules
using 649 language tokens. These appear mostly in 41 statements,
verbs such as DISPLAY and READ.  Each statement can be quite long,
consisting of a dozen or more words. 

The files linked above illustrate a relatively simple verb, START.  In
the PDF, you can see my version.  For a non-railroad comparison, see
the texinfo output (which I also contributed to) at
https://gnucobol.sourceforge.io/HTML/gnucobpg.html#START.

For another point of comparison, consider the SQL DELETE statement: 

        https://sqlite.org/lang_delete.html
vs      https://www.postgresql.org/docs/current/sql-delete.html

I suspect railroad diagrams might be sniffed at by many (like me, once)
who like how their manpages look.  Don't we have typographical
conventions to distinguish keywords and literals and arguments, and
optionality?  We do.  But I posit that those conventions break down in
the large: as the number of terms grow, it becomes more difficult to
follow, and the amount visual noise, excuse me, "notation convention"
grows.

For proof, I offer that railroad diagrams are popular with users.
SQLite and IBM both use them.  I would venture that the less one knows
about BNF, the more attractive they are.  

I suggest they're not more widely used because tools are lacking, which
in turn is because there is a widespread mistaken belief documentation
should be automatically generated and, when that's insufficient, Use
the Source, Luke.  And because folks who contribute to open source
projects tend to have BNF in their quiver.  

On the pattern of dformat, I'd like eventually to design a macro
language to generate these diagrams.  That processor's inputs could be
generated from yacc outputs, plus some ancillary placement and style
information.  As a baseline, I'm using pic.  That at least shows what
can be done.  Which, I hope you agree, is not enough.  

There are two classes of problems with output currently: those that can
be remedied with better use of existing tools, and those that cannot.

Things that could be done better:

1.  PDF TOC, especially  using Deri's new work.
2.  PDF hyperlinks within pic, so that common terms can be cross
referenced, and so the user can jump to the term's definition.
3.  Extract the diagram as a graphic artifact, and reference it in the
HTML.  Use some kind of conditional logic to tell groff to link to the
artifact if it's rendering HTML.
4.  Make the boxes in the #3 artifact "clickable", so they include a
URL.
5.  Wrap the graphic in some kind of CSS to make them expand/collapse.
Not a PDF option at this time, like the VAX atomic clock.  

I don't know the best way to do those things.  If you'd like to suggest
command-line options, that's why I'm asking.  I haven't tried Deri's
branch or the convenience macros he described recently.  Until I read
about them, I had planned to use pdfmark directly.

Things that can't be done:

1.  Render diagrams as SVG.  I think that would be optimal.

2.  Useful results from

    $ groff -T utf8  -mandoc -pet ./gcobol-start.3c

Actually, I'm not sure there's any better way inside a character-cell
terminal to render the diagram than by reverting to standard manpage
notation, with braces and brackets and boldface.  To do that would
require something like my nonexistent macro language, because pic lacks
the necessary semantic information. Please tell me if you have a better
idea.

3.  In the PDF, and HTML, to render a "right mouse click" menu on
linked items.  Just clicking on the link takes to you the target.
Right-clicking presents a menu of cross-referenced items, other
references to the same term.  That way, for, say, a "FILE", you could
jump to the rules for how it is used in this context, or to its global
definition, or to how it's used in other contexts, say by "WRITE".

4.  As far as I know, there is no tool for producing a TOC or an index
for a set of HTML pages.

I haven't mentioned tables in HTML only because I haven't gotten that
far yet. Last I checked, grohtml gave terrible tables from tbl. Again,
if there's good news on that front, I'd be happy to hear.

Thank you for your kind consideration.  I would be glad to try new
constructs and demonstrate outputs from other commands using different
options.  I would like to get first to a place where the HTML is
comparable to the PDF, and as good as the SQLite diagrams.

--jkl