Proposal for handling .hx files with Sphinx

[Peter Maydell]

Currently our manual creation includes some .texi files which
are autogenerated from .hx files by running scripts/hxtool.
.hx files are a simple format, where where a line is either a
directive or literal text to be output:

HXCOMM
 -- comment lines, ignored
STEXI/ETEXI
 -- mark start/end of chunks of text to put into the texinfo output only
DEFHEADING/ARCHHEADING
 -- appear in the .h file output verbatim (they are defined as C macros);
    for texi output they are parsed to add in header sections

For Sphinx, rather than creating a file to include, the most
natural way to handle this is to have a small custom Sphinx
extension which will read the .hx file and process it. So
instead of "makefile produces foo.texi from foo.hx, qemu-doc.texi
says '@include foo.texi'", we have "qemu-doc.rst says
'hxtool-doc:: foo.hx', the Sphinx extension for hxtool has
code that runs to handle that Sphinx directive, it reads the .hx
file and emits the appropriate documentation contents". (This is
pretty much the same way the kerneldoc extension works right now.
It also has the advantage that it should work for third-party
services like readthedocs that expect to build the docs directly
with sphinx rather than by invoking our makefiles.)

We'll need to update what the markup is to handle having rST
fragments in it. A very minimalist approach to this would
simply define a new pair of SRST/ERST directives marking the
start/end of chunks of rST text to go into the rST only.
(We might be able to do better than that later, as there's
some repetition currently going on. But we'll probably get
a better idea of how easy it is to avoid the repetition if
we start with a simple conversion.)

Here's what we do with hx files at the moment. We have four:

 hmp-commands.hx
   -- defines monitor commands used by monitor.c; generates
      qemu-monitor.texi, used by qemu-doc.texi
 hmp-commands-info.hx
   -- ditto, for the "info" command's subcommand;
      generates qemu-monitor-info.texi, used by qemu-doc.texi

These two use only the "put this in the texi or in the .h file"
functionality, alternating "raw C code defining an entry for the
monitor command array" with "lump of raw texi for the docs".

 qemu-img-cmds.hx
   -- defines options for qemu-img, used by qemu-img.texi

This uses the STEXI/ETEXI directives to alternate C and texi.
In the for-the-h-file section the only content is always a DEF()
macro invocation defining the option; in the texi is only the
synopsis of the command. This means there's a lot of repetition,
as the DEF macro includes an argument giving the text of the
option synopsis, and then the texi also has that synopsis with
some extra markup. Finally the main qemu-img.texi repeats the
marked-up synopsis later on when it has the actual main documentation
of each command.

 qemu-options.hx
   -- options for qemu proper, used by qemu-doc.texi

This uses only the DEF, DEFHEADING, ARCHHEADING macros
in the for-the-h-file sections (and the DEFHEADING/ARCHHEADING
are read also for texi generation). This also repeats the
synopsis in the DEF macro and in the texi fragment.

So I think my current view is that we should do the very
simple "add SRST/ERST directives" to start with:
 * scripts/hxtool needs to recognize them and just ignore
   the text inside them
 * write the hxtool sphinx extension (shouldn't be too hard)
 * conversion of any particular .hx file then involves
   replacing the STEXI ...texi stuff... ETEXI sections with
   SRST ...rst stuff... ERST. There's no need for any
   particular .hx file to support both texi and rst output
   at the same time

I would initially start with qemu-img-cmds.hx, since that's
pulled in by qemu-img.texi, which we can convert in the
same way I've been doing qemu-nbd and other standalone-ish
manpages. The others are part of the big fat qemu-doc.texi,
which is probably going to be the very last thing we convert...

thanks
-- PMM