[PATCH 00/27] Add qapi-domain Sphinx extension

[John Snow]

This series adds a new qapi-domain extension for Sphinx, which adds a
series of custom directives for documenting QAPI definitions.

GitLab CI: https://gitlab.com/jsnow/qemu/-/pipelines/1259566476

(Link to a demo HTML page at the end of this cover letter, but I want
you to read the cover letter first to explain what you're seeing.)

This adds a new QAPI Index page, cross-references for QMP commands,
events, and data types, and improves the aesthetics of the QAPI/QMP
documentation.

This series adds only the new ReST syntax, *not* the autogenerator. The
ReST syntax used in this series is, in general, not intended for anyone
to actually write by hand. This mimics how Sphinx's own autodoc
extension generates Python domain directives, which are then re-parsed
to produce the final result.

I have prototyped such a generator, but it isn't ready for inclusion
yet. (Rest assured: error context reporting is preserved down to the
line, even in generated ReST. There is no loss in usability for this
approach. It will likely either supplant qapidoc.py or heavily alter
it.) The generator requires only extremely minor changes to
scripts/qapi/parser.py to preserve nested indentation and provide more
accurate line information. It is less invasive than you may
fear. Relying on a secondary ReST parse phase eliminates much of the
complexity of qapidoc.py. Sleep soundly.

The purpose of sending this series in its current form is largely to
solicit feedback on general aesthetics, layout, and features. Sphinx is
a wily beast, and feedback at this stage will dictate how and where
certain features are implemented.

A goal for this syntax (and the generator) is to fully in-line all
command/event/object members, inherited or local, boxed or not, branched
or not. This should provide a boon to using these docs as a reference,
because users will not have to grep around the page looking for various
types, branches, or inherited members. Any arguments types will be
hyperlinked to their definition, further aiding usability. Commands can
be hotlinked from anywhere else in the manual, and will provide a
complete reference directly on the first screenful of information.

(Okay, maybe two screenfuls for commands with a ton of
arguments... There's only so much I can do!)

This RFC series includes a "sandbox" .rst document that showcases the
features of this domain by writing QAPI directives by hand; this
document will be removed from the series before final inclusion. It's
here to serve as a convenient test-bed for y'all to give feedback.

All that said, here's the sandbox document fully rendered:
https://jsnow.gitlab.io/qemu/qapi/index.html

And here's the new QAPI index page created by that sandbox document:
https://jsnow.gitlab.io/qemu/qapi-index.html

Known issues / points of interest:

- The formatting upsets checkpatch. The default line length for the
  "black" formatting tool is a little long. I'll fix it next spin.

- I did my best to preserve cross-version compatibility, but some
  problems have crept in here and there. This series may require
  Sphinx>= 4.0, like the dbus extension. Notably, the Ubuntu build fails
  on Gitlab CI currently. The next version will text against more Sphinx
  versions more rigorously. Sphinx version 5.3.0 and above should work
  perfectly.

- There's a bug in Sphinx itself that may manifest in your testing,
  concerning reported error locations. There's a patch in this series
  that fixes it, but it's later in the series. If you run into the bug
  while testing with this series, try applying that patch first.

- QAPI 'namespaces' aren't yet handled. i.e., there's nothing to
  distinguish entities between QMP, QGA and QSD yet. That feature will
  be added to a future version of this patchset (Likely when the
  generator is ready for inclusion: without it, references will clash
  and the index will gripe about duplicated entries.)

- Per-member features and ifcond are not yet accounted for; though
  definition-scoped features and ifconds are. Please feel free to
  suggest how you'd like to see those represented.

- Inlining all members means there is some ambiguity on what to do with
  doc block sections on inlined entities; features and members have an
  obvious home - body, since, and misc sections are not as obvious on
  how to handle. This will feature more prominently in the generator
  series.

- Some features could be implemented in either the QAPI domain syntax
  *or* the generator, or some combination of both. Depending on
  aesthetic feedback, this may influence where those features should be
  implemented.

- The formatting and aesthetics of branches are a little up in the air,
  see the qapi:union patch for more details.

- It's late, maybe other things. Happy Friday!

Hope you like it!

--js

Harmonie Snow (1):
  docs/qapi-domain: add CSS styling

John Snow (26):
  docs/sphinx: create QAPI domain extension stub
  docs/qapi-domain: add qapi:module directive
  docs/qapi-module: add QAPI domain object registry
  docs/qapi-domain: add QAPI index
  docs/qapi-domain: add resolve_any_xref()
  docs/qapi-domain: add QAPI xref roles
  docs/qapi-domain: add qapi:command directive
  docs/qapi-domain: add :since: directive option
  docs/qapi-domain: add "Arguments:" field lists
  docs/qapi-domain: add "Features:" field lists
  docs/qapi-domain: add "Errors:" field lists
  docs/qapi-domain: add "Returns:" field lists
  docs/qapi-domain: add qapi:enum directive
  docs/qapi-domain: add qapi:alternate directive
  docs/qapi-domain: add qapi:event directive
  docs/qapi-domain: add qapi:struct directive
  docs/qapi-domain: add qapi:union and qapi:branch directives
  docs/qapi-domain: add :deprecated: directive option
  docs/qapi-domain: add :unstable: directive option
  docs/qapi-domain: add :ifcond: directive option
  docs/qapi-domain: RFC patch - add malformed field list entries
  docs/qapi-domain: add warnings for malformed field lists
  docs/qapi-domain: RFC patch - delete malformed field lists
  docs/qapi-domain: add type cross-refs to field lists
  docs/qapi-domain: implement error context reporting fix
  docs/qapi-domain: RFC patch - Add one last sample command

 docs/conf.py                           |   15 +-
 docs/index.rst                         |    1 +
 docs/qapi/index.rst                    |  364 ++++++++
 docs/sphinx-static/theme_overrides.css |   92 +-
 docs/sphinx/qapi-domain.py             | 1061 ++++++++++++++++++++++++
 5 files changed, 1530 insertions(+), 3 deletions(-)
 create mode 100644 docs/qapi/index.rst
 create mode 100644 docs/sphinx/qapi-domain.py

-- 
2.44.0