[PATCH v2 00/23] docs: add basic sphinx-domain rST generator to qapidoc

[John Snow]

based-on: https://patchew.org/QEMU/20241213011307.2942030-1-jsnow@redhat.com/

Hi!

This series is a very, very barebones implementation for the new QAPI
doc generator. It does not have many features that I presented on at KVM
Forum; the point of this patch set is instead to present a stripped down
basis for ongoing work so we can discuss on-list with full context of
the code available to do so.

The documentation this series generates is *not suitable* for replacing
the current document generator, it has a few glaring omissions - on
purpose - those features have been factored out intentionally so they
can be reviewed with fuller context and more careful review.

What this series does:

- Adds the new "Transmogrifier" rST generator to qapidoc.py, which
  generates an in-memory rST document using qapi-domain directives.  -
  Adds a test document that showcases this new transmogrifier. (located
  at build/docs/manual/qapi/index.html; also visible at the bottom of
  the left column TOC on generated docs.)

What this series currently *regresses* /if/ it were to replace the
current doc generator:

- "ifcond" data for anything other than top-level entities is not
  considered or rendered. This means "if" statements for features and
  members are entirely absent. This includes "special features" that
  have been hoisted into the preamble as well, such as :unstable: or
  :deprecated:. This issue is largely unaddressed in my WIP so far,
  I need to solve this in future patch discussion.

- *branches* are themselves not considered at all; they're skipped
   entirely for now. They will be included alongside the inliner later.

- "inherited" members (Markus hates that I use this term, I'm sorry, I
  know it isn't 1:1 with OO inheritance I just don't have a nicer word
  that my brain remembers, forgive me) are skipped; "The members of ..."
  messages are not rendered either. This is addressed by the inliner.

- Undocumented members are not auto-generated.

Other things unhandled, but are not regressions per se:

- The inliner is not present at all. This series renders only
  documentation exactly as it is exists in the source files.

- Undocumented return types are not autogenerated.

- Pseudofeatures (Things like allow-oob) are not generated as documented
  features.

- Documentation culling: all entities are documented whether or not
  they're relevant to the wire format.

Everything else *should* be no worse off than it was, and in many cases
better.

My current goal is to tentatively agree that these patches look good
(save for some minor nits like terminology/comments/docs/commit nits),
then to send an updated version of this series with additional commits
that start addressing the regressions and adding the new features like
the inliner.

Once the new generator as a whole looks good, I will shift focus back to
review on the qapi-domain extension itself. The reason for doing the
review in reverse order is because the design of the generator informs
the design of the domain, but reviewers are currently more familiar with
the generator - so we're doing the review backwards. Treat it as a
magical black box for now that somehow magically transforms rST into
pretty docs.

V2:
 - Mostly adjusted commit messages and comments per list feedback
 - Changed "tag" to "kind" and adjusted code, comments and commits
   accordingly
 - removed assertion around special features + conditionals, for now

John Snow (23):
  docs/qapidoc: support header-less freeform sections
  qapi/parser: adjust info location for doc body section
  docs/qapidoc: remove example section support
  qapi: expand tags to all doc sections
  qapi/schema: add __repr__ to QAPIDoc.Section
  docs/qapidoc: add transmogrifier stub
  docs/qapidoc: add transmogrifier class stub
  docs/qapidoc: add visit_module() method
  qapi/source: allow multi-line QAPISourceInfo advancing
  docs/qapidoc: add visit_freeform() method
  docs/qapidoc: add preamble() method
  docs/qapidoc: add visit_paragraph() method
  docs/qapidoc: add visit_errors() method
  docs/qapidoc: add format_type() method
  docs/qapidoc: add add_field() and generate_field() helper methods
  docs/qapidoc: add visit_feature() method
  docs/qapidoc: prepare to record entity being transmogrified
  docs/qapidoc: add visit_returns() method
  docs/qapidoc: add visit_member() method
  docs/qapidoc: add visit_sections() method
  docs/qapidoc: add visit_entity()
  docs/qapidoc: implement transmogrify() method
  docs/qapidoc: add transmogrifier test document

 docs/index.rst                 |   1 +
 docs/qapi/index.rst            |  53 +++++
 docs/sphinx/qapidoc.py         | 418 ++++++++++++++++++++++++++++++---
 scripts/qapi/parser.py         | 118 +++++++---
 scripts/qapi/source.py         |   4 +-
 tests/qapi-schema/doc-good.out |  10 +-
 tests/qapi-schema/test-qapi.py |   2 +-
 7 files changed, 541 insertions(+), 65 deletions(-)
 create mode 100644 docs/qapi/index.rst

-- 
2.47.1