[PATCH v3 00/19] monitor: explicitly permit QMP commands to be added for all use cases

[Daniel P . Berrangé]

Previous postings:

  v1: https://lists.gnu.org/archive/html/qemu-devel/2021-09/msg02295.html
  v2: https://lists.gnu.org/archive/html/qemu-devel/2021-09/msg03703.html

We are still adding HMP commands without any QMP counterparts. This is
done because there are a reasonable number of scenarios where the cost
of designing a QAPI data type for the command is not justified.

This has the downside, however, that we will never be able to fully
isolate the monitor code from the remainder of QEMU internals. It is
desirable to be able to get to a point where subsystems in QEMU are
exclusively implemented using QAPI types and never need to have any
knowledge of the monitor.

The way to get there is to stop adding commands to HMP only. All
commands must be implemented using QMP and any HMP equivalent be
a shim around the QMP implemetation. We don't want to compromise
our supportability of QMP long term though.

This series proposes that we relax our requirements around fine grained
QAPI data design, but with the caveat that any command taking this
design approach is mandated to use the 'x-' name prefix. This tradeoff
should be suitable for any commands we have been adding exclusively to
HMP in recent times, and thus mean we have mandate QMP support for all
new commands going forward.

The series then converts the following HMP commands to be QMP shims.

    info opcount
    info jit
    info irq
    info lapic
    info cmma
    info skeys
    info ramblock
    info rdma
    info usb
    info numa
    info profile
    info roms

A full conversion would also enable HMP to be emulated entirely
outside QEMU. This could be interesting if we introduce a new QEMU
system emulator binary which is legacy free and 100% controlled
via QMP, as it would let us provide HMP backcompat around it
without the burden of HMP being integrated directly.

There are still many HMP commands with no QMP counterpart after
this series.

 - A few are not relevant to port as they directly
   reflect HMP functionality (help, info history).
 - A few are sort of available in QMP but look quite
   different (drive_add vs blockdev_add)
 - A few are complicated. "info usbhost" is a dynamically
   loaded HMP command inside a loadable module and we
   don't have a way to dynamically register QMP handlers
   at runtime.
 - Most are just tedious gruntwork.

Changed in v3:

 - Temporarily spun off the 'info registers' and 'info tlb'
   conversions. These required 30+ patches across all the
   targets and was making this series too large and conflict
   prone and spanning too many subsystems.

   I'll re-submit those two later

 - Pull in a fix for the 'info lapic' command

 - Misc improvements to the documentation after reviews

 - Add helper for GString -> HumanReadableText conversion

Changed in v2:

 - Improved documentation in response to feedback
 - Finished "info registers" conversion on all targets
 - Got a bit carried away and converted many many more
   commands

Daniel P. Berrangé (18):
  docs/devel: rename file for writing monitor commands
  docs/devel: tweak headings in monitor command docs
  docs/devel: document expectations for QAPI data modelling for QMP
  docs/devel: add example of command returning unstructured text
  docs/devel: document expectations for HMP commands in the future
  monitor: remove 'info ioapic' HMP command
  qapi: introduce x-query-roms QMP command
  qapi: introduce x-query-profile QMP command
  qapi: introduce x-query-numa QMP command
  qapi: introduce x-query-usb QMP command
  qapi: introduce x-query-rdma QMP command
  qapi: introduce x-query-ramblock QMP command
  qapi: introduce x-query-skeys QMP command
  qapi: introduce x-query-cmma QMP command
  qapi: introduce x-query-lapic QMP command
  qapi: introduce x-query-irq QMP command
  qapi: introduce x-query-jit QMP command
  qapi: introduce x-query-opcount QMP command

Dongli Zhang (1):
  hmp: synchronize cpu state for lapic info

 accel/tcg/cpu-exec.c                          |  51 +++++-
 accel/tcg/hmp.c                               |  24 ++-
 accel/tcg/translate-all.c                     |  84 ++++-----
 docs/devel/index.rst                          |   2 +-
 ...mands.rst => writing-monitor-commands.rst} | 134 ++++++++++++++-
 hmp-commands-info.hx                          |  15 --
 hw/core/cpu-common.c                          |   7 +
 hw/core/loader.c                              |  53 ++++--
 hw/core/machine-hmp-cmds.c                    |  33 +---
 hw/core/machine-qmp-cmds.c                    |  40 +++++
 hw/rdma/rdma_rm.c                             | 104 +++++------
 hw/rdma/rdma_rm.h                             |   2 +-
 hw/rdma/vmw/pvrdma_main.c                     |  31 ++--
 hw/s390x/s390-skeys.c                         |  35 +++-
 hw/s390x/s390-stattrib.c                      |  56 ++++--
 hw/usb/bus.c                                  |  36 +++-
 include/exec/cpu-all.h                        |   6 +-
 include/exec/ramlist.h                        |   2 +-
 include/hw/core/cpu.h                         |  10 ++
 include/hw/rdma/rdma.h                        |   2 +-
 include/monitor/hmp-target.h                  |   1 -
 include/qapi/type-helpers.h                   |  14 ++
 include/tcg/tcg.h                             |   4 +-
 monitor/hmp-cmds.c                            |  81 +++------
 monitor/misc.c                                |  30 +---
 monitor/qmp-cmds.c                            | 116 +++++++++++++
 qapi/common.json                              |  11 ++
 qapi/machine-target.json                      |  47 +++++
 qapi/machine.json                             | 110 ++++++++++++
 qapi/meson.build                              |   3 +
 qapi/qapi-type-helpers.c                      |  23 +++
 scripts/qapi/commands.py                      |   1 +
 softmmu/physmem.c                             |  19 ++-
 stubs/usb-dev-stub.c                          |   8 +
 target/i386/cpu-dump.c                        | 161 +++++++++---------
 target/i386/cpu.h                             |   4 +-
 target/i386/monitor.c                         |  50 ++++--
 tcg/tcg.c                                     |  98 ++++++-----
 tests/qtest/qmp-cmd-test.c                    |   8 +
 39 files changed, 1063 insertions(+), 453 deletions(-)
 rename docs/devel/{writing-qmp-commands.rst => writing-monitor-commands.rst} 
(78%)
 create mode 100644 include/qapi/type-helpers.h
 create mode 100644 qapi/qapi-type-helpers.c

-- 
2.31.1