Re: Making QEMU easier for management tools and applications

[Markus Armbruster]

Daniel P. Berrangé <address@hidden> writes:

> On Thu, Jan 23, 2020 at 04:07:09PM -0500, John Snow wrote:
>> 
>> 
>> On 1/23/20 2:01 PM, Daniel P. Berrangé wrote:
[...]
>> > I guess my point is that with a scrap & startover view point, we
>> > should arguably completely ignore the design question of how to
>> > flatten JSON for humans/command line, as it is the wrong problem.
>> > Instead focus on the problem of making use of JSON the best way
>> > to deal with QEMU both functionally and practically, for humans
>> > and machines alike.

Note: I understand "JSON" to stand for "JSON and possibly another
concrete syntax better suited for humans".  Your examples below show
YAML in that role.

[...]
> Here's one conceptual vision of how a better QEMU might look:
>
>   * qemu-runtime-$TARGET
>
>     A binary that contains the implementation for the machine
>     emulator for $TARGET.
>
>     This has no command line arguments except for a UNIX
>     socket path which is a QMP server
>
>
>   * qemu-launcher-$TARGET
>
>     A binary that is able to launch qemu-runtime-$TARGET
>     with jailers active.
>
>     This has no command line arguments except for a pair
>     of UNIX socket paths. One is a QMP server, the other
>     is the path for the QMP of qemu-runtime-$TARGET.
>
>     Commands it processes will be in automatically proxied
>     through to the qemu-runtime-$TARGET QMP, with appropriate
>     jailer updates being done in between.

See Paolo's reply.

>   * qemu-client
>
>     A binary that speaks QMP, connects, runs a single command,
>     disconnects.

The single command is specified how?

Ah, "Example usage" below shows it's taken from a file argument.

>     It is used to talk to either qemu-runtime-$TARGET or
>     qemu-launcher-$TARGET, depending on whether the mgmt app
>     or user wants to be making use of the jailer facilities
>     or not.  

Effectively ditches the CLI in favor of QMP.  One major external host
interface instead of two.

The connection to "making use of JSON the best way to deal with QEMU
both functionally and practically, for humans and machines alike" is
"make it possible to use just QMP for *everything*".  Correct?

We've toyed with the "Just QMP" idea almost since QMP exists.  We never
went beyond "wouldn't it be nice if".

The main chunk of work is providing the CLI functionality we need in
QMP.  Requires QAPIfication of the parts that aren't QAPIfied, yet.

Since parts of CLI overlap with QMP, providing CLI functionality need
not require new QMP commands.  For instance, CLI does cold plug, QMP
does hot plug.  The existing QMP interface could do both: hot while the
machine runs, else cold.

Some functionality makes sense only during initial startup.  Implied in
CLI.  In QMP, it has to be tied to the run state.  No big deal.

"Just QMP" is of course not the only way to do "JSON first".  Another
one is bringing JSON to the CLI.  I explored that: CLI QAPIfication.

Requires the same QAPIfication as "Just QMP" does.

Where "Just QMP" is a hard compatibility break by design, "JSON in CLI"
gives us a choice: we can choose to break compatibility, e.g. by
creating the exact same set of new executables you propose.  Or we can
try to evolve the existing CLI compatibly.  The latter is a tar pit,
because the existing CLI is.  But it's a tar pit *we* get to choose.  We
can't blame "JSON in CLI" for it, only ourselves.

Even though I've worked on "JSON in CLI", I'm not overly attached to it.
If we decide "Just QMP" is the better path forward, I'll happily go
along.  I believe the bulk of the work will be the same: QAPIfying
stuff.

>   * qemu-system-$TARGET
>
>     The current binaries that exist today.
>
>     qemu-system-$TARGET should not be part of our formal
>     stability promise. We won't gratuitously / knowingly
>     break without good reason, but we will accept that
>     breakage can happen. Stability is only offered by
>     the qemu-{runtime,launcher}-$TARGET.
>
>     Several choices for their future in long term:
>
>       - Leave them as-is and basically ignore them
>         whereever practical going forward, so we
>       minimally worry about backcompat breakage
>
>       - Plan to re-write them so that they are simply
>         a shim the forks+execs qemu-runtime-$TARGET
>       and does syntax translation from CLI/HMP/QMP.
>
>       - Deprecate them with a view to deletion entirely
>         in $NNN years. For some large-ish value of NNN,
>       given how well known they are

How do the other complex executables like qemu-img, qemu-nbd fit into
this picture?

Do they become redundant somehow for non-human users?

If not, will they get a QMP-only sibling, like qemu-system-$TARGET gets
qemu-runtime-$TARGET?

> Example usage:
>
> 1. Launch the QEMU runtime for the desired target
>
>      $ qemu-runtime-x86_64 myvm.sock
>
> 2. Load the configuration to define the VM
>
>    $ cat myvm.yaml
>    commands:
>      - machine_declare:
>          name: pc-q35-5.0
>        ...
>      - blockdev_add:
>          ...
>      - device_add:
>          ...
>      - blockdev_add:
>          ...
>      - device_add:
>          ...
>    $ qemu-client myvm.sock myvm.yaml
>
>
> 3. Hotplug a disk
>
>    $ cat mynewdisk.yaml
>    commands:
>      - blockdev_add:
>          ...
>      - device_add:
>          ...
>    $ qemu-client myvm.sock mynewdisk.yaml
>
>
> 3. Hotunplug a disk
>
>    $ cat myolddisk.yaml
>    commands:
>      - device_del:
>          ...
>      - blockdev_del:
>          ...
>    $ qemu-client myvm.sock myolddisk.yaml
>
> Using jailers, just means adding in a use of qemu-launcher-$TARGET
> at the start.
>
>
> Regards,
> Daniel