[Qemu-commits] [qemu/qemu] 025720: python/aqmp: add greeting property to QMPClient

[Richard Henderson]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 0257209a09793bfaf1b662faad5558e7011e748a
      
https://github.com/qemu/qemu/commit/0257209a09793bfaf1b662faad5558e7011e748a
  Author: John Snow <jsnow@redhat.com>
  Date:   2021-10-12 (Tue, 12 Oct 2021)

  Changed paths:
    M python/qemu/aqmp/qmp_client.py

  Log Message:
  -----------
  python/aqmp: add greeting property to QMPClient

Expose the greeting as a read-only property of QMPClient so it can be
retrieved at-will.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20210923004938.3999963-2-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>


  Commit: 16cce725ed87e3ed5901c93e61514840c989216a
      
https://github.com/qemu/qemu/commit/16cce725ed87e3ed5901c93e61514840c989216a
  Author: John Snow <jsnow@redhat.com>
  Date:   2021-10-12 (Tue, 12 Oct 2021)

  Changed paths:
    M python/qemu/aqmp/events.py

  Log Message:
  -----------
  python/aqmp: add .empty() method to EventListener

Synchronous clients may want to know if they're about to block waiting
for an event or not. A method such as this is necessary to implement a
compatible interface for the old QEMUMonitorProtocol using the new async
internals.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20210923004938.3999963-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>


  Commit: 6bfebc7306e42910cd33553d9ed385ef475d8196
      
https://github.com/qemu/qemu/commit/6bfebc7306e42910cd33553d9ed385ef475d8196
  Author: John Snow <jsnow@redhat.com>
  Date:   2021-10-12 (Tue, 12 Oct 2021)

  Changed paths:
    M python/qemu/aqmp/events.py

  Log Message:
  -----------
  python/aqmp: Return cleared events from EventListener.clear()

This serves two purposes:

(1) It is now possible to discern whether or not clear() removed any
event(s) from the queue with absolute certainty, and

(2) It is now very easy to get a List of all pending events in one
chunk, which is useful for the sync bridge.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20210923004938.3999963-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>


  Commit: 6e2f6ec5615f50c38e21542bb66bb690a289cbfc
      
https://github.com/qemu/qemu/commit/6e2f6ec5615f50c38e21542bb66bb690a289cbfc
  Author: John Snow <jsnow@redhat.com>
  Date:   2021-10-12 (Tue, 12 Oct 2021)

  Changed paths:
    M python/qemu/aqmp/qmp_client.py

  Log Message:
  -----------
  python/aqmp: add send_fd_scm

Add an implementation for send_fd_scm to the async QMP implementation.
Like socket_scm_helper mentions, a non-empty payload is required for
QEMU to process the ancillary data. A space is most useful because it
does not disturb the parsing of subsequent JSON objects.

A note on "voiding the warranty":

Python 3.11 removes support for calling sendmsg directly from a
transport's socket. There is no other interface for doing this, our use
case is, I suspect, "quite unique".

As far as I can tell, this is safe to do -- send_fd_scm is a synchronous
function and we can be guaranteed that the async coroutines will *not* be
running when it is invoked. In testing, it works correctly.

I investigated quite thoroughly the possibility of creating my own
asyncio Transport (The class that ultimately manages the raw socket
object) so that I could manage the socket myself, but this is so wildly
invasive and unportable I scrapped the idea. It would involve a lot of
copy-pasting of various python utilities and classes just to re-create
the same infrastructure, and for extremely little benefit. Nah.

Just boldly void the warranty instead, while I try to follow up on
https://bugs.python.org/issue43232

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20210923004938.3999963-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>


  Commit: 58026b11f36f9a19fd35fdfbfa50be7bd78e70a5
      
https://github.com/qemu/qemu/commit/58026b11f36f9a19fd35fdfbfa50be7bd78e70a5
  Author: John Snow <jsnow@redhat.com>
  Date:   2021-10-12 (Tue, 12 Oct 2021)

  Changed paths:
    M python/qemu/aqmp/models.py

  Log Message:
  -----------
  python/aqmp: Add dict conversion method to Greeting object

The iotests interface expects to return the greeting as a dict; AQMP
offers it as a rich object.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20210923004938.3999963-6-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>


  Commit: 3e55dc35b8ed8cea53ef2604874d2fe882fbadc1
      
https://github.com/qemu/qemu/commit/3e55dc35b8ed8cea53ef2604874d2fe882fbadc1
  Author: John Snow <jsnow@redhat.com>
  Date:   2021-10-12 (Tue, 12 Oct 2021)

  Changed paths:
    M python/qemu/aqmp/protocol.py

  Log Message:
  -----------
  python/aqmp: Reduce severity of EOFError-caused loop terminations

When we encounter an EOFError, we don't know if it's an "error" in the
perspective of the user of the library yet. Therefore, we should not log
it as an error. Reduce the severity of this logging message to "INFO" to
indicate that it's something that we expect to occur during the normal
operation of the library.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20210923004938.3999963-7-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>


  Commit: 3a3d84f5ec0bca2586a1a83691182576a9447130
      
https://github.com/qemu/qemu/commit/3a3d84f5ec0bca2586a1a83691182576a9447130
  Author: John Snow <jsnow@redhat.com>
  Date:   2021-10-12 (Tue, 12 Oct 2021)

  Changed paths:
    M python/qemu/aqmp/__init__.py

  Log Message:
  -----------
  python/aqmp: Disable logging messages by default

AQMP is a library, and ideally it should not print error diagnostics
unless a user opts into seeing them. By default, Python will print all
WARNING, ERROR or CRITICAL messages to screen if no logging
configuration has been created by a client application.

In AQMP's case, ERROR logging statements are used to report additional
detail about runtime failures that will also eventually be reported to the
client library via an Exception, so these messages should not be
rendered by default.

(Why bother to have them at all, then? In async contexts, there may be
multiple Exceptions and we are only able to report one of them back to
the client application. It is not reasonably easy to predict ahead of
time if one or more of these Exceptions will be squelched. Therefore,
it's useful to log intermediate failures to help make sense of the
ultimate, resulting failure.)

Add a NullHandler that will suppress these messages until a client
application opts into logging via logging.basicConfig or similar. Note
that upon calling basicConfig(), this handler will *not* suppress these
messages from being displayed by the client's configuration.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20210923004938.3999963-8-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>


  Commit: d911accf0a88070120f7cc71c065fb797484d9b7
      
https://github.com/qemu/qemu/commit/d911accf0a88070120f7cc71c065fb797484d9b7
  Author: John Snow <jsnow@redhat.com>
  Date:   2021-10-12 (Tue, 12 Oct 2021)

  Changed paths:
    M python/qemu/machine/machine.py
    M python/qemu/qmp/__init__.py
    M python/qemu/qmp/qmp_shell.py

  Log Message:
  -----------
  python/qmp: clear events on get_events() call

All callers in the tree *already* clear the events after a call to
get_events(). Do it automatically instead and update callsites to remove
the manual clear call.

These semantics are quite a bit easier to emulate with async QMP, and
nobody appears to be abusing some emergent properties of what happens if
you decide not to clear them, so let's dial down to the dumber, simpler
thing.

Specifically: callers of clear() right after a call to get_events() are
more likely expressing their desire to not see any events they just
retrieved, whereas callers of clear_events() not in relation to a recent
call to pull_event/get_events are likely expressing their desire to
simply drop *all* pending events straight onto the floor. In the sync
world, this is safe enough; in the async world it's nearly impossible to
promise that nothing happens between getting and clearing the
events.

Making the retrieval also clear the queue is vastly simpler.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20210923004938.3999963-9-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>


  Commit: 514d00df5f44f220d0b97cc71323275067d3e60e
      
https://github.com/qemu/qemu/commit/514d00df5f44f220d0b97cc71323275067d3e60e
  Author: John Snow <jsnow@redhat.com>
  Date:   2021-10-12 (Tue, 12 Oct 2021)

  Changed paths:
    M python/qemu/machine/machine.py
    M python/qemu/qmp/__init__.py

  Log Message:
  -----------
  python/qmp: add send_fd_scm directly to QEMUMonitorProtocol

It turns out you can do this directly from Python ... and because of
this, you don't need to worry about setting the inheritability of the
fds or spawning another process.

Doing this is helpful because it allows QEMUMonitorProtocol to keep its
file descriptor and socket object as private implementation
details. /that/ is helpful in turn because it allows me to write a
compatible, alternative implementation.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20210923004938.3999963-10-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>


  Commit: c163c723ef92d0f629d015902396f2c67328b2e5
      
https://github.com/qemu/qemu/commit/c163c723ef92d0f629d015902396f2c67328b2e5
  Author: John Snow <jsnow@redhat.com>
  Date:   2021-10-12 (Tue, 12 Oct 2021)

  Changed paths:
    M python/qemu/machine/machine.py
    M python/qemu/machine/qtest.py
    M tests/Makefile.include
    M tests/meson.build
    M tests/qemu-iotests/iotests.py
    R tests/qemu-iotests/meson.build
    R tests/qemu-iotests/socket_scm_helper.c
    M tests/qemu-iotests/testenv.py

  Log Message:
  -----------
  python, iotests: remove socket_scm_helper

It's not used anymore, now.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20210923004938.3999963-11-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>


  Commit: ee26ce674a93c824713542cec3b6a9ca85459165
      
https://github.com/qemu/qemu/commit/ee26ce674a93c824713542cec3b6a9ca85459165
  Author: Richard Henderson <richard.henderson@linaro.org>
  Date:   2021-10-12 (Tue, 12 Oct 2021)

  Changed paths:
    M python/qemu/aqmp/__init__.py
    M python/qemu/aqmp/events.py
    M python/qemu/aqmp/models.py
    M python/qemu/aqmp/protocol.py
    M python/qemu/aqmp/qmp_client.py
    M python/qemu/machine/machine.py
    M python/qemu/machine/qtest.py
    M python/qemu/qmp/__init__.py
    M python/qemu/qmp/qmp_shell.py
    M tests/Makefile.include
    M tests/meson.build
    M tests/qemu-iotests/iotests.py
    R tests/qemu-iotests/meson.build
    R tests/qemu-iotests/socket_scm_helper.c
    M tests/qemu-iotests/testenv.py

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/jsnow/tags/python-pull-request' into 
staging

Pull request

# gpg: Signature made Tue 12 Oct 2021 02:36:07 PM PDT
# gpg:                using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full]

* remotes/jsnow/tags/python-pull-request:
  python, iotests: remove socket_scm_helper
  python/qmp: add send_fd_scm directly to QEMUMonitorProtocol
  python/qmp: clear events on get_events() call
  python/aqmp: Disable logging messages by default
  python/aqmp: Reduce severity of EOFError-caused loop terminations
  python/aqmp: Add dict conversion method to Greeting object
  python/aqmp: add send_fd_scm
  python/aqmp: Return cleared events from EventListener.clear()
  python/aqmp: add .empty() method to EventListener
  python/aqmp: add greeting property to QMPClient

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>


Compare: https://github.com/qemu/qemu/compare/8be1d4ed9838...ee26ce674a93