[Qemu-commits] [qemu/qemu] f25e7a: block/nvme: Add driver statistics for access align...

[Peter Maydell]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: f25e7ab2b091fae1ae3e9f55b5244ddcdb1653bb
      
https://github.com/qemu/qemu/commit/f25e7ab2b091fae1ae3e9f55b5244ddcdb1653bb
  Author: Philippe Mathieu-Daudé <philmd@redhat.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M block/nvme.c
    M qapi/block-core.json

  Log Message:
  -----------
  block/nvme: Add driver statistics for access alignment and hw errors

Keep statistics of some hardware errors, and number of
aligned/unaligned I/O accesses.

QMP example booting a full RHEL 8.3 aarch64 guest:

{ "execute": "query-blockstats" }
{
    "return": [
        {
            "device": "",
            "node-name": "drive0",
            "stats": {
                "flush_total_time_ns": 6026948,
                "wr_highest_offset": 3383991230464,
                "wr_total_time_ns": 807450995,
                "failed_wr_operations": 0,
                "failed_rd_operations": 0,
                "wr_merged": 3,
                "wr_bytes": 50133504,
                "failed_unmap_operations": 0,
                "failed_flush_operations": 0,
                "account_invalid": false,
                "rd_total_time_ns": 1846979900,
                "flush_operations": 130,
                "wr_operations": 659,
                "rd_merged": 1192,
                "rd_bytes": 218244096,
                "account_failed": false,
                "idle_time_ns": 2678641497,
                "rd_operations": 7406,
            },
            "driver-specific": {
                "driver": "nvme",
                "completion-errors": 0,
                "unaligned-accesses": 2959,
                "aligned-accesses": 4477
            },
            "qdev": "/machine/peripheral-anon/device[0]/virtio-backend"
        }
    ]
}

Suggested-by: Stefan Hajnoczi <stefanha@gmail.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20201001162939.1567915-1-philmd@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: 049f55502a020f2d56dece94164bf8cf901f855d
      
https://github.com/qemu/qemu/commit/049f55502a020f2d56dece94164bf8cf901f855d
  Author: Coiby Xu <coiby.xu@gmail.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M contrib/libvhost-user/libvhost-user-glib.c
    M contrib/libvhost-user/libvhost-user.c
    M contrib/libvhost-user/libvhost-user.h
    M tests/vhost-user-bridge.c
    M tools/virtiofsd/fuse_virtio.c

  Log Message:
  -----------
  libvhost-user: Allow vu_message_read to be replaced

Allow vu_message_read to be replaced by one which will make use of the
QIOChannel functions. Thus reading vhost-user message won't stall the
guest. For slave channel, we still use the default vu_message_read.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Coiby Xu <coiby.xu@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20200918080912.321299-2-coiby.xu@gmail.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: f1baeee9ffeddcc068d3536f90b5c3e9f81d9309
      
https://github.com/qemu/qemu/commit/f1baeee9ffeddcc068d3536f90b5c3e9f81d9309
  Author: Coiby Xu <coiby.xu@gmail.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M contrib/libvhost-user/libvhost-user.c

  Log Message:
  -----------
  libvhost-user: remove watch for kick_fd when de-initialize vu-dev

When the client is running in gdb and quit command is run in gdb,
QEMU will still dispatch the event which will cause segment fault in
the callback function.

Signed-off-by: Coiby Xu <coiby.xu@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-id: 20200918080912.321299-3-coiby.xu@gmail.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: 70eb2c079cdfa835c70aec8887710b18092516a0
      
https://github.com/qemu/qemu/commit/70eb2c079cdfa835c70aec8887710b18092516a0
  Author: Coiby Xu <coiby.xu@gmail.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M util/meson.build
    A util/vhost-user-server.c
    A util/vhost-user-server.h

  Log Message:
  -----------
  util/vhost-user-server: generic vhost user server

Sharing QEMU devices via vhost-user protocol.

Only one vhost-user client can connect to the server one time.

Suggested-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Coiby Xu <coiby.xu@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-id: 20200918080912.321299-4-coiby.xu@gmail.com
[Fixed size_t %lu -> %zu format string compiler error.
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: 5937835ac4cfb2f5e16bebf13b9ea42770c96785
      
https://github.com/qemu/qemu/commit/5937835ac4cfb2f5e16bebf13b9ea42770c96785
  Author: Coiby Xu <coiby.xu@gmail.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M hw/core/qdev-properties-system.c
    A util/block-helpers.c
    A util/block-helpers.h
    M util/meson.build

  Log Message:
  -----------
  block: move logical block size check function to a common utility function

Move the constants from hw/core/qdev-properties.c to
util/block-helpers.h so that knowledge of the min/max values is

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Coiby Xu <coiby.xu@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Acked-by: Eduardo Habkost <ehabkost@redhat.com>
Message-id: 20200918080912.321299-5-coiby.xu@gmail.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: 3578389bcf76c824a5d82e6586a6f0c71e56f2aa
      
https://github.com/qemu/qemu/commit/3578389bcf76c824a5d82e6586a6f0c71e56f2aa
  Author: Coiby Xu <coiby.xu@gmail.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    A block/export/vhost-user-blk-server.c
    A block/export/vhost-user-blk-server.h
    M block/meson.build
    M softmmu/vl.c

  Log Message:
  -----------
  block/export: vhost-user block device backend server

By making use of libvhost-user, block device drive can be shared to
the connected vhost-user client. Only one client can connect to the
server one time.

Since vhost-user-server needs a block drive to be created first, delay
the creation of this object.

Suggested-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Coiby Xu <coiby.xu@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-id: 20200918080912.321299-6-coiby.xu@gmail.com
[Shorten "vhost_user_blk_server" string to "vhost_user_blk" to avoid the
following compiler warning:
../block/export/vhost-user-blk-server.c:178:50: error: ‘%s’ directive output 
truncated writing 21 bytes into a region of size 20 [-Werror=format-truncation=]
and fix "Invalid size %ld ..." ssize_t format string arguments for
32-bit hosts.
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: 8675a37681d99af94fca6664056d402b05453c78
      
https://github.com/qemu/qemu/commit/8675a37681d99af94fca6664056d402b05453c78
  Author: Coiby Xu <coiby.xu@gmail.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M MAINTAINERS

  Log Message:
  -----------
  MAINTAINERS: Add vhost-user block device backend server maintainer

Suggested-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: Coiby Xu <coiby.xu@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-id: 20200918080912.321299-8-coiby.xu@gmail.com
[Removed reference to vhost-user-blk-test.c, it will be sent in a
separate pull request.
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: 1d7874568b0984ae7880e6ccd0d31c0aba442254
      
https://github.com/qemu/qemu/commit/1d7874568b0984ae7880e6ccd0d31c0aba442254
  Author: Stefan Hajnoczi <stefanha@redhat.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M util/vhost-user-server.c

  Log Message:
  -----------
  util/vhost-user-server: s/fileds/fields/ typo fix

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20200924151549.913737-3-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: 46a096c87a6464dfbdd71a89e2143c55ef45395c
      
https://github.com/qemu/qemu/commit/46a096c87a6464dfbdd71a89e2143c55ef45395c
  Author: Stefan Hajnoczi <stefanha@redhat.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M util/vhost-user-server.c

  Log Message:
  -----------
  util/vhost-user-server: drop unnecessary QOM cast

We already have access to the value with the correct type (ioc and sioc
are the same QIOChannel).

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20200924151549.913737-4-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: dad4f19431653abf2a576ed062f8de4223eb228b
      
https://github.com/qemu/qemu/commit/dad4f19431653abf2a576ed062f8de4223eb228b
  Author: Stefan Hajnoczi <stefanha@redhat.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M util/vhost-user-server.c

  Log Message:
  -----------
  util/vhost-user-server: drop unnecessary watch deletion

Explicitly deleting watches is not necessary since libvhost-user calls
remove_watch() during vu_deinit(). Add an assertion to check this
though.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20200924151549.913737-5-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: df6af7ce77447be89e0d054dc06fb496ec4e69b4
      
https://github.com/qemu/qemu/commit/df6af7ce77447be89e0d054dc06fb496ec4e69b4
  Author: Stefan Hajnoczi <stefanha@redhat.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M block/export/vhost-user-blk-server.c

  Log Message:
  -----------
  block/export: consolidate request structs into VuBlockReq

Only one struct is needed per request. Drop req_data and the separate
VuBlockReq instance. Instead let vu_queue_pop() allocate everything at
once.

This fixes the req_data memory leak in vu_block_virtio_process_req().

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20200924151549.913737-6-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: 47ba680466d83adfa8c58620d4f5855c0de1a144
      
https://github.com/qemu/qemu/commit/47ba680466d83adfa8c58620d4f5855c0de1a144
  Author: Stefan Hajnoczi <stefanha@redhat.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M block/export/vhost-user-blk-server.c
    M util/vhost-user-server.c
    M util/vhost-user-server.h

  Log Message:
  -----------
  util/vhost-user-server: drop unused DevicePanicNotifier

The device panic notifier callback is not used. Drop it.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20200924151549.913737-7-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: 8c7f7cbca0eb49cc86333ef8fa8068abb400520e
      
https://github.com/qemu/qemu/commit/8c7f7cbca0eb49cc86333ef8fa8068abb400520e
  Author: Stefan Hajnoczi <stefanha@redhat.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M util/vhost-user-server.c

  Log Message:
  -----------
  util/vhost-user-server: fix memory leak in vu_message_read()

fds[] is leaked when qio_channel_readv_full() fails.

Use vmsg->fds[] instead of keeping a local fds[] array. Then we can
reuse goto fail to clean up fds. vmsg->fd_num must be zeroed before the
loop to make this safe.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20200924151549.913737-8-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: edaf6205a3a58e09bf31b7ebd73d8fc470e96e07
      
https://github.com/qemu/qemu/commit/edaf6205a3a58e09bf31b7ebd73d8fc470e96e07
  Author: Stefan Hajnoczi <stefanha@redhat.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M util/vhost-user-server.c

  Log Message:
  -----------
  util/vhost-user-server: check EOF when reading payload

Unexpected EOF is an error that must be reported.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20200924151549.913737-9-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: 7185c8577653533ee89e7d070d77473096f0ff74
      
https://github.com/qemu/qemu/commit/7185c8577653533ee89e7d070d77473096f0ff74
  Author: Stefan Hajnoczi <stefanha@redhat.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M block/export/vhost-user-blk-server.c
    M util/vhost-user-server.c
    M util/vhost-user-server.h

  Log Message:
  -----------
  util/vhost-user-server: rework vu_client_trip() coroutine lifecycle

The vu_client_trip() coroutine is leaked during AioContext switching. It
is also unsafe to destroy the vu_dev in panic_cb() since its callers
still access it in some cases.

Rework the lifecycle to solve these safety issues.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20200924151549.913737-10-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: 0534b1b227ef85c59d65b5ec5bee9bcfb8f91b1f
      
https://github.com/qemu/qemu/commit/0534b1b227ef85c59d65b5ec5bee9bcfb8f91b1f
  Author: Stefan Hajnoczi <stefanha@redhat.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M block/export/vhost-user-blk-server.c

  Log Message:
  -----------
  block/export: report flush errors

Propagate the flush return value since errors are possible.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20200924151549.913737-11-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: 90fc91d50b76cf3f32e2ad5b79484ac9680e1d6a
      
https://github.com/qemu/qemu/commit/90fc91d50b76cf3f32e2ad5b79484ac9680e1d6a
  Author: Stefan Hajnoczi <stefanha@redhat.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M block/export/export.c
    M block/export/meson.build
    M block/export/vhost-user-blk-server.c
    M block/export/vhost-user-blk-server.h
    M block/meson.build
    M qapi/block-export.json
    M util/vhost-user-server.c

  Log Message:
  -----------
  block/export: convert vhost-user-blk server to block export API

Use the new QAPI block exports API instead of defining our own QOM
objects.

This is a large change because the lifecycle of VuBlockDev needs to
follow BlockExportDriver. QOM properties are replaced by QAPI options
objects.

VuBlockDev is renamed VuBlkExport and contains a BlockExport field.
Several fields can be dropped since BlockExport already has equivalents.

The file names and meson build integration will be adjusted in a future
patch. libvhost-user should probably be built as a static library that
is linked into QEMU instead of as a .c file that results in duplicate
compilation.

The new command-line syntax is:

  $ qemu-storage-daemon \
      --blockdev file,node-name=drive0,filename=test.img \
      --export 
vhost-user-blk,node-name=drive0,id=export0,unix-socket=/tmp/vhost-user-blk.sock

Note that unix-socket is optional because we may wish to accept chardevs
too in the future.

Markus noted that supported address families are not explicit in the
QAPI schema. It is unlikely that support for more address families will
be added since file descriptor passing is required and few address
families support it. If a new address family needs to be added, then the
QAPI 'features' syntax can be used to advertize them.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20200924151549.913737-12-stefanha@redhat.com
[Skip test on big-endian host architectures because this device doesn't
support them yet (as already mentioned in a code comment).
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: 80a06cc52bcf7a9771407306991a816834c04941
      
https://github.com/qemu/qemu/commit/80a06cc52bcf7a9771407306991a816834c04941
  Author: Stefan Hajnoczi <stefanha@redhat.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M MAINTAINERS
    M block/export/vhost-user-blk-server.c
    A include/qemu/vhost-user-server.h
    M util/vhost-user-server.c
    R util/vhost-user-server.h

  Log Message:
  -----------
  util/vhost-user-server: move header to include/

Headers used by other subsystems are located in include/. Also add the
vhost-user-server and vhost-user-blk-server headers to MAINTAINERS.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20200924151549.913737-13-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: 3a213f83d99687e6648702421675a922ea71d593
      
https://github.com/qemu/qemu/commit/3a213f83d99687e6648702421675a922ea71d593
  Author: Stefan Hajnoczi <stefanha@redhat.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M block/export/export.c
    M block/export/meson.build
    M contrib/libvhost-user/meson.build
    M meson.build
    M util/meson.build

  Log Message:
  -----------
  util/vhost-user-server: use static library in meson.build

Don't compile contrib/libvhost-user/libvhost-user.c again. Instead build
the static library once and then reuse it throughout QEMU.

Also switch from CONFIG_LINUX to CONFIG_VHOST_USER, which is what the
vhost-user tools (vhost-user-gpu, etc) do.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20200924151549.913737-14-stefanha@redhat.com
[Added CONFIG_LINUX again because libvhost-user doesn't build on macOS.
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: 4fb9071f65aa871bb46d665899ee328a94d9f781
      
https://github.com/qemu/qemu/commit/4fb9071f65aa871bb46d665899ee328a94d9f781
  Author: Stefan Hajnoczi <stefanha@redhat.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M meson.build
    M storage-daemon/meson.build

  Log Message:
  -----------
  qemu-storage-daemon: avoid compiling blockdev_ss twice

Introduce libblkdev.fa to avoid recompiling blockdev_ss twice.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20200929125516.186715-3-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: cbc20bfb8fc293333abbef6aacf052f5f8d72f98
      
https://github.com/qemu/qemu/commit/cbc20bfb8fc293333abbef6aacf052f5f8d72f98
  Author: Stefan Hajnoczi <stefanha@redhat.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M block/export/meson.build
    M meson.build
    M nbd/meson.build
    M qemu-nbd.c
    A stubs/blk-exp-close-all.c
    M stubs/meson.build

  Log Message:
  -----------
  block: move block exports to libblockdev

Block exports are used by softmmu, qemu-storage-daemon, and qemu-nbd.
They are not used by other programs and are not otherwise needed in
libblock.

Undo the recent move of blockdev-nbd.c from blockdev_ss into block_ss.
Since bdrv_close_all() (libblock) calls blk_exp_close_all()
(libblockdev) a stub function is required..

Make qemu-nbd.c use signal handling utility functions instead of
duplicating the code. This helps because os-posix.c is in libblockdev
and it depends on a qemu_system_killed() symbol that qemu-nbd.c lacks.
Once we use the signal handling utility functions we also end up
providing the necessary symbol.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20200929125516.186715-4-stefanha@redhat.com
[Fixed s/ndb/nbd/ typo in commit description as suggested by Eric Blake
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: f51d23c80af73c95e0ce703ad06a300f1b3d63ef
      
https://github.com/qemu/qemu/commit/f51d23c80af73c95e0ce703ad06a300f1b3d63ef
  Author: Stefan Hajnoczi <stefanha@redhat.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M block/export/export.c
    M block/export/vhost-user-blk-server.c
    M nbd/server.c
    M qapi/block-export.json

  Log Message:
  -----------
  block/export: add iothread and fixed-iothread options

Make it possible to specify the iothread where the export will run. By
default the block node can be moved to other AioContexts later and the
export will follow. The fixed-iothread option forces strict behavior
that prevents changing AioContext while the export is active. See the
QAPI docs for details.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20200929125516.186715-5-stefanha@redhat.com
[Fix stray '#' character in block-export.json and add missing "(since:
5.2)" as suggested by Eric Blake.
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: d9b495f9c6a943c9bbd50f7469efb645c23009c3
      
https://github.com/qemu/qemu/commit/d9b495f9c6a943c9bbd50f7469efb645c23009c3
  Author: Stefan Hajnoczi <stefanha@redhat.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M block/export/vhost-user-blk-server.c
    M qapi/block-export.json

  Log Message:
  -----------
  block/export: add vhost-user-blk multi-queue support

Allow the number of queues to be configured using --export
vhost-user-blk,num-queues=N. This setting should match the QEMU --device
vhost-user-blk-pci,num-queues=N setting but QEMU vhost-user-blk.c lowers
its own value if the vhost-user-blk backend offers fewer queues than
QEMU.

The vhost-user-blk-server.c code is already capable of multi-queue. All
virtqueue processing runs in the same AioContext. No new locking is
needed.

Add the num-queues=N option and set the VIRTIO_BLK_F_MQ feature bit.
Note that the feature bit only announces the presence of the num_queues
configuration space field. It does not promise that there is more than 1
virtqueue, so we can set it unconditionally.

I tested multi-queue by running a random read fio test with numjobs=4 on
an -smp 4 guest. After the benchmark finished the guest /proc/interrupts
file showed activity on all 4 virtio-blk MSI-X. The /sys/block/vda/mq/
directory shows that Linux blk-mq has 4 queues configured.

An automated test is included in the next commit.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20201001144604.559733-2-stefanha@redhat.com
[Fixed accidental tab characters as suggested by Markus Armbruster
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: 67c095c8b8097748edf6c4521d8eb6cfc4d2a0ef
      
https://github.com/qemu/qemu/commit/67c095c8b8097748edf6c4521d8eb6cfc4d2a0ef
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M block/io.c
    M block/qcow2.c

  Log Message:
  -----------
  block/io: fix bdrv_co_block_status_above

bdrv_co_block_status_above has several design problems with handling
short backing files:

1. With want_zeros=true, it may return ret with BDRV_BLOCK_ZERO but
without BDRV_BLOCK_ALLOCATED flag, when actually short backing file
which produces these after-EOF zeros is inside requested backing
sequence.

2. With want_zero=false, it may return pnum=0 prior to actual EOF,
because of EOF of short backing file.

Fix these things, making logic about short backing files clearer.

With fixed bdrv_block_status_above we also have to improve is_zero in
qcow2 code, otherwise iotest 154 will fail, because with this patch we
stop to merge zeros of different types (produced by fully unallocated
in the whole backing chain regions vs produced by short backing files).

Note also, that this patch leaves for another day the general problem
around block-status: misuse of BDRV_BLOCK_ALLOCATED as is-fs-allocated
vs go-to-backing.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20200924194003.22080-2-vsementsov@virtuozzo.com
[Fix s/comes/come/ as suggested by Eric Blake
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: 3555a43261ac0010a34377d0db804d194145511c
      
https://github.com/qemu/qemu/commit/3555a43261ac0010a34377d0db804d194145511c
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M block/coroutines.h
    M block/io.c

  Log Message:
  -----------
  block/io: bdrv_common_block_status_above: support include_base

In order to reuse bdrv_common_block_status_above in
bdrv_is_allocated_above, let's support include_base parameter.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20200924194003.22080-3-vsementsov@virtuozzo.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: 624f27bbe9615ba7a763ccc4632a4df5f0721fd0
      
https://github.com/qemu/qemu/commit/624f27bbe9615ba7a763ccc4632a4df5f0721fd0
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M block/io.c

  Log Message:
  -----------
  block/io: bdrv_common_block_status_above: support bs == base

We are going to reuse bdrv_common_block_status_above in
bdrv_is_allocated_above. bdrv_is_allocated_above may be called with
include_base == false and still bs == base (for ex. from img_rebase()).

So, support this corner case.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-id: 20200924194003.22080-4-vsementsov@virtuozzo.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: 7e7e510077f386b7ec286c52f65b4552568bc65e
      
https://github.com/qemu/qemu/commit/7e7e510077f386b7ec286c52f65b4552568bc65e
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M block/io.c

  Log Message:
  -----------
  block/io: fix bdrv_is_allocated_above

bdrv_is_allocated_above wrongly handles short backing files: it reports
after-EOF space as UNALLOCATED which is wrong, as on read the data is
generated on the level of short backing file (if all overlays have
unallocated areas at that place).

Reusing bdrv_common_block_status_above fixes the issue and unifies code
path.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-id: 20200924194003.22080-5-vsementsov@virtuozzo.com
[Fix s/has/have/ as suggested by Eric Blake. Fix s/area/areas/.
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: 4f193168af097ef664287f4c801724f063a60fc2
      
https://github.com/qemu/qemu/commit/4f193168af097ef664287f4c801724f063a60fc2
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-10-23 (Fri, 23 Oct 2020)

  Changed paths:
    M tests/qemu-iotests/274
    M tests/qemu-iotests/274.out

  Log Message:
  -----------
  iotests: add commit top->base cases to 274

These cases are fixed by previous patches around block_status and
is_allocated.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-id: 20200924194003.22080-6-vsementsov@virtuozzo.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>


  Commit: a5fac424c76d6401ecde4ecb7d846e656d0d6e89
      
https://github.com/qemu/qemu/commit/a5fac424c76d6401ecde4ecb7d846e656d0d6e89
  Author: Peter Maydell <peter.maydell@linaro.org>
  Date:   2020-10-26 (Mon, 26 Oct 2020)

  Changed paths:
    M MAINTAINERS
    M block/coroutines.h
    M block/export/export.c
    M block/export/meson.build
    A block/export/vhost-user-blk-server.c
    A block/export/vhost-user-blk-server.h
    M block/io.c
    M block/nvme.c
    M block/qcow2.c
    M contrib/libvhost-user/libvhost-user-glib.c
    M contrib/libvhost-user/libvhost-user.c
    M contrib/libvhost-user/libvhost-user.h
    M contrib/libvhost-user/meson.build
    M hw/core/qdev-properties-system.c
    A include/qemu/vhost-user-server.h
    M meson.build
    M nbd/meson.build
    M nbd/server.c
    M qapi/block-core.json
    M qapi/block-export.json
    M qemu-nbd.c
    M softmmu/vl.c
    M storage-daemon/meson.build
    A stubs/blk-exp-close-all.c
    M stubs/meson.build
    M tests/qemu-iotests/274
    M tests/qemu-iotests/274.out
    M tests/vhost-user-bridge.c
    M tools/virtiofsd/fuse_virtio.c
    A util/block-helpers.c
    A util/block-helpers.h
    M util/meson.build
    A util/vhost-user-server.c

  Log Message:
  -----------
  Merge remote-tracking branch 
'remotes/stefanha-gitlab/tags/block-pull-request' into staging

Pull request

v3:
 * Fix linker error on FreeBSD and other !CONFIG_LINUX host OSes [Peter]

# gpg: Signature made Fri 23 Oct 2020 16:21:19 BST
# gpg:                using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha-gitlab/tags/block-pull-request: (28 commits)
  iotests: add commit top->base cases to 274
  block/io: fix bdrv_is_allocated_above
  block/io: bdrv_common_block_status_above: support bs == base
  block/io: bdrv_common_block_status_above: support include_base
  block/io: fix bdrv_co_block_status_above
  block/export: add vhost-user-blk multi-queue support
  block/export: add iothread and fixed-iothread options
  block: move block exports to libblockdev
  qemu-storage-daemon: avoid compiling blockdev_ss twice
  util/vhost-user-server: use static library in meson.build
  util/vhost-user-server: move header to include/
  block/export: convert vhost-user-blk server to block export API
  block/export: report flush errors
  util/vhost-user-server: rework vu_client_trip() coroutine lifecycle
  util/vhost-user-server: check EOF when reading payload
  util/vhost-user-server: fix memory leak in vu_message_read()
  util/vhost-user-server: drop unused DevicePanicNotifier
  block/export: consolidate request structs into VuBlockReq
  util/vhost-user-server: drop unnecessary watch deletion
  util/vhost-user-server: drop unnecessary QOM cast
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>


Compare: https://github.com/qemu/qemu/compare/4c5b97bfd0dd...a5fac424c76d