[Qemu-commits] [qemu/qemu] 15d40e: qemu-storage-daemon: Enable object-add

[Peter Maydell]

  Branch: refs/heads/staging
  Home:   https://github.com/qemu/qemu
  Commit: 15d40e9204eb3d89577187f117a1dde2237bdc4d
      
https://github.com/qemu/qemu/commit/15d40e9204eb3d89577187f117a1dde2237bdc4d
  Author: Kevin Wolf <kwolf@redhat.com>
  Date:   2021-02-15 (Mon, 15 Feb 2021)

  Changed paths:
    M storage-daemon/qemu-storage-daemon.c

  Log Message:
  -----------
  qemu-storage-daemon: Enable object-add

As we don't have a fully QAPIfied version of object-add yet and it still
has 'gen': false in the schema, it needs to be registered explicitly in
init_qmp_commands() to be available for users.

Fixes: 2af282ec51a27116d0402cab237b8970800f870c
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20210204072137.19663-1-kwolf@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>


  Commit: b34cdf32256e59e0183fe0f7ee5b5f4865d9ccc0
      
https://github.com/qemu/qemu/commit/b34cdf32256e59e0183fe0f7ee5b5f4865d9ccc0
  Author: Max Reitz <mreitz@redhat.com>
  Date:   2021-02-15 (Mon, 15 Feb 2021)

  Changed paths:
    M tests/qemu-iotests/common.rc

  Log Message:
  -----------
  iotests: Consistent $IMGOPTS boundary matching

To disallow certain refcount_bits values, some _unsupported_imgopts
invocations look like "refcount_bits=1[^0-9]", i.e. they match an
integer boundary with [^0-9].  This expression does not match the end of
the string, though, so it breaks down when refcount_bits is the last
option (which it tends to be after the rewrite of the check script in
Python).

Those invocations could use \b or \> instead, but those are not
portable.  They could use something like \([^0-9]\|$\), but that would
be cumbersome.  To make it simple and keep the existing invocations
working, just let _unsupported_imgopts match the regex against $IMGOPTS
plus a trailing space.

Suggested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210210095128.22732-1-mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>


  Commit: 076d467aacdf6dc5d01e2e61740b1795f2aec2f6
      
https://github.com/qemu/qemu/commit/076d467aacdf6dc5d01e2e61740b1795f2aec2f6
  Author: Michael Qiu <qiudayu@huayun.com>
  Date:   2021-02-15 (Mon, 15 Feb 2021)

  Changed paths:
    M blockjob.c

  Log Message:
  -----------
  blockjob: Fix crash with IOthread when block commit after snapshot

Currently, if guest has workloads, IO thread will acquire aio_context
lock before do io_submit, it leads to segmentfault when do block commit
after snapshot. Just like below:

Program received signal SIGSEGV, Segmentation fault.

[Switching to Thread 0x7f7c7d91f700 (LWP 99907)]
0x00005576d0f65aab in bdrv_mirror_top_pwritev at ../block/mirror.c:1437
1437    ../block/mirror.c: No such file or directory.
(gdb) p s->job
$17 = (MirrorBlockJob *) 0x0
(gdb) p s->stop
$18 = false

Call trace of IO thread:
0  0x00005576d0f65aab in bdrv_mirror_top_pwritev at ../block/mirror.c:1437
1  0x00005576d0f7f3ab in bdrv_driver_pwritev at ../block/io.c:1174
2  0x00005576d0f8139d in bdrv_aligned_pwritev at ../block/io.c:1988
3  0x00005576d0f81b65 in bdrv_co_pwritev_part at ../block/io.c:2156
4  0x00005576d0f8e6b7 in blk_do_pwritev_part at ../block/block-backend.c:1260
5  0x00005576d0f8e84d in blk_aio_write_entry at ../block/block-backend.c:1476
...

Switch to qemu main thread:
0  0x00007f903be704ed in __lll_lock_wait at
/lib/../lib64/libpthread.so.0
1  0x00007f903be6bde6 in _L_lock_941 at /lib/../lib64/libpthread.so.0
2  0x00007f903be6bcdf in pthread_mutex_lock at
/lib/../lib64/libpthread.so.0
3  0x0000564b21456889 in qemu_mutex_lock_impl at
../util/qemu-thread-posix.c:79
4  0x0000564b213af8a5 in block_job_add_bdrv at ../blockjob.c:224
5  0x0000564b213b00ad in block_job_create at ../blockjob.c:440
6  0x0000564b21357c0a in mirror_start_job at ../block/mirror.c:1622
7  0x0000564b2135a9af in commit_active_start at ../block/mirror.c:1867
8  0x0000564b2133d132 in qmp_block_commit at ../blockdev.c:2768
9  0x0000564b2141fef3 in qmp_marshal_block_commit at
qapi/qapi-commands-block-core.c:346
10 0x0000564b214503c9 in do_qmp_dispatch_bh at
../qapi/qmp-dispatch.c:110
11 0x0000564b21451996 in aio_bh_poll at ../util/async.c:164
12 0x0000564b2146018e in aio_dispatch at ../util/aio-posix.c:381
13 0x0000564b2145187e in aio_ctx_dispatch at ../util/async.c:306
14 0x00007f9040239049 in g_main_context_dispatch at
/lib/../lib64/libglib-2.0.so.0
15 0x0000564b21447368 in main_loop_wait at ../util/main-loop.c:232
16 0x0000564b21447368 in main_loop_wait at ../util/main-loop.c:255
17 0x0000564b21447368 in main_loop_wait at ../util/main-loop.c:531
18 0x0000564b212304e1 in qemu_main_loop at ../softmmu/runstate.c:721
19 0x0000564b20f7975e in main at ../softmmu/main.c:50

In IO thread when do bdrv_mirror_top_pwritev, the job is NULL, and stop field
is false, this means the MirrorBDSOpaque "s" object has not been initialized
yet, and this object is initialized by block_job_create(), but the initialize
process is stuck in acquiring the lock.

In this situation, IO thread come to bdrv_mirror_top_pwritev(),which means that
mirror-top node is already inserted into block graph, but its bs->opaque->job
is not initialized.

The root cause is that qemu main thread do release/acquire when hold the lock,
at the same time, IO thread get the lock after release stage, and the crash
occured.

Actually, in this situation, job->job.aio_context will not equal to
qemu_get_aio_context(), and will be the same as bs->aio_context,
thus, no need to release the lock, becasue bdrv_root_attach_child()
will not change the context.

This patch fix this issue.

Fixes: 132ada80 "block: Adjust AioContexts when attaching nodes"

Signed-off-by: Michael Qiu <qiudayu@huayun.com>
Message-Id: <20210203024059.52683-1-08005325@163.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>


  Commit: 26941eb4ca485150494379cf8f0751f426208a5b
      
https://github.com/qemu/qemu/commit/26941eb4ca485150494379cf8f0751f426208a5b
  Author: Alexander Bulekov <alxndr@bu.edu>
  Date:   2021-02-15 (Mon, 15 Feb 2021)

  Changed paths:
    M hw/ide/ahci.c

  Log Message:
  -----------
  hw/ide/ahci: map cmd_fis as DMA_DIRECTION_TO_DEVICE

cmd_fis is mapped as DMA_DIRECTION_FROM_DEVICE, however, it is read
from, and not written to anywhere. Fix the DMA_DIRECTION and mark
cmd_fis as read-only in the code.

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20210119164051.89268-1-alxndr@bu.edu>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>


  Commit: 7f22201ac85186dd476d5ff8da278ab603df72b6
      
https://github.com/qemu/qemu/commit/7f22201ac85186dd476d5ff8da278ab603df72b6
  Author: Roger Pau Monne <roger.pau@citrix.com>
  Date:   2021-02-15 (Mon, 15 Feb 2021)

  Changed paths:
    M hw/block/xen-block.c

  Log Message:
  -----------
  xen-block: fix reporting of discard feature

Linux blkfront expects both "discard-granularity" and
"discard-alignment" present on xenbus in order to properly enable the
feature, not exposing "discard-alignment" left some Linux blkfront
versions with a broken discard setup. This has also been addressed in
Linux with:

https://lore.kernel.org/lkml/20210118151528.81668-1-roger.pau@citrix.com/T/#u

Fix QEMU to report a "discard-alignment" of 0, in order for it to work
with older Linux frontends.

Reported-by: Arthur Borsboom <arthurborsboom@gmail.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Message-Id: <20210118153330.82324-1-roger.pau@citrix.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>


  Commit: f8b467b443fb3e1c531e0a8b3201104277401ee5
      
https://github.com/qemu/qemu/commit/f8b467b443fb3e1c531e0a8b3201104277401ee5
  Author: Thomas Huth <thuth@redhat.com>
  Date:   2021-02-15 (Mon, 15 Feb 2021)

  Changed paths:
    M tests/qemu-iotests/259

  Log Message:
  -----------
  tests/qemu-iotests: Remove test 259 from the "auto" group

Tests in the "auto" group should support qcow2 so that they can
be run during "make check-block". Test 259 only supports "raw", so
it currently always gets skipped when running "make check-block".
Let's skip this unnecessary step and remove it from the auto group.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210215103835.1129145-1-thuth@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>


  Commit: dcb6699512b6235eac383fc84e8e27302e08d7e3
      
https://github.com/qemu/qemu/commit/dcb6699512b6235eac383fc84e8e27302e08d7e3
  Author: Maxim Levitsky <mlevitsk@redhat.com>
  Date:   2021-02-15 (Mon, 15 Feb 2021)

  Changed paths:
    M block/crypto.c

  Log Message:
  -----------
  crypto: luks: Fix tiny memory leak

When the underlying block device doesn't support the
bdrv_co_delete_file interface, an 'Error' object was leaked.

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201217170904.946013-2-mlevitsk@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>


  Commit: a890f08e586fd81946c51dae6a878d64486b1364
      
https://github.com/qemu/qemu/commit/a890f08e586fd81946c51dae6a878d64486b1364
  Author: Maxim Levitsky <mlevitsk@redhat.com>
  Date:   2021-02-15 (Mon, 15 Feb 2021)

  Changed paths:
    M block.c
    M block/crypto.c
    M include/block/block.h

  Log Message:
  -----------
  block: add bdrv_co_delete_file_noerr

This function wraps bdrv_co_delete_file for the common case of removing a file,
which was just created by format driver, on an error condition.

It hides the -ENOTSUPP error, and reports all other errors otherwise.

Use it in luks driver

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-Id: <20201217170904.946013-3-mlevitsk@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>


  Commit: 6094cbeb72117204f3302a4581415ee1dc33a879
      
https://github.com/qemu/qemu/commit/6094cbeb72117204f3302a4581415ee1dc33a879
  Author: Maxim Levitsky <mlevitsk@redhat.com>
  Date:   2021-02-15 (Mon, 15 Feb 2021)

  Changed paths:
    M block/qcow2.c

  Log Message:
  -----------
  block: qcow2: remove the created file on initialization error

If the qcow initialization fails, we should remove the file if it was
already created, to avoid leaving stale files around.

We already do this for luks raw images.

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-Id: <20201217170904.946013-4-mlevitsk@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>


  Commit: c81219a7dd36a815bd85beed9932fc973d4f5d51
      
https://github.com/qemu/qemu/commit/c81219a7dd36a815bd85beed9932fc973d4f5d51
  Author: Kevin Wolf <kwolf@redhat.com>
  Date:   2021-02-15 (Mon, 15 Feb 2021)

  Changed paths:
    M monitor/monitor.c

  Log Message:
  -----------
  monitor: Fix assertion failure on shutdown

Commit 357bda95 already tried to fix the order in monitor_cleanup() by
moving shutdown of the dispatcher coroutine further to the start.
However, it didn't go far enough:

iothread_stop() makes sure that all pending work (bottom halves) in the
AioContext of the monitor iothread is completed. iothread_destroy()
depends on this and fails an assertion if there is still a pending BH.

While the dispatcher coroutine is running, it will try to resume the
monitor after taking a request out of the queue, which involves a BH.
The dispatcher is run until it terminates in the AIO_WAIT_WHILE() loop.
However, adding new BHs between iothread_stop() and iothread_destroy()
is forbidden.

Fix this by stopping the dispatcher first before shutting down the other
parts of the monitor. This means we can now receive requests that aren't
handled any more when QEMU is shutting down, but this is unlikely to be
a problem for QMP clients.

Fixes: 357bda9590784ff75803d52de43150d4107ed98e
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20210212172028.288825-2-kwolf@redhat.com>
Tested-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>


  Commit: b248e61652e20c3353af4b0ccb90f17d76f4db21
      
https://github.com/qemu/qemu/commit/b248e61652e20c3353af4b0ccb90f17d76f4db21
  Author: Kevin Wolf <kwolf@redhat.com>
  Date:   2021-02-15 (Mon, 15 Feb 2021)

  Changed paths:
    M monitor/qmp.c

  Log Message:
  -----------
  monitor/qmp: Stop processing requests when shutdown is requested

Before this patch, monitor_qmp_dispatcher_co() used to check whether
shutdown is requested only when it would have to wait for new requests.
If there were still some queued requests, it would try to execute all of
them before shutting down.

This can be surprising when the queued QMP commands take long or hang
because Ctrl-C may not actually exit QEMU as soon as possible.

Change monitor_qmp_dispatcher_co() so that it additionally checks
whether shutdown is request before it gets a new request from the queue.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20210212172028.288825-3-kwolf@redhat.com>
Tested-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>


  Commit: 8ba4bca570ace1e60614a0808631a517cf5df67a
      
https://github.com/qemu/qemu/commit/8ba4bca570ace1e60614a0808631a517cf5df67a
  Author: Peter Maydell <peter.maydell@linaro.org>
  Date:   2021-02-15 (Mon, 15 Feb 2021)

  Changed paths:
    M block.c
    M block/crypto.c
    M block/qcow2.c
    M blockjob.c
    M hw/block/xen-block.c
    M hw/ide/ahci.c
    M include/block/block.h
    M monitor/monitor.c
    M monitor/qmp.c
    M storage-daemon/qemu-storage-daemon.c
    M tests/qemu-iotests/259
    M tests/qemu-iotests/common.rc

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- qemu-storage-daemon: Enable object-add
- blockjob: Fix crash with IOthread when block commit after snapshot
- monitor: Shutdown fixes
- xen-block: fix reporting of discard feature
- qcow2: Remove half-initialised image file after failed image creation
- ahci: Fix DMA direction
- iotests fixes

# gpg: Signature made Mon 15 Feb 2021 14:58:47 GMT
# gpg:                using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg:                issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
  monitor/qmp: Stop processing requests when shutdown is requested
  monitor: Fix assertion failure on shutdown
  block: qcow2: remove the created file on initialization error
  block: add bdrv_co_delete_file_noerr
  crypto: luks: Fix tiny memory leak
  tests/qemu-iotests: Remove test 259 from the "auto" group
  xen-block: fix reporting of discard feature
  hw/ide/ahci: map cmd_fis as DMA_DIRECTION_TO_DEVICE
  blockjob: Fix crash with IOthread when block commit after snapshot
  iotests: Consistent $IMGOPTS boundary matching
  qemu-storage-daemon: Enable object-add

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>


Compare: https://github.com/qemu/qemu/compare/35f15acbc15d...8ba4bca570ac