qemu-commits
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Qemu-commits] [qemu/qemu] b4af1d: iotests/291: Filter irrelevant parts


From: Peter Maydell
Subject: [Qemu-commits] [qemu/qemu] b4af1d: iotests/291: Filter irrelevant parts of img-info
Date: Mon, 02 Nov 2020 01:53:56 -0800

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: b4af1df08f9433d858ec01aff4ec9f06cb28128b
      
https://github.com/qemu/qemu/commit/b4af1df08f9433d858ec01aff4ec9f06cb28128b
  Author: Max Reitz <mreitz@redhat.com>
  Date:   2020-10-30 (Fri, 30 Oct 2020)

  Changed paths:
    M tests/qemu-iotests/291
    M tests/qemu-iotests/291.out

  Log Message:
  -----------
  iotests/291: Filter irrelevant parts of img-info

We need to let _img_info emit the format-specific information so we get
the list of bitmaps we want, but we do not need anything but the
bitmaps.  So filter out everything that is irrelevant to us.  (Ideally,
this would be a generalized function in common.filters that takes a list
of things to keep, but that would require implementing an anti-bitmap
filter, which would be hard, and which we do not need here.  So that is
why this function is just a local hack.)

This lets 291 pass with qcow2 options like refcount_bits or data_file
again.

Fixes: 14f16bf9474c860ecc127a66a86961942319f7af
       ("qemu-img: Support bitmap --merge into backing image")
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201027164416.144115-2-mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>


  Commit: cb7223ac33fa35976d64f31145122965c1ba332d
      
https://github.com/qemu/qemu/commit/cb7223ac33fa35976d64f31145122965c1ba332d
  Author: Max Reitz <mreitz@redhat.com>
  Date:   2020-10-30 (Fri, 30 Oct 2020)

  Changed paths:
    M tests/qemu-iotests/291

  Log Message:
  -----------
  iotests/291: Stop NBD server

nbd_server_start_unix_socket() includes an implicit nbd_server_stop(),
but we still need an explicit one at the end of the test (where there
follows no next nbd_server_start_unix_socket()), or qemu-nbd will linger
until the test exits.

This will become important when enabling this test to run on FUSE
exports, because then the export (which is the image used by qemu-nbd)
will go away before qemu-nbd exits, which will lead to qemu-nbd
complaining that it cannot flush the bitmaps in the image.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201027164416.144115-3-mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>


  Commit: 159f8442382cff6ef6d858dbf02f03f0cc95b4a7
      
https://github.com/qemu/qemu/commit/159f8442382cff6ef6d858dbf02f03f0cc95b4a7
  Author: Eric Blake <eblake@redhat.com>
  Date:   2020-10-30 (Fri, 30 Oct 2020)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: Simplify QAPI_LIST_ADD

There is no need to rely on the verbosity of the gcc/clang compiler
extension of g_new(typeof(X), 1) when we can instead use the standard
g_malloc(sizeof(X)).  In general, we like g_new over g_malloc for
returning type X rather than void* to let the compiler catch more
potential typing mistakes, but in this particular macro, our other use
of typeof on the same line already ensures we are getting correct
results.

Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20201027050556.269064-2-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>


  Commit: 9812e7125b83ba6e4645237150ca5f61bf66197f
      
https://github.com/qemu/qemu/commit/9812e7125b83ba6e4645237150ca5f61bf66197f
  Author: Eric Blake <eblake@redhat.com>
  Date:   2020-10-30 (Fri, 30 Oct 2020)

  Changed paths:
    M block.c
    M include/qapi/util.h

  Log Message:
  -----------
  qapi: Add QAPI_LIST_PREPEND() macro

block.c has a useful macro QAPI_LIST_ADD() for inserting at the front
of any QAPI-generated list; move it from block.c to qapi/util.h so
more places can use it, including one earlier place in block.c, and
rename it to something more obvious (since we also have a lot of
places that append, rather than prepend, to a list).

There are many more places in the codebase that can benefit from using
the macro, but converting them will be left to later patches.

In theory, all QAPI list types are child classes of GenericList; but
in practice, that relationship is not explicitly spelled out in the C
type declarations (rather, it is something that happens implicitly due
to C compatible layouts), and the macro does not actually depend on
the GenericList type.  We considered moving GenericList from visitor.h
into util.h to group related code; however, such a move would be
awkward if we do not also move GenericAlternate.  Unfortunately,
moving GenericAlternate would introduce its own problems of
declaration circularity (qapi-builtin-types.h needs a complete
definition of QEnumLookup from util.h, but GenericAlternate needs a
complete definition of QType from qapi-builtin-types.h).

Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20201027050556.269064-3-eblake@redhat.com>
[eblake: s/ADD/PREPEND/ per suggestion by Markus]


  Commit: 8675cbd68be7da35a61e391fbb5b95ac67bc7d72
      
https://github.com/qemu/qemu/commit/8675cbd68be7da35a61e391fbb5b95ac67bc7d72
  Author: Eric Blake <eblake@redhat.com>
  Date:   2020-10-30 (Fri, 30 Oct 2020)

  Changed paths:
    M blockdev-nbd.c

  Log Message:
  -----------
  nbd: Utilize QAPI_CLONE for type conversion

Rather than open-coding the translation from the deprecated
NbdServerAddOptions type to the preferred BlockExportOptionsNbd, it's
better to utilize QAPI_CLONE_MEMBERS.  This solves a couple of issues:
first, if we do any more refactoring of the base type (which an
upcoming patch plans to do), we don't have to revisit the open-coding.
Second, our assignment to arg->name is fishy: the generated QAPI code
for qapi_free_NbdServerAddOptions does not visit arg->name if
arg->has_name is false, but if it DID visit it, we would have
introduced a double-free situation when arg is finally freed.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20201027050556.269064-4-eblake@redhat.com>


  Commit: cbad81cef8cc7b220f04600997ea29d7302bae00
      
https://github.com/qemu/qemu/commit/cbad81cef8cc7b220f04600997ea29d7302bae00
  Author: Eric Blake <eblake@redhat.com>
  Date:   2020-10-30 (Fri, 30 Oct 2020)

  Changed paths:
    M blockdev-nbd.c
    M docs/system/deprecated.rst
    M nbd/server.c
    M qapi/block-export.json
    M qemu-nbd.c

  Log Message:
  -----------
  nbd: Update qapi to support exporting multiple bitmaps

Since 'block-export-add' is new to 5.2, we can still tweak the
interface; there, allowing 'bitmaps':['str'] is nicer than
'bitmap':'str'.  This wires up the qapi and qemu-nbd changes to permit
passing multiple bitmaps as distinct metadata contexts that the NBD
client may request, but the actual support for more than one will
require a further patch to the server.

Note that there are no changes made to the existing deprecated
'nbd-server-add' command; this required splitting the QAPI type
BlockExportOptionsNbd, which fortunately does not affect QMP
introspection.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20201027050556.269064-5-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>


  Commit: 02e87e3b1c77422b71855102b9cb884e4c7823f7
      
https://github.com/qemu/qemu/commit/02e87e3b1c77422b71855102b9cb884e4c7823f7
  Author: Eric Blake <eblake@redhat.com>
  Date:   2020-10-30 (Fri, 30 Oct 2020)

  Changed paths:
    M nbd/server.c

  Log Message:
  -----------
  nbd: Simplify qemu bitmap context name

Each dirty bitmap already knows its name; by reducing the scope of the
places where we construct "qemu:dirty-bitmap:NAME" strings, tracking
the name is more localized, and there are fewer per-export fields to
worry about.  This in turn will make it easier for an upcoming patch
to export more than one bitmap at once.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201027050556.269064-6-eblake@redhat.com>


  Commit: 47ec485e8d5dba978126d31b9484d717ac1222ba
      
https://github.com/qemu/qemu/commit/47ec485e8d5dba978126d31b9484d717ac1222ba
  Author: Eric Blake <eblake@redhat.com>
  Date:   2020-10-30 (Fri, 30 Oct 2020)

  Changed paths:
    M nbd/server.c

  Log Message:
  -----------
  nbd: Refactor counting of metadata contexts

Rather than open-code the count of negotiated contexts at several
sites, embed it directly into the struct.  This will make it easier
for upcoming commits to support even more simultaneous contexts.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20201027050556.269064-7-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>


  Commit: 3b1f244c59b7045680e615d50dc444a316abd891
      
https://github.com/qemu/qemu/commit/3b1f244c59b7045680e615d50dc444a316abd891
  Author: Eric Blake <eblake@redhat.com>
  Date:   2020-10-30 (Fri, 30 Oct 2020)

  Changed paths:
    M nbd/server.c
    M tests/qemu-iotests/291

  Log Message:
  -----------
  nbd: Allow export of multiple bitmaps for one device

With this, 'qemu-nbd -B b0 -B b1 -f qcow2 img.qcow2' can let you sniff
out multiple bitmaps from one server.  qemu-img as client can still
only read one bitmap per client connection, but other NBD clients
(hello libnbd) can now read multiple bitmaps in a single pass.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201027050556.269064-8-eblake@redhat.com>


  Commit: a92b1b065eed385d9077be735eb8e92f5a6e150a
      
https://github.com/qemu/qemu/commit/a92b1b065eed385d9077be735eb8e92f5a6e150a
  Author: Eric Blake <eblake@redhat.com>
  Date:   2020-10-30 (Fri, 30 Oct 2020)

  Changed paths:
    M block/commit.c
    M block/coroutines.h
    M block/io.c
    M block/mirror.c
    M block/stream.c

  Log Message:
  -----------
  block: Return depth level during bdrv_is_allocated_above

When checking for allocation across a chain, it's already easy to
count the depth within the chain at which the allocation is found.
Instead of throwing that information away, return it to the caller.
Existing callers only cared about allocated/non-allocated, but having
a depth available will be used by NBD in the next patch.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20201027050556.269064-9-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: rebase to master]
Signed-off-by: Eric Blake <eblake@redhat.com>


  Commit: 71719cd57fc02ddfd91a4a3ca3f469bfb4d221bc
      
https://github.com/qemu/qemu/commit/71719cd57fc02ddfd91a4a3ca3f469bfb4d221bc
  Author: Eric Blake <eblake@redhat.com>
  Date:   2020-10-30 (Fri, 30 Oct 2020)

  Changed paths:
    M docs/interop/nbd.txt
    M include/block/nbd.h
    M nbd/server.c

  Log Message:
  -----------
  nbd: Add new qemu:allocation-depth metadata context

'qemu-img map' provides a way to determine which extents of an image
come from the top layer vs. inherited from a backing chain.  This is
useful information worth exposing over NBD.  There is a proposal to
add a QMP command block-dirty-bitmap-populate which can create a dirty
bitmap that reflects allocation information, at which point the
qemu:dirty-bitmap:NAME metadata context can expose that information
via the creation of a temporary bitmap, but we can shorten the effort
by adding a new qemu:allocation-depth metadata context that does the
same thing without an intermediate bitmap (this patch does not
eliminate the need for that proposal, as it will have other uses as
well).

While documenting things, remember that although the NBD protocol has
NBD_OPT_SET_META_CONTEXT, the rest of its documentation refers to
'metadata context', which is a more apt description of what is
actually being used by NBD_CMD_BLOCK_STATUS: the user is requesting
metadata by passing one or more context names.  So I also touched up
some existing wording to prefer the term 'metadata context' where it
makes sense.

Note that this patch does not actually enable any way to request a
server to enable this context; that will come in the next patch.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20201027050556.269064-10-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>


  Commit: dbc7b01492371e4a54b92d2b6d968f9b863cc794
      
https://github.com/qemu/qemu/commit/dbc7b01492371e4a54b92d2b6d968f9b863cc794
  Author: Eric Blake <eblake@redhat.com>
  Date:   2020-10-30 (Fri, 30 Oct 2020)

  Changed paths:
    M block/nbd.c
    M docs/tools/qemu-nbd.rst
    M nbd/server.c
    M qapi/block-core.json
    M qapi/block-export.json
    M qemu-nbd.c
    A tests/qemu-iotests/309
    A tests/qemu-iotests/309.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  nbd: Add 'qemu-nbd -A' to expose allocation depth

Allow the server to expose an additional metacontext to be requested
by savvy clients.  qemu-nbd adds a new option -A to expose the
qemu:allocation-depth metacontext through NBD_CMD_BLOCK_STATUS; this
can also be set via QMP when using block-export-add.

qemu as client is hacked into viewing the key aspects of this new
context by abusing the already-experimental x-dirty-bitmap option to
collapse all depths greater than 2, which results in a tri-state value
visible in the output of 'qemu-img map --output=json' (yes, that means
x-dirty-bitmap is now a bit of a misnomer, but I didn't feel like
renaming it as it would introduce a needless break of back-compat,
even though we make no compat guarantees with x- members):

unallocated (depth 0) => "zero":false, "data":true
local (depth 1)       => "zero":false, "data":false
backing (depth 2+)    => "zero":true,  "data":true

libnbd as client is probably a nicer way to get at the information
without having to decipher such hacks in qemu as client. ;)

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20201027050556.269064-11-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>


  Commit: 6f2ef80b0ce87d258b4736471a81747da2a7a881
      
https://github.com/qemu/qemu/commit/6f2ef80b0ce87d258b4736471a81747da2a7a881
  Author: Peter Maydell <peter.maydell@linaro.org>
  Date:   2020-11-01 (Sun, 01 Nov 2020)

  Changed paths:
    M block.c
    M block/commit.c
    M block/coroutines.h
    M block/io.c
    M block/mirror.c
    M block/nbd.c
    M block/stream.c
    M blockdev-nbd.c
    M docs/interop/nbd.txt
    M docs/system/deprecated.rst
    M docs/tools/qemu-nbd.rst
    M include/block/nbd.h
    M include/qapi/util.h
    M nbd/server.c
    M qapi/block-core.json
    M qapi/block-export.json
    M qemu-nbd.c
    M tests/qemu-iotests/291
    M tests/qemu-iotests/291.out
    A tests/qemu-iotests/309
    A tests/qemu-iotests/309.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2020-10-27-v2' into 
staging

nbd patches for 2020-10-27

- Tweak the new block-export-add QMP command
- Allow multiple -B options for qemu-nbd
- Add qemu:allocation-depth metadata context as qemu-nbd -A
- Improve iotest use of NBD

# gpg: Signature made Fri 30 Oct 2020 20:22:42 GMT
# gpg:                using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg:                 aka "Eric Blake (Free Software Programmer) 
<ebb9@byu.net>" [full]
# gpg:                 aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2  F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2020-10-27-v2:
  nbd: Add 'qemu-nbd -A' to expose allocation depth
  nbd: Add new qemu:allocation-depth metadata context
  block: Return depth level during bdrv_is_allocated_above
  nbd: Allow export of multiple bitmaps for one device
  nbd: Refactor counting of metadata contexts
  nbd: Simplify qemu bitmap context name
  nbd: Update qapi to support exporting multiple bitmaps
  nbd: Utilize QAPI_CLONE for type conversion
  qapi: Add QAPI_LIST_PREPEND() macro
  block: Simplify QAPI_LIST_ADD
  iotests/291: Stop NBD server
  iotests/291: Filter irrelevant parts of img-info

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>


Compare: https://github.com/qemu/qemu/compare/700d20b49e30...6f2ef80b0ce8



reply via email to

[Prev in Thread] Current Thread [Next in Thread]