[Qemu-commits] [qemu/qemu] f4de0f: blockdev: abort transactions in reverse order

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: f4de0f8c40b70c4c9308b4670e0a6ad9faed0262
      
https://github.com/qemu/qemu/commit/f4de0f8c40b70c4c9308b4670e0a6ad9faed0262
  Author: John Snow <address@hidden>
  Date:   2019-01-14 (Mon, 14 Jan 2019)

  Changed paths:
    M blockdev.c

  Log Message:
  -----------
  blockdev: abort transactions in reverse order

Presently, we abort transactions in the same order they were processed in.
Bitmap commands, though, attempt to restore backup data structures on abort.

That's not valid, they need to be aborted in reverse chronological order.

Replace the QSIMPLEQ data structure with a QTAILQ one, so we can iterate
in reverse for the abort phase of the transaction.

Signed-off-by: John Snow <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>
[eblake: rebase]
Signed-off-by: Eric Blake <address@hidden>


  Commit: 07d5a8df6a0018d831baef6d50f53f31a06b5a60
      
https://github.com/qemu/qemu/commit/07d5a8df6a0018d831baef6d50f53f31a06b5a60
  Author: John Snow <address@hidden>
  Date:   2019-01-14 (Mon, 14 Jan 2019)

  Changed paths:
    M block/dirty-bitmap.c

  Log Message:
  -----------
  block/dirty-bitmap: remove assertion from restore

When making a backup of a dirty bitmap (for transactions), we want to
restore that backup whether or not the bitmap is enabled.

It is perfectly valid to write into bitmaps that are disabled. It is
only illegitimate for the guest to have done so.

Remove this assertion.

Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Signed-off-by: John Snow <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eric Blake <address@hidden>


  Commit: 360d4e4e9a501d92fb8866ac307d33a25f70c6d1
      
https://github.com/qemu/qemu/commit/360d4e4e9a501d92fb8866ac307d33a25f70c6d1
  Author: John Snow <address@hidden>
  Date:   2019-01-14 (Mon, 14 Jan 2019)

  Changed paths:
    M blockdev.c
    M qapi/block-core.json

  Log Message:
  -----------
  blockdev: n-ary bitmap merge

Especially outside of transactions, it is helpful to provide
all-or-nothing semantics for bitmap merges. This facilitates
the coalescing of multiple bitmaps into a single target for
the "checkpoint" interpretation when assembling bitmaps that
represent arbitrary points in time from component bitmaps.

This is an incompatible change from the preliminary version
of the API.

Signed-off-by: John Snow <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eric Blake <address@hidden>


  Commit: 0e2b7f09837f1e2828d428af1e4ebb61e3b3ea5f
      
https://github.com/qemu/qemu/commit/0e2b7f09837f1e2828d428af1e4ebb61e3b3ea5f
  Author: John Snow <address@hidden>
  Date:   2019-01-14 (Mon, 14 Jan 2019)

  Changed paths:
    M blockdev.c
    M qapi/block-core.json
    M qapi/transaction.json
    M tests/qemu-iotests/223

  Log Message:
  -----------
  block: remove 'x' prefix from experimental bitmap APIs

The 'x' prefix was added because I was uncertain of the direction we'd
take for the libvirt API. With the general approach solidified, I feel
comfortable committing to this API for 4.0.

Signed-off-by: John Snow <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eric Blake <address@hidden>


  Commit: 58ebcb65d8e38fd693bf1e58dd941a4d7a8dfdef
      
https://github.com/qemu/qemu/commit/58ebcb65d8e38fd693bf1e58dd941a4d7a8dfdef
  Author: John Snow <address@hidden>
  Date:   2019-01-14 (Mon, 14 Jan 2019)

  Changed paths:
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests.py: don't abort if IMGKEYSECRET is undefined

Instead of using os.environ[], use .get with a default of empty string
to match the setup in check to allow us to import the iotests module
(for debugging, say) without needing a crafted environment just to
import the module.

Signed-off-by: John Snow <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eric Blake <address@hidden>


  Commit: fa1151f811ffa1973b6980b09f095d0e2ebea08b
      
https://github.com/qemu/qemu/commit/fa1151f811ffa1973b6980b09f095d0e2ebea08b
  Author: John Snow <address@hidden>
  Date:   2019-01-14 (Mon, 14 Jan 2019)

  Changed paths:
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests: add filter_generated_node_ids

To mimic the common filter of the same name, but for the python tests.

Signed-off-by: John Snow <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eric Blake <address@hidden>


  Commit: 0706e87d72b02f28bfa04400388f9c9df1b9c943
      
https://github.com/qemu/qemu/commit/0706e87d72b02f28bfa04400388f9c9df1b9c943
  Author: John Snow <address@hidden>
  Date:   2019-01-14 (Mon, 14 Jan 2019)

  Changed paths:
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests: add qmp recursive sorting function

Python before 3.6 does not sort dictionaries (including kwargs).
Therefore, printing QMP objects involves sorting the keys to have
a predictable ordering in the iotests output. This means that
iotests output will sometimes show arguments in an order not
specified by the test author.

Presently, we accomplish this by using json.dumps' sort_keys argument,
where we only serialize the arguments dictionary, but not the command.

However, if we want to pretty-print QMP objects being sent to the
QEMU process, we need to build the entire command before logging it.
Ordinarily, this would then involve "arguments" being sorted above
"execute", which would necessitate a rather ugly and harder-to-read
change to many iotests outputs.

To facilitate pretty-printing AND maintaining predictable output AND
having "arguments" sort after "execute", add a custom sort function
that takes a dictionary and recursively builds an OrderedDict that
maintains the specific key order we wish to see in iotests output.

The qmp_log function uses this to build a QMP object that keeps
"execute" above "arguments", but sorts all keys and keys in any
subdicts in "arguments" lexicographically to maintain consistent
iotests output, with no incompatible changes to any current test.

Signed-off-by: John Snow <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eric Blake <address@hidden>


  Commit: f8ca8609d8549def45b28e82ecac64adaeee9f12
      
https://github.com/qemu/qemu/commit/f8ca8609d8549def45b28e82ecac64adaeee9f12
  Author: John Snow <address@hidden>
  Date:   2019-01-14 (Mon, 14 Jan 2019)

  Changed paths:
    M tests/qemu-iotests/206
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests: remove default filters from qmp_log

Several places in iotests deal with serializing objects into JSON
strings, but to add pretty-printing it seems desirable to localize
all of those cases.

log() seems like a good candidate for that centralized behavior.
log() can already serialize json objects, but when it does so,
it assumes filters=[] operates on QMP objects, not strings.

qmp_log currently operates by dumping outgoing and incoming QMP
objects into strings and filtering them assuming that filters=[]
are string filters.

To have qmp_log use log's serialization, qmp_log will need to
accept only qmp filters, not text filters.

However, only a single caller of qmp_log actually requires any
filters at all. I remove the default filter and add it explicitly
to the caller in preparation for refactoring qmp_log to use rich
filters instead.

test 206 is amended to name the filter explicitly and the default
is removed.

Signed-off-by: John Snow <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eric Blake <address@hidden>


  Commit: 08fcd6111e1949f456e1b232ebeeb0cc17019a92
      
https://github.com/qemu/qemu/commit/08fcd6111e1949f456e1b232ebeeb0cc17019a92
  Author: John Snow <address@hidden>
  Date:   2019-01-14 (Mon, 14 Jan 2019)

  Changed paths:
    M tests/qemu-iotests/206
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests: change qmp_log filters to expect QMP objects only

As laid out in the previous commit's message:

```
Several places in iotests deal with serializing objects into JSON
strings, but to add pretty-printing it seems desirable to localize
all of those cases.

log() seems like a good candidate for that centralized behavior.
log() can already serialize json objects, but when it does so,
it assumes filters=[] operates on QMP objects, not strings.

qmp_log currently operates by dumping outgoing and incoming QMP
objects into strings and filtering them assuming that filters=[]
are string filters.
```

Therefore:

Change qmp_log to treat filters as if they're always qmp object filters,
then change the logging call to rely on log()'s ability to serialize QMP
objects, so we're not duplicating that effort.

Add a qmp version of filter_testfiles and adjust the only caller using
it for qmp_log to use the qmp version.

Signed-off-by: John Snow <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Signed-off-by: Eric Blake <address@hidden>


  Commit: 55cd64eab5cb7958c629edbf5f2233b87dfbd1b0
      
https://github.com/qemu/qemu/commit/55cd64eab5cb7958c629edbf5f2233b87dfbd1b0
  Author: John Snow <address@hidden>
  Date:   2019-01-14 (Mon, 14 Jan 2019)

  Changed paths:
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests: implement pretty-print for log and qmp_log

If iotests have lines exceeding >998 characters long, git doesn't
want to send it plaintext to the list. We can solve this by allowing
the iotests to use pretty printed QMP output that we can match against
instead.

As a bonus, it's much nicer for human eyes too.

Signed-off-by: John Snow <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eric Blake <address@hidden>


  Commit: 14da540f2a68b3f730b1a7c31de783f3d68f6fc7
      
https://github.com/qemu/qemu/commit/14da540f2a68b3f730b1a7c31de783f3d68f6fc7
  Author: John Snow <address@hidden>
  Date:   2019-01-14 (Mon, 14 Jan 2019)

  Changed paths:
    A tests/qemu-iotests/236
    A tests/qemu-iotests/236.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  iotests: add iotest 236 for testing bitmap merge

New interface, new smoke test.

Signed-off-by: John Snow <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
[eblake: fix last-minute change to echo text]
Signed-off-by: Eric Blake <address@hidden>


  Commit: 9d97658020db922b68da05faadcdd61f49fbbdc7
      
https://github.com/qemu/qemu/commit/9d97658020db922b68da05faadcdd61f49fbbdc7
  Author: Philippe Mathieu-Daudé <address@hidden>
  Date:   2019-01-14 (Mon, 14 Jan 2019)

  Changed paths:
    M qemu-nbd.c

  Log Message:
  -----------
  qemu-nbd: Rename 'exp' variable clashing with math::exp() symbol

The use of a variable named 'exp' prevents includes to import <math.h>.

Rename it to avoid:

  qemu-nbd.c:64:19: error: ‘exp’ redeclared as different kind of symbol
   static NBDExport *exp;
               ^~~
  In file included from /usr/include/features.h:428,
             from /usr/include/bits/libc-header-start.h:33,
             from /usr/include/stdint.h:26,
             from /usr/lib/gcc/x86_64-redhat-linux/8/include/stdint.h:9,
             from /source/qemu/include/qemu/osdep.h:80,
             from /source/qemu/qemu-nbd.c:19:
  /usr/include/bits/mathcalls.h:95:1: note: previous declaration of ‘exp’ was 
here
    __MATHCALL_VEC (exp,, (_Mdouble_ __x));
    ^~~~~~~~~~~~~~

Signed-off-by: Philippe Mathieu-Daudé <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eric Blake <address@hidden>


  Commit: 2d2fd67428f0cfbffea16969d2635af3e2d78d3d
      
https://github.com/qemu/qemu/commit/2d2fd67428f0cfbffea16969d2635af3e2d78d3d
  Author: Eric Blake <address@hidden>
  Date:   2019-01-14 (Mon, 14 Jan 2019)

  Changed paths:
    M tests/qemu-iotests/223
    M tests/qemu-iotests/223.out

  Log Message:
  -----------
  nbd: Add some error case testing to iotests 223

Testing success paths is important, but it's also nice to highlight
expected failure handling, to show that we don't crash, and so that
upcoming tests that change behavior can demonstrate the resulting
effects on error paths.

Add the following errors:
Attempting to export without a running server
Attempting to start a second server
Attempting to export a bad node name
Attempting to export a name that is already exported
Attempting to export an enabled bitmap
Attempting to remove an already removed export
Attempting to quit server a second time

All of these properly complain except for a second server-stop,
which will be fixed next.

Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>


  Commit: 7801c3a7fd7042fd2f9435af4b9a6d2094073174
      
https://github.com/qemu/qemu/commit/7801c3a7fd7042fd2f9435af4b9a6d2094073174
  Author: Eric Blake <address@hidden>
  Date:   2019-01-14 (Mon, 14 Jan 2019)

  Changed paths:
    M blockdev-nbd.c
    M tests/qemu-iotests/223
    M tests/qemu-iotests/223.out

  Log Message:
  -----------
  nbd: Forbid nbd-server-stop when server is not running

Since we already forbid other nbd-server commands when not
in the right state, it is unlikely that any caller was relying
on a second stop to behave as a silent no-op.  Update iotest
223 to show the improved behavior.

Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>


  Commit: 702aa50d61497d29ef3b5e9c75d1404b3e6fd831
      
https://github.com/qemu/qemu/commit/702aa50d61497d29ef3b5e9c75d1404b3e6fd831
  Author: Eric Blake <address@hidden>
  Date:   2019-01-14 (Mon, 14 Jan 2019)

  Changed paths:
    M nbd/server.c
    M tests/qemu-iotests/223
    M tests/qemu-iotests/223.out

  Log Message:
  -----------
  nbd: Only require disabled bitmap for read-only exports

Our initial implementation of x-nbd-server-add-bitmap put
in a restriction because of incremental backups: in that
usage, we are exporting one qcow2 file (the temporary overlay
target of a blockdev-backup sync:none job) and a dirty bitmap
owned by a second qcow2 file (the source of the
blockdev-backup, which is the backing file of the temporary).
While both qcow2 files are still writable (the target in
order to capture copy-on-write of old contents, and the
source in order to track live guest writes in the meantime),
the NBD client expects to see constant data, including the
dirty bitmap.  An enabled bitmap in the source would be
modified by guest writes, which is at odds with the NBD
export being a read-only constant view, hence the initial
code choice of enforcing a disabled bitmap (the intent is
that the exposed bitmap was disabled in the same transaction
that started the blockdev-backup job, although we don't want
to track enough state to actually enforce that).

However, consider the case of a bitmap contained in a read-only
node (including when the bitmap is found in a backing layer of
the active image).  Because the node can't be modified, the
bitmap won't change due to writes, regardless of whether it is
still enabled.  Forbidding the export unless the bitmap is
disabled is awkward, paritcularly since we can't change the
bitmap to be disabled (because the node is read-only).

Alternatively, consider the case of live storage migration,
where management directs the destination to create a writable
NBD server, then performs a drive-mirror from the source to
the target, prior to doing the rest of the live migration.
Since storage migration can be time-consuming, it may be wise
to let the destination include a dirty bitmap to track which
portions it has already received, where even if the migration
is interrupted and restarted, the source can query the
destination block status in order to potentially minimize
re-sending data that has not changed in the meantime on a
second attempt. Such code has not been written, and might not
be trivial (after all, a cluster being marked dirty in the
bitmap does not necessarily guarantee it has the desired
contents), but it makes sense that letting an active dirty
bitmap be exposed and changing alongside writes may prove
useful in the future.

Solve both issues by gating the restriction against a
disabled bitmap to only happen when the caller has requested
a read-only export, and where the BDS that owns the bitmap
(whether or not it is the BDS handed to nbd_export_new() or
from its backing chain) is still writable.  We could drop
the check altogether (if management apps are prepared to
deal with a changing bitmap even on a read-only image), but
for now keeping a check for the read-only case still stands
a chance of preventing management errors.

Update iotest 223 to show the looser behavior by leaving
a bitmap enabled the whole run; note that we have to tear
down and re-export a node when handling an error.

Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>


  Commit: 3fa4c76590569f9dc128beb3eee65aaefcb6321e
      
https://github.com/qemu/qemu/commit/3fa4c76590569f9dc128beb3eee65aaefcb6321e
  Author: Eric Blake <address@hidden>
  Date:   2019-01-14 (Mon, 14 Jan 2019)

  Changed paths:
    M blockdev-nbd.c
    M include/block/nbd.h
    M nbd/server.c
    M qemu-nbd.c

  Log Message:
  -----------
  nbd: Merge nbd_export_set_name into nbd_export_new

The existing NBD code had a weird split where nbd_export_new()
created an export but did not add it to the list of exported
names until a later nbd_export_set_name() came along and grabbed
a second reference on the object; later, the first call to
nbd_export_close() drops the second reference while removing
the export from the list.  This is in part because the QAPI
NbdServerRemoveNode enum documents the possibility of adding a
mode where we could do a soft disconnect: preventing new clients,
but waiting for existing clients to gracefully quit, based on
the mode used when calling nbd_export_close().

But in spite of all that, note that we never change the name of
an NBD export while it is exposed, which means it is easier to
just inline the process of setting the name as part of creating
the export.

Inline the contents of nbd_export_set_name() and
nbd_export_set_description() into the two points in an export
lifecycle where they matter, then adjust both callers to pass
the name up front.  Note that for creation, all callers pass a
non-NULL name, (passing NULL at creation was for old style
servers, but we removed support for that in commit 7f7dfe2a),
so we can add an assert and do things unconditionally; but for
cleanup, because of the dual nature of nbd_export_close(), we
still have to be careful to avoid use-after-free.  Along the
way, add a comment reminding ourselves of the potential of
adding a middle mode disconnect.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>


  Commit: 5fcbeb06812685a2c6d7e0e6f28f018987d08b79
      
https://github.com/qemu/qemu/commit/5fcbeb06812685a2c6d7e0e6f28f018987d08b79
  Author: Eric Blake <address@hidden>
  Date:   2019-01-14 (Mon, 14 Jan 2019)

  Changed paths:
    M blockdev-nbd.c
    M hmp.c
    M qapi/block.json
    M tests/qemu-iotests/223
    M tests/qemu-iotests/223.out

  Log Message:
  -----------
  nbd: Allow bitmap export during QMP nbd-server-add

With the experimental x-nbd-server-add-bitmap command, there was
a window of time where an NBD client could see the export but not
the associated dirty bitmap, which can cause a client that planned
on using the dirty bitmap to be forced to treat the entire image
as dirty as a safety fallback.  Furthermore, if the QMP client
successfully exports a disk but then fails to add the bitmap, it
has to take on the burden of removing the export.  Since we don't
allow changing the exposed dirty bitmap (whether to a different
bitmap, or removing advertisement of the bitmap), it is nicer to
make the bitmap tied to the export at the time the export is
created, with automatic failure to export if the bitmap is not
available.

The experimental command included an optional 'bitmap-export-name'
field for remapping the name exposed over NBD to be different from
the bitmap name stored on disk.  However, my libvirt demo code
for implementing differential backups on top of persistent bitmaps
did not need to take advantage of that feature (it is instead
possible to create a new temporary bitmap with the desired name,
use block-dirty-bitmap-merge to merge one or more persistent
bitmaps into the temporary, then associate the temporary with the
NBD export, if control is needed over the exported bitmap name).
Hence, I'm not copying that part of the experiment over to the
stable addition. For more details on the libvirt demo, see
https://www.redhat.com/archives/libvir-list/2018-October/msg01254.html,
https://kvmforum2018.sched.com/event/FzuB/facilitating-incremental-backup-eric-blake-red-hat

This patch focuses on the user interface, and reduces (but does
not completely eliminate) the window where an NBD client can see
the export but not the dirty bitmap, with less work to clean up
after errors.  Later patches will add further cleanups now that
this interface is declared stable via a single QMP command,
including removing the race window.

Update test 223 to use the new interface.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>


  Commit: 7dc570b3806e5b0a4c9219061556ed5a4a0de80c
      
https://github.com/qemu/qemu/commit/7dc570b3806e5b0a4c9219061556ed5a4a0de80c
  Author: Eric Blake <address@hidden>
  Date:   2019-01-14 (Mon, 14 Jan 2019)

  Changed paths:
    M blockdev-nbd.c
    M qapi/block.json

  Log Message:
  -----------
  nbd: Remove x-nbd-server-add-bitmap

Now that nbd-server-add can do the same functionality (well, other
than making the exported bitmap name different than the underlying
bitamp - but we argued that was not essential, since it is just as
easy to create a new non-persistent bitmap with the desired name),
we no longer need the experimental separate command.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>


  Commit: 678ba275c77b5b12f3bc9bb369a1b824fc9f679f
      
https://github.com/qemu/qemu/commit/678ba275c77b5b12f3bc9bb369a1b824fc9f679f
  Author: Eric Blake <address@hidden>
  Date:   2019-01-14 (Mon, 14 Jan 2019)

  Changed paths:
    M blockdev-nbd.c
    M include/block/nbd.h
    M nbd/server.c
    M qemu-nbd.c

  Log Message:
  -----------
  nbd: Merge nbd_export_bitmap into nbd_export_new

We only have one caller that wants to export a bitmap name,
which it does right after creation of the export. But there is
still a brief window of time where an NBD client could see the
export but not the dirty bitmap, which a robust client would
have to interpret as meaning the entire image should be treated
as dirty.  Better is to eliminate the window entirely, by
inlining nbd_export_bitmap() into nbd_export_new(), and refusing
to create the bitmap in the first place if the requested bitmap
can't be located.

We also no longer need logic for setting a different bitmap
name compared to the bitmap being exported.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>


  Commit: 636192c4b6052820ea126a5287c58a8f53f3c84f
      
https://github.com/qemu/qemu/commit/636192c4b6052820ea126a5287c58a8f53f3c84f
  Author: Eric Blake <address@hidden>
  Date:   2019-01-14 (Mon, 14 Jan 2019)

  Changed paths:
    M qemu-nbd.c
    M qemu-nbd.texi
    M tests/qemu-iotests/223
    M tests/qemu-iotests/223.out

  Log Message:
  -----------
  qemu-nbd: Add --bitmap=NAME option

Having to fire up qemu, then use QMP commands for nbd-server-start
and nbd-server-add, just to expose a persistent dirty bitmap, is
rather tedious.  Make it possible to expose a dirty bitmap using
just qemu-nbd (of course, for now this only works when qemu-nbd is
visiting a BDS formatted as qcow2).

Of course, any good feature also needs unit testing, so expand
iotest 223 to cover it.

Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>


  Commit: 44ba6010635641a538c9b9b1f377dfa288751906
      
https://github.com/qemu/qemu/commit/44ba6010635641a538c9b9b1f377dfa288751906
  Author: Peter Maydell <address@hidden>
  Date:   2019-01-15 (Tue, 15 Jan 2019)

  Changed paths:
    M block/dirty-bitmap.c
    M blockdev-nbd.c
    M blockdev.c
    M hmp.c
    M include/block/nbd.h
    M nbd/server.c
    M qapi/block-core.json
    M qapi/block.json
    M qapi/transaction.json
    M qemu-nbd.c
    M qemu-nbd.texi
    M tests/qemu-iotests/206
    M tests/qemu-iotests/223
    M tests/qemu-iotests/223.out
    A tests/qemu-iotests/236
    A tests/qemu-iotests/236.out
    M tests/qemu-iotests/group
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-01-14' into 
staging

nbd patches for 2019-01-14

Promote bitmap/NBD interfaces to stable for use in incremental
backups. Add 'qemu-nbd --bitmap'.

- John Snow: 0/11 bitmaps: remove x- prefix from QMP api
- Philippe Mathieu-Daudé: qemu-nbd: Rename 'exp' variable clashing with 
math::exp() symbol
- Eric Blake: 0/8 Promote x-nbd-server-add-bitmap to stable

# gpg: Signature made Mon 14 Jan 2019 16:13:45 GMT
# gpg:                using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <address@hidden>"
# gpg:                 aka "Eric Blake (Free Software Programmer) 
<address@hidden>"
# gpg:                 aka "[jpeg image of size 6874]"
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2  F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2019-01-14:
  qemu-nbd: Add --bitmap=NAME option
  nbd: Merge nbd_export_bitmap into nbd_export_new
  nbd: Remove x-nbd-server-add-bitmap
  nbd: Allow bitmap export during QMP nbd-server-add
  nbd: Merge nbd_export_set_name into nbd_export_new
  nbd: Only require disabled bitmap for read-only exports
  nbd: Forbid nbd-server-stop when server is not running
  nbd: Add some error case testing to iotests 223
  qemu-nbd: Rename 'exp' variable clashing with math::exp() symbol
  iotests: add iotest 236 for testing bitmap merge
  iotests: implement pretty-print for log and qmp_log
  iotests: change qmp_log filters to expect QMP objects only
  iotests: remove default filters from qmp_log
  iotests: add qmp recursive sorting function
  iotests: add filter_generated_node_ids
  iotests.py: don't abort if IMGKEYSECRET is undefined
  block: remove 'x' prefix from experimental bitmap APIs
  blockdev: n-ary bitmap merge
  block/dirty-bitmap: remove assertion from restore
  blockdev: abort transactions in reverse order

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/b2f7c27f56bf...44ba60106356
      **NOTE:** This service has been marked for deprecation: 
https://developer.github.com/changes/2018-04-25-github-services-deprecation/

      Functionality will be removed from GitHub.com on January 31st, 2019.