[Qemu-commits] [qemu/qemu] 7606c9: nbd/server: Fix dirty bitmap logic regression

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 7606c99a0421be7e9d984766fe239f7791a2fd9c
      
https://github.com/qemu/qemu/commit/7606c99a0421be7e9d984766fe239f7791a2fd9c
  Author: Eric Blake <address@hidden>
  Date:   2018-07-02 (Mon, 02 Jul 2018)

  Changed paths:
    M nbd/server.c

  Log Message:
  -----------
  nbd/server: Fix dirty bitmap logic regression

In my hurry to fix a build failure, I introduced a logic bug.
The assertion conditional is backwards, meaning that qemu will
now abort instead of reporting dirty bitmap status.

The bug can only be tickled by an NBD client using an exported
dirty bitmap (which is still an experimental QMP command), so
it's not the end of the world for supported usage (and neither
'make check' nor qemu-iotests fails); but it also shows that we
really want qemu-io support for reading dirty bitmaps if only
so that I can add iotests coverage to prevent future
brown-bag-of-shame events like this one.

Fixes: 45eb6fb6
Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>


  Commit: 8ee1cef4593a7bda076891470c0620e79333c0d0
      
https://github.com/qemu/qemu/commit/8ee1cef4593a7bda076891470c0620e79333c0d0
  Author: Eric Blake <address@hidden>
  Date:   2018-07-02 (Mon, 02 Jul 2018)

  Changed paths:
    M block/iscsi.c

  Log Message:
  -----------
  iscsi: Avoid potential for get_status overflow

Detected by Coverity: Multiplying two 32-bit int and assigning
the result to a 64-bit number is a risk of overflow.  Prior to
the conversion to byte-based interfaces, the block layer took
care of ensuring that a status request never exceeded 2G in
the driver; but after that conversion, the block layer expects
drivers to deal with any size request (the driver can always
truncate the request size back down, as long as it makes
progress).  So, in the off-chance that someone makes a large
request, we are at the mercy of whether iscsi_get_lba_status_task()
will cap things to at most INT_MAX / iscsilun->block_size when
it populates lbasd->num_blocks; since I could not easily audit
that, it's better to be safe than sorry by just forcing a 64-bit
multiply.

Fixes: 92809c36
CC: address@hidden
Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>


  Commit: 930fe17f9900e9c879834f2d2e5c301992623332
      
https://github.com/qemu/qemu/commit/930fe17f9900e9c879834f2d2e5c301992623332
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2018-07-02 (Mon, 02 Jul 2018)

  Changed paths:
    M blockdev.c

  Log Message:
  -----------
  blockdev: enable non-root nodes for backup source

This is needed to implement the image-fleecing workflow where we
create a temporary node backed by an active node, then start
backupdev-backup sync=none from the active node to the temp node.

In this case, the active node is now a root node AND a backing node,
so it no longer qualifies as a root node, so we loosen the restriction
on which nodes can be considered as the source for a backup.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: John Snow <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eric Blake <address@hidden>


  Commit: bacebdedbf921a2c641a34486ff543089d338f32
      
https://github.com/qemu/qemu/commit/bacebdedbf921a2c641a34486ff543089d338f32
  Author: John Snow <address@hidden>
  Date:   2018-07-02 (Mon, 02 Jul 2018)

  Changed paths:
    A tests/qemu-iotests/222
    A tests/qemu-iotests/222.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  iotests: add 222 to test basic fleecing

Signed-off-by: John Snow <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Eric Blake <address@hidden>


  Commit: 216ee3657e14013505abe7853cecb632199fb13e
      
https://github.com/qemu/qemu/commit/216ee3657e14013505abe7853cecb632199fb13e
  Author: Eric Blake <address@hidden>
  Date:   2018-07-02 (Mon, 02 Jul 2018)

  Changed paths:
    M block/nbd-client.c
    M block/nbd-client.h
    M block/nbd.c
    M include/block/nbd.h
    M nbd/client.c
    M qapi/block-core.json

  Log Message:
  -----------
  nbd/client: Add x-dirty-bitmap to query bitmap from server

In order to test that the NBD server is properly advertising
dirty bitmaps, we need a bare minimum client that can request
and read the context.  Since feature freeze for 3.0 is imminent,
this is the smallest workable patch, which replaces the qemu
block status report with the results of the NBD server's dirty
bitmap (making it very easy to use 'qemu-img map --output=json'
to learn where the dirty portions are).  Note that the NBD
protocol defines a dirty section with the same bit but opposite
sense that normal "base:allocation" uses to report an allocated
section; so in qemu-img map output, "data":true corresponds to
clean, "data":false corresponds to dirty.

A more complete solution that allows dirty bitmaps to be queried
at the same time as normal block status will be required before
this addition can lose the x- prefix.  Until then, the fact that
this replaces normal status with dirty status means actions
like 'qemu-img convert' will likely misbehave due to treating
dirty regions of the file as if they are unallocated.

The next patch adds an iotest to exercise this new code.

Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>


  Commit: a1532a225a183c9fa60b9c1e8ac8a00c7771f64d
      
https://github.com/qemu/qemu/commit/a1532a225a183c9fa60b9c1e8ac8a00c7771f64d
  Author: Eric Blake <address@hidden>
  Date:   2018-07-02 (Mon, 02 Jul 2018)

  Changed paths:
    A tests/qemu-iotests/223
    A tests/qemu-iotests/223.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  iotests: New test 223 for exporting dirty bitmap over NBD

Although this test is NOT a full test of image fleecing (as it
intentionally uses just a single block device directly exported
over NBD, rather than trying to set up a blockdev-backup job with
multiple BDS involved), it DOES prove that qemu as a server is
able to properly expose a dirty bitmap over NBD.

When coupled with image fleecing, it is then possible for a
third-party client to do an incremental backup by using
qemu-img map with the x-dirty-bitmap option to learn which parts
of the file are dirty (perhaps confusingly, they are the portions
mapped as "data":false - which is part of the reason this is
still in the x- experimental namespace), along with another
normal client (perhaps 'qemu-nbd -c' to expose the server over
/dev/nbd0 and then just use normal I/O on that block device) to
read the dirty sections.

Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Tested-by: John Snow <address@hidden>
Reviewed-by: John Snow <address@hidden>


  Commit: 9b75dcb15f562577a937ae01f324946513586e59
      
https://github.com/qemu/qemu/commit/9b75dcb15f562577a937ae01f324946513586e59
  Author: Peter Maydell <address@hidden>
  Date:   2018-07-03 (Tue, 03 Jul 2018)

  Changed paths:
    M block/iscsi.c
    M block/nbd-client.c
    M block/nbd-client.h
    M block/nbd.c
    M blockdev.c
    M include/block/nbd.h
    M nbd/client.c
    M nbd/server.c
    M qapi/block-core.json
    A tests/qemu-iotests/222
    A tests/qemu-iotests/222.out
    A tests/qemu-iotests/223
    A tests/qemu-iotests/223.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2018-07-02' into 
staging

nbd patches for 2018-07-02

Bug fixes and iotest exposure of fleecing via NBD (serving a
read-only point-in-time view via blockdev-backup sync:none,
as well as serving dirty bitmaps over NBD), including a new
x-dirty-bitmap parameter when opening NBD clients as the
counterpart to x-nbd-server-add-bitmap. Also a random fix
for iscsi block_status spotted by Coverity that missed other
miscellaneous trees.

- Eric Blake: nbd/server: Fix dirty bitmap logic regression
- Eric Blake: iscsi: Avoid potential for get_status overflow
- John Snow/Vladimir Sementsov-Ogievskiy: 0/2 block: formalize and test fleecing
- Eric Blake: 0/2 test NBD bitmap export

# gpg: Signature made Tue 03 Jul 2018 02:33:03 BST
# gpg:                using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <address@hidden>"
# gpg:                 aka "Eric Blake (Free Software Programmer) 
<address@hidden>"
# gpg:                 aka "[jpeg image of size 6874]"
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2  F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2018-07-02:
  iotests: New test 223 for exporting dirty bitmap over NBD
  nbd/client: Add x-dirty-bitmap to query bitmap from server
  iotests: add 222 to test basic fleecing
  blockdev: enable non-root nodes for backup source
  iscsi: Avoid potential for get_status overflow
  nbd/server: Fix dirty bitmap logic regression

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/46d0885adff9...9b75dcb15f56
      **NOTE:** This service been marked for deprecation: 
https://developer.github.com/changes/2018-04-25-github-services-deprecation/

      Functionality will be removed from GitHub.com on January 31st, 2019.