[Qemu-commits] [qemu/qemu] 82dc8b: block: Fix .bdrv_open flags

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 82dc8b411040fa8a7418a012ff39b8d06f68e639
      
https://github.com/qemu/qemu/commit/82dc8b411040fa8a7418a012ff39b8d06f68e639
  Author: Kevin Wolf <address@hidden>
  Date:   2016-01-19 (Tue, 19 Jan 2016)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: Fix .bdrv_open flags

bdrv_common_open() modified bs->open_flags after inferring the set of
options to pass to the driver's .bdrv_open callback. This means that the
cache options were correctly set in bs->open_flags (and therefore
correctly displayed in 'info block'), but the image would actually be
opened with the default cache mode instead.

This patch removes the flags parameter to bdrv_common_open() (except for
BDRV_O_NO_BACKING it's the same as bs->open_flags anyway, and having two
names for the same thing is confusing), and moves the assignment of
open_flags down to immediately before calling into the block drivers. In
all other places, bs->open_flags is now used consistently.

Signed-off-by: Kevin Wolf <address@hidden>
Tested-by: Christian Borntraeger <address@hidden>
Reviewed-by: Denis V. Lunev <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>


  Commit: 972b543c6b63579aee590b738d21af09f01569f7
      
https://github.com/qemu/qemu/commit/972b543c6b63579aee590b738d21af09f01569f7
  Author: Christian Borntraeger <address@hidden>
  Date:   2016-01-19 (Tue, 19 Jan 2016)

  Changed paths:
    M block/raw-posix.c

  Log Message:
  -----------
  block/raw-posix: avoid bogus fixup for cylinders on DASD disks

large volume DASD that have > 64k cylinders do claim to have
0xFFFE cylinders as special value in the old 16 bit field. We
want to pass this "token" along to the guest, instead of
calculating the real number. Otherwise qemu might fail with
"cyls must be between 1 and 65535"

Cc: address@hidden
Acked-by: Cornelia Huck <address@hidden>
Signed-off-by: Christian Borntraeger <address@hidden>
Reviewed-by: Markus Armbruster <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 25ad8e6e95770d4f01844dc2542d861e73b174b2
      
https://github.com/qemu/qemu/commit/25ad8e6e95770d4f01844dc2542d861e73b174b2
  Author: Fam Zheng <address@hidden>
  Date:   2016-01-20 (Wed, 20 Jan 2016)

  Changed paths:
    M qemu-img.c

  Log Message:
  -----------
  qemu-img: Speed up comparing empty/zero images

Two empty raw files are always compared by actually reading data even if
there is no data, because BDRV_BLOCK_ZERO is considered "allocated" in
bdrv_is_allocated_above().  That is inefficient.

Use bdrv_get_block_status_above() for more information, and skip the
consecutive zero sectors.

This brings a huge speed up in comparing sparse/empty raw images:

    $ qemu-img create a 1G

    $ time ~/build/master/bin/qemu-img compare a a
    Images are identical.

    real    0m6.583s
    user    0m0.191s
    sys     0m6.367s

    $ time qemu-img compare a a
    Images are identical.

    real    0m0.033s
    user    0m0.003s
    sys     0m0.031s

Signed-off-by: Fam Zheng <address@hidden>
Reviewed-by: Paolo Bonzini <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: a174da3613f548d58433f64cf3c99dd302c6154a
      
https://github.com/qemu/qemu/commit/a174da3613f548d58433f64cf3c99dd302c6154a
  Author: Eric Blake <address@hidden>
  Date:   2016-01-20 (Wed, 20 Jan 2016)

  Changed paths:
    M tests/qemu-iotests/028
    M tests/qemu-iotests/028.out

  Log Message:
  -----------
  qemu-iotests: Reduce racy output in 028

On my machine, './check -qcow2 028' was failing about 80% of the
time, due to a race in how many times the repeated attempts
to run 'info block-jobs' could occur before the job was done,
showing up as a failure of fewer '(qemu) ' prompts than in the
expected output.  Silence the output during the repetitions, then
add a final clean command to keep the expected output useful;
once patched, I was finally able to run the test 20 times in a
row with no failures.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 80c71a241ae3cd3b89527865ba730b2fa1f9e46f
      
https://github.com/qemu/qemu/commit/80c71a241ae3cd3b89527865ba730b2fa1f9e46f
  Author: Peter Maydell <address@hidden>
  Date:   2016-01-20 (Wed, 20 Jan 2016)

  Changed paths:
    M block/accounting.c
    M block/archipelago.c
    M block/backup.c
    M block/blkdebug.c
    M block/blkverify.c
    M block/block-backend.c
    M block/bochs.c
    M block/cloop.c
    M block/commit.c
    M block/curl.c
    M block/dmg.c
    M block/gluster.c
    M block/io.c
    M block/iscsi.c
    M block/linux-aio.c
    M block/mirror.c
    M block/nbd-client.c
    M block/nbd.c
    M block/nfs.c
    M block/null.c
    M block/parallels.c
    M block/qapi.c
    M block/qcow.c
    M block/qcow2-cache.c
    M block/qcow2-cluster.c
    M block/qcow2-refcount.c
    M block/qcow2-snapshot.c
    M block/qcow2.c
    M block/qed-check.c
    M block/qed-cluster.c
    M block/qed-gencb.c
    M block/qed-l2-cache.c
    M block/qed-table.c
    M block/qed.c
    M block/quorum.c
    M block/raw-posix.c
    M block/raw-win32.c
    M block/raw_bsd.c
    M block/rbd.c
    M block/sheepdog.c
    M block/snapshot.c
    M block/ssh.c
    M block/stream.c
    M block/throttle-groups.c
    M block/vdi.c
    M block/vhdx-endian.c
    M block/vhdx-log.c
    M block/vhdx.c
    M block/vmdk.c
    M block/vpc.c
    M block/vvfat.c
    M block/win32-aio.c
    M block/write-threshold.c
    M hw/block/block.c
    M hw/block/cdrom.c
    M hw/block/dataplane/virtio-blk.c
    M hw/block/ecc.c
    M hw/block/fdc.c
    M hw/block/hd-geometry.c
    M hw/block/m25p80.c
    M hw/block/nvme.c
    M hw/block/onenand.c
    M hw/block/pflash_cfi01.c
    M hw/block/pflash_cfi02.c
    M hw/block/tc58128.c
    M hw/block/virtio-blk.c
    M hw/block/xen_disk.c
    M qemu-img.c
    M qemu-io-cmds.c
    M qemu-io.c

  Log Message:
  -----------
  block: Clean up includes

Clean up includes so that osdep.h is included first and headers
which it implies are not included manually.

This commit was created with scripts/clean-includes.

Signed-off-by: Peter Maydell <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 1a4828c7934f592c146bc3dc979e78df430545cf
      
https://github.com/qemu/qemu/commit/1a4828c7934f592c146bc3dc979e78df430545cf
  Author: Kevin Wolf <address@hidden>
  Date:   2016-01-20 (Wed, 20 Jan 2016)

  Changed paths:
    M block/qcow2.c
    M tests/qemu-iotests/031.out
    M tests/qemu-iotests/061.out

  Log Message:
  -----------
  qcow2: Write feature table only for v3 images

Version 2 images don't have feature bits, so writing a feature table to
those images is kind of pointless.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>


  Commit: b527c9b392a87abff698ca435da0dfa2bd6324a2
      
https://github.com/qemu/qemu/commit/b527c9b392a87abff698ca435da0dfa2bd6324a2
  Author: Kevin Wolf <address@hidden>
  Date:   2016-01-20 (Wed, 20 Jan 2016)

  Changed paths:
    M block/qcow2.c
    M tests/qemu-iotests/031.out
    M tests/qemu-iotests/036
    M tests/qemu-iotests/036.out
    M tests/qemu-iotests/061.out

  Log Message:
  -----------
  qcow2: Write full header on image creation

When creating a qcow2 image, we didn't necessarily call
qcow2_update_header(), but could end up with the basic header that
qcow2_create2() created manually. One thing that this basic header
lacks is the feature table. Let's make sure that it's always present.

This requires a few updates to test cases as well.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>


  Commit: 09e0c771e47e02278c264bafff6bfc0771732d72
      
https://github.com/qemu/qemu/commit/09e0c771e47e02278c264bafff6bfc0771732d72
  Author: Kevin Wolf <address@hidden>
  Date:   2016-01-20 (Wed, 20 Jan 2016)

  Changed paths:
    M block/io.c

  Log Message:
  -----------
  block: Assert no write requests under BDRV_O_INCOMING

As long as BDRV_O_INCOMING is set, the image file is only opened so we
have a file descriptor for it. We're definitely not supposed to modify
the image, it's still owned by the migration source.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>


  Commit: 23c88b24721ee907639705bf9de03dcd6b1e66ad
      
https://github.com/qemu/qemu/commit/23c88b24721ee907639705bf9de03dcd6b1e66ad
  Author: Kevin Wolf <address@hidden>
  Date:   2016-01-20 (Wed, 20 Jan 2016)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: Fix error path in bdrv_invalidate_cache()

We can only clear BDRV_O_INCOMING if the caches were actually
invalidated.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>


  Commit: 04c01a5c8f006b6e45fa5be8ea857efe7d9c41f9
      
https://github.com/qemu/qemu/commit/04c01a5c8f006b6e45fa5be8ea857efe7d9c41f9
  Author: Kevin Wolf <address@hidden>
  Date:   2016-01-20 (Wed, 20 Jan 2016)

  Changed paths:
    M block.c
    M block/io.c
    M block/qcow2.c
    M block/qed.c
    M include/block/block.h
    M nbd/server.c

  Log Message:
  -----------
  block: Rename BDRV_O_INCOMING to BDRV_O_INACTIVE

Instead of covering only the state of images on the migration
destination before the migration is completed, the flag will also cover
the state of images on the migration source after completion. This
common state implies that the image is technically still open, but no
writes will happen and any cached contents will be reloaded from disk if
and when the image leaves this state.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>


  Commit: 76b1c7fe1cbc45f46b2cccd471369ccd4b49b6fd
      
https://github.com/qemu/qemu/commit/76b1c7fe1cbc45f46b2cccd471369ccd4b49b6fd
  Author: Kevin Wolf <address@hidden>
  Date:   2016-01-20 (Wed, 20 Jan 2016)

  Changed paths:
    M block.c
    M include/block/block.h
    M include/block/block_int.h
    M migration/migration.c
    M qmp.c

  Log Message:
  -----------
  block: Inactivate BDS when migration completes

So far, live migration with shared storage meant that the image is in a
not-really-ready don't-touch-me state on the destination while the
source is still actively using it, but after completing the migration,
the image was fully opened on both sides. This is bad.

This patch adds a block driver callback to inactivate images on the
source before completing the migration. Inactivation means that it goes
to a state as if it was just live migrated to the qemu instance on the
source (i.e. BDRV_O_INACTIVE is set). You're then supposed to continue
either on the source or on the destination, which takes ownership of the
image.

A typical migration looks like this now with respect to disk images:

1. Destination qemu is started, the image is opened with
   BDRV_O_INACTIVE. The image is fully opened on the source.

2. Migration is about to complete. The source flushes the image and
   inactivates it. Now both sides have the image opened with
   BDRV_O_INACTIVE and are expecting the other side to still modify it.

3. One side (the destination on success) continues and calls
   bdrv_invalidate_all() in order to take ownership of the image again.
   This removes BDRV_O_INACTIVE on the resuming side; the flag remains
   set on the other side.

This ensures that the same image isn't written to by both instances
(unless both are resumed, but then you get what you deserve). This is
important because .bdrv_close for non-BDRV_O_INACTIVE images could write
to the image file, which is definitely forbidden while another host is
using the image.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>


  Commit: ec6d891224f708b2cda2a1edf68ffc0ff1316fca
      
https://github.com/qemu/qemu/commit/ec6d891224f708b2cda2a1edf68ffc0ff1316fca
  Author: Kevin Wolf <address@hidden>
  Date:   2016-01-20 (Wed, 20 Jan 2016)

  Changed paths:
    M block/qcow2.c

  Log Message:
  -----------
  qcow2: Implement .bdrv_inactivate

The callback has to ensure that closing or flushing the image afterwards
wouldn't cause a write access to the image files. This means that just
the caches have to be written out, which is part of the existing
.bdrv_close implementation.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>


  Commit: 140fd5a69cf19460b8daa8a9bb83bd869f6db14d
      
https://github.com/qemu/qemu/commit/140fd5a69cf19460b8daa8a9bb83bd869f6db14d
  Author: Kevin Wolf <address@hidden>
  Date:   2016-01-20 (Wed, 20 Jan 2016)

  Changed paths:
    M block/qcow2.c

  Log Message:
  -----------
  qcow2: Fix BDRV_O_INACTIVE handling in qcow2_invalidate_cache()

What qcow2_invalidate_cache() should do is close the image with
BDRV_O_INACTIVE set and reopen it with the flag cleared. In fact, it
used to do exactly the opposite: qcow2_close() relied on bs->open_flags,
which is already updated to have cleared BDRV_O_INACTIVE at this point,
whereas qcow2_open() was called with s->flags, which has the flag still
set. Fix this.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>


  Commit: 191fb11bdfbf53b33068268ce995a5c84045a8d9
      
https://github.com/qemu/qemu/commit/191fb11bdfbf53b33068268ce995a5c84045a8d9
  Author: Kevin Wolf <address@hidden>
  Date:   2016-01-20 (Wed, 20 Jan 2016)

  Changed paths:
    M block/qcow2.c

  Log Message:
  -----------
  qcow2: Make image inaccessible after failed qcow2_invalidate_cache()

If qcow2_invalidate_cache() fails, we are in a state where qcow2_close()
has already been completed, but the image hasn't been reopened yet.
Calling into any qcow2 function for an image in this state will cause
crashes.

The real solution would be to get rid of the close/open pair and instead
do an atomic reset of the involved data structures, but this isn't
trivial, so let's just make the image inaccessible for now.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>


  Commit: d62d9dc4b814950dcc8bd261a3e2e9300d9065e6
      
https://github.com/qemu/qemu/commit/d62d9dc4b814950dcc8bd261a3e2e9300d9065e6
  Author: Fam Zheng <address@hidden>
  Date:   2016-01-20 (Wed, 20 Jan 2016)

  Changed paths:
    M block/vmdk.c

  Log Message:
  -----------
  vmdk: Create streamOptimized as version 3

VMware products accept only version 3 for streamOptimized, let's bump
the version.

Reported-by: Radoslav Gerganov <address@hidden>
Signed-off-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 972606c4db826f286f7f475551180502859f49b9
      
https://github.com/qemu/qemu/commit/972606c4db826f286f7f475551180502859f49b9
  Author: Fam Zheng <address@hidden>
  Date:   2016-01-20 (Wed, 20 Jan 2016)

  Changed paths:
    M blockdev.c
    M include/qemu/throttle.h
    M util/throttle.c

  Log Message:
  -----------
  blockdev: Error out on negative throttling option values

extract_common_blockdev_options() uses qemu_opt_get_number() to parse
the bps/iops numbers to uint64_t, then converts to double and stores in
ThrottleConfig.  The actual parsing is done by strtoull() in
parse_option_number().  Negative numbers are wrapped to large positive
ones, and stored.

We used to reject negative numbers since 7d81c1413c9, but this regressed
when the option parsing code was changed later. Now fix this again.

This time, define an arbitrary large upper limit (1e15),  and check the
values so both negative and impractically big numbers are caught and
reported.

Signed-off-by: Fam Zheng <address@hidden>
Reviewed-by: Markus Armbruster <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: e9b155501da2638e4e319af9960d35da1bc8662b
      
https://github.com/qemu/qemu/commit/e9b155501da2638e4e319af9960d35da1bc8662b
  Author: Fam Zheng <address@hidden>
  Date:   2016-01-20 (Wed, 20 Jan 2016)

  Changed paths:
    M tests/qemu-iotests/051
    M tests/qemu-iotests/051.out
    M tests/qemu-iotests/051.pc.out

  Log Message:
  -----------
  iotests: Test that throttle values ranges

Signed-off-by: Fam Zheng <address@hidden>
Reviewed-by: Markus Armbruster <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 3c9331c47f22224118d5019b0af8eac704824d8d
      
https://github.com/qemu/qemu/commit/3c9331c47f22224118d5019b0af8eac704824d8d
  Author: Peter Maydell <address@hidden>
  Date:   2016-01-21 (Thu, 21 Jan 2016)

  Changed paths:
    M block.c
    M block/accounting.c
    M block/archipelago.c
    M block/backup.c
    M block/blkdebug.c
    M block/blkverify.c
    M block/block-backend.c
    M block/bochs.c
    M block/cloop.c
    M block/commit.c
    M block/curl.c
    M block/dmg.c
    M block/gluster.c
    M block/io.c
    M block/iscsi.c
    M block/linux-aio.c
    M block/mirror.c
    M block/nbd-client.c
    M block/nbd.c
    M block/nfs.c
    M block/null.c
    M block/parallels.c
    M block/qapi.c
    M block/qcow.c
    M block/qcow2-cache.c
    M block/qcow2-cluster.c
    M block/qcow2-refcount.c
    M block/qcow2-snapshot.c
    M block/qcow2.c
    M block/qed-check.c
    M block/qed-cluster.c
    M block/qed-gencb.c
    M block/qed-l2-cache.c
    M block/qed-table.c
    M block/qed.c
    M block/quorum.c
    M block/raw-posix.c
    M block/raw-win32.c
    M block/raw_bsd.c
    M block/rbd.c
    M block/sheepdog.c
    M block/snapshot.c
    M block/ssh.c
    M block/stream.c
    M block/throttle-groups.c
    M block/vdi.c
    M block/vhdx-endian.c
    M block/vhdx-log.c
    M block/vhdx.c
    M block/vmdk.c
    M block/vpc.c
    M block/vvfat.c
    M block/win32-aio.c
    M block/write-threshold.c
    M blockdev.c
    M hw/block/block.c
    M hw/block/cdrom.c
    M hw/block/dataplane/virtio-blk.c
    M hw/block/ecc.c
    M hw/block/fdc.c
    M hw/block/hd-geometry.c
    M hw/block/m25p80.c
    M hw/block/nvme.c
    M hw/block/onenand.c
    M hw/block/pflash_cfi01.c
    M hw/block/pflash_cfi02.c
    M hw/block/tc58128.c
    M hw/block/virtio-blk.c
    M hw/block/xen_disk.c
    M include/block/block.h
    M include/block/block_int.h
    M include/qemu/throttle.h
    M migration/migration.c
    M nbd/server.c
    M qemu-img.c
    M qemu-io-cmds.c
    M qemu-io.c
    M qmp.c
    M tests/qemu-iotests/028
    M tests/qemu-iotests/028.out
    M tests/qemu-iotests/031.out
    M tests/qemu-iotests/036
    M tests/qemu-iotests/036.out
    M tests/qemu-iotests/051
    M tests/qemu-iotests/051.out
    M tests/qemu-iotests/051.pc.out
    M tests/qemu-iotests/061.out
    M util/throttle.c

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Wed 20 Jan 2016 15:37:57 GMT using RSA key ID C88F2FD6
# gpg: Good signature from "Kevin Wolf <address@hidden>"

* remotes/kevin/tags/for-upstream:
  iotests: Test that throttle values ranges
  blockdev: Error out on negative throttling option values
  vmdk: Create streamOptimized as version 3
  qcow2: Make image inaccessible after failed qcow2_invalidate_cache()
  qcow2: Fix BDRV_O_INACTIVE handling in qcow2_invalidate_cache()
  qcow2: Implement .bdrv_inactivate
  block: Inactivate BDS when migration completes
  block: Rename BDRV_O_INCOMING to BDRV_O_INACTIVE
  block: Fix error path in bdrv_invalidate_cache()
  block: Assert no write requests under BDRV_O_INCOMING
  qcow2: Write full header on image creation
  qcow2: Write feature table only for v3 images
  block: Clean up includes
  qemu-iotests: Reduce racy output in 028
  qemu-img: Speed up comparing empty/zero images
  block/raw-posix: avoid bogus fixup for cylinders on DASD disks
  block: Fix .bdrv_open flags

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/3ed0b659ca81...3c9331c47f22