[Qemu-commits] [qemu/qemu] e0a597: iotests: Fix test 200 on s390x without virtio-pci

[Peter Maydell]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: e0a59749efc246646bb208e553489b894450cbcd
      
https://github.com/qemu/qemu/commit/e0a59749efc246646bb208e553489b894450cbcd
  Author: Thomas Huth <address@hidden>
  Date:   2019-04-02 (Tue, 02 Apr 2019)

  Changed paths:
    M tests/qemu-iotests/200

  Log Message:
  -----------
  iotests: Fix test 200 on s390x without virtio-pci

virtio-pci is optional on s390x, e.g. in downstream RHEL builds, it
is disabled. On s390x, virtio-ccw should be used instead. Other tests
like 051 or 240 already use virtio-scsi-ccw instead of virtio-scsi-pci
on s390x, so let's do the same here and always use virtio-scsi-ccw on
s390x.

Signed-off-by: Thomas Huth <address@hidden>
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: a3d6ae2299eaab1bced05551d0a0abfbcd9d08d0
      
https://github.com/qemu/qemu/commit/a3d6ae2299eaab1bced05551d0a0abfbcd9d08d0
  Author: Nir Soffer <address@hidden>
  Date:   2019-04-02 (Tue, 02 Apr 2019)

  Changed paths:
    M qemu-img.c

  Log Message:
  -----------
  qemu-img: Enable BDRV_REQ_MAY_UNMAP in convert

With Kevin's "block: Fix slow pre-zeroing in qemu-img convert"[1]
(commit c9fdcf202f, 'qemu-img: Use BDRV_REQ_NO_FALLBACK for
pre-zeroing') we skip the pre zero step called like this:

    blk_make_zero(s->target, BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK)

And we write zeroes later using:

    blk_co_pwrite_zeroes(s->target,
                         sector_num << BDRV_SECTOR_BITS,
                         n << BDRV_SECTOR_BITS, 0);

Since we use flags=0, this is translated to NBD_CMD_WRITE_ZEROES with
NBD_CMD_FLAG_NO_HOLE flag, which cause the NBD server to allocated space
instead of punching a hole.

Here is an example failure:

$ dd if=/dev/urandom of=src.img bs=1M count=5
$ truncate -s 50m src.img
$ truncate -s 50m dst.img
$ nbdkit -f -v -e '' -U nbd.sock file file=dst.img

$ ./qemu-img convert -n src.img nbd:unix:nbd.sock

We can see in nbdkit log that it received the NBD_CMD_FLAG_NO_HOLE
(may_trim=0):

nbdkit: file[1]: debug: newstyle negotiation: flags: export 0x4d
nbdkit: file[1]: debug: pwrite count=2097152 offset=0
nbdkit: file[1]: debug: pwrite count=2097152 offset=2097152
nbdkit: file[1]: debug: pwrite count=1048576 offset=4194304
nbdkit: file[1]: debug: zero count=33554432 offset=5242880 may_trim=0
nbdkit: file[1]: debug: zero count=13631488 offset=38797312 may_trim=0
nbdkit: file[1]: debug: flush

And the image became fully allocated:

$ qemu-img info dst.img
virtual size: 50M (52428800 bytes)
disk size: 50M

With this change we see that nbdkit did not receive the
NBD_CMD_FLAG_NO_HOLE (may_trim=1):

nbdkit: file[1]: debug: newstyle negotiation: flags: export 0x4d
nbdkit: file[1]: debug: pwrite count=2097152 offset=0
nbdkit: file[1]: debug: pwrite count=2097152 offset=2097152
nbdkit: file[1]: debug: pwrite count=1048576 offset=4194304
nbdkit: file[1]: debug: zero count=33554432 offset=5242880 may_trim=1
nbdkit: file[1]: debug: zero count=13631488 offset=38797312 may_trim=1
nbdkit: file[1]: debug: flush

And the file is sparse as expected:

$ qemu-img info dst.img
virtual size: 50M (52428800 bytes)
disk size: 5.0M

[1] http://lists.nongnu.org/archive/html/qemu-block/2019-03/msg00761.html

Signed-off-by: Nir Soffer <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: de23e72bb7515888fdea2a58c58a2e02370123bd
      
https://github.com/qemu/qemu/commit/de23e72bb7515888fdea2a58c58a2e02370123bd
  Author: Stefano Garzarella <address@hidden>
  Date:   2019-04-02 (Tue, 02 Apr 2019)

  Changed paths:
    M block/gluster.c

  Log Message:
  -----------
  block/gluster: limit the transfer size to 512 MiB

Several versions of GlusterFS (3.12? -> 6.0.1) fail when the
transfer size is greater or equal to 1024 MiB, so we are
limiting the transfer size to 512 MiB to avoid this rare issue.

Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1691320
Signed-off-by: Stefano Garzarella <address@hidden>
Reviewed-by: Niels de Vos <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 38e694fcc918f0f4ac1500a5c92acfa421a69773
      
https://github.com/qemu/qemu/commit/38e694fcc918f0f4ac1500a5c92acfa421a69773
  Author: Thomas Huth <address@hidden>
  Date:   2019-04-02 (Tue, 02 Apr 2019)

  Changed paths:
    R tests/qemu-iotests/COPYING

  Log Message:
  -----------
  tests/qemu-iotests: Remove redundant COPYING file

The file tests/qemu-iotests/COPYING is the same text as in the
COPYING file in the main directory. So as far as I can see, we don't
need the duplicate here.

Signed-off-by: Thomas Huth <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 696aaaed579ac5bf5fa336216909b46d3d8f07a8
      
https://github.com/qemu/qemu/commit/696aaaed579ac5bf5fa336216909b46d3d8f07a8
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-04-02 (Tue, 02 Apr 2019)

  Changed paths:
    M block/file-posix.c

  Log Message:
  -----------
  block/file-posix: do not fail on unlock bytes

bdrv_replace_child() calls bdrv_check_perm() with error_abort on
loosening permissions. However file-locking operations may fail even
in this case, for example on NFS. And this leads to Qemu crash.

Let's avoid such errors. Note, that we ignore such things anyway on
permission update commit and abort.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 0f0998f6219b25f54f8ab20e3803eff552af3b9c
      
https://github.com/qemu/qemu/commit/0f0998f6219b25f54f8ab20e3803eff552af3b9c
  Author: Alberto Garcia <address@hidden>
  Date:   2019-04-02 (Tue, 02 Apr 2019)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: continue until base is found in bdrv_freeze_backing_chain() et al

All three functions that handle the BdrvChild.frozen attribute walk
the backing chain from 'bs' to 'base' and stop either when 'base' is
found or at the end of the chain if 'base' is NULL.

However if 'base' is not found then the functions return without
errors as if it was NULL.

This is wrong: if the caller passed an incorrect parameter that means
that there is a bug in the code.

Signed-off-by: Alberto Garcia <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 20509c4b8b3fb2b058640d5f2e5f6d3527318792
      
https://github.com/qemu/qemu/commit/20509c4b8b3fb2b058640d5f2e5f6d3527318792
  Author: Alberto Garcia <address@hidden>
  Date:   2019-04-02 (Tue, 02 Apr 2019)

  Changed paths:
    M block/stream.c

  Log Message:
  -----------
  block: freeze the backing chain earlier in stream_start()

Commit 6585493369819a48d34a86d57ec6b97cb5cd9bc0 added code to freeze
the backing chain from 'top' to 'base' for the duration of the
block-stream job.

The problem is that the freezing happens too late in stream_start():
during the bdrv_reopen_set_read_only() call earlier in that function
another job can jump in and remove the base image. If that happens we
have an invalid chain and QEMU crashes.

This patch puts the bdrv_freeze_backing_chain() call at the beginning
of the function.

Signed-off-by: Alberto Garcia <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: d20ba603f2d1462f7c39c6aa1f7e0968a18fda0c
      
https://github.com/qemu/qemu/commit/d20ba603f2d1462f7c39c6aa1f7e0968a18fda0c
  Author: Alberto Garcia <address@hidden>
  Date:   2019-04-02 (Tue, 02 Apr 2019)

  Changed paths:
    M tests/qemu-iotests/030
    M tests/qemu-iotests/030.out

  Log Message:
  -----------
  block: test block-stream with a base node that is used by block-commit

The base node of a block-stream operation indicates the first image
from the backing chain starting from which no data is copied to the
top node.

The block-stream job allows others to use that base image, so a second
block-stream job could be writing to it at the same time. An important
restriction is that the base image must not disappear while the stream
job is ongoing. stream_start() freezes the backing chain from top to
base with that purpose but it does it too late in the code so there is
a race condition there.

This bug was fixed in the previous commit, and this patch contains an
iotest for this scenario.

Signed-off-by: Alberto Garcia <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: f18957b854495129f2e8d6838df3bf47d1c1479c
      
https://github.com/qemu/qemu/commit/f18957b854495129f2e8d6838df3bf47d1c1479c
  Author: Thomas Huth <address@hidden>
  Date:   2019-04-02 (Tue, 02 Apr 2019)

  Changed paths:
    M tests/qemu-iotests/235

  Log Message:
  -----------
  tests/qemu-iotests/235: Allow fallback to tcg

iotest 235 currently only works with KVM - this is bad for systems where
it is not available, e.g. CI pipelines. The test also works when using
"tcg" as accelerator, so we can simply add that to the list of accelerators,
too.

Signed-off-by: Thomas Huth <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 9a363f0bcc65f5805396241d16973d0c62615162
      
https://github.com/qemu/qemu/commit/9a363f0bcc65f5805396241d16973d0c62615162
  Author: Peter Maydell <address@hidden>
  Date:   2019-04-02 (Tue, 02 Apr 2019)

  Changed paths:
    M block.c
    M block/file-posix.c
    M block/gluster.c
    M block/stream.c
    M qemu-img.c
    M tests/qemu-iotests/030
    M tests/qemu-iotests/030.out
    M tests/qemu-iotests/200
    M tests/qemu-iotests/235
    R tests/qemu-iotests/COPYING

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- file-posix: Ignore unlock failure instead of crashing
- gluster: Limit the transfer size to 512 MiB
- stream: Fix backing chain freezing
- qemu-img: Enable BDRV_REQ_MAY_UNMAP for zero writes in convert
- iotests fixes

# gpg: Signature made Tue 02 Apr 2019 13:47:43 BST
# gpg:                using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <address@hidden>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
  tests/qemu-iotests/235: Allow fallback to tcg
  block: test block-stream with a base node that is used by block-commit
  block: freeze the backing chain earlier in stream_start()
  block: continue until base is found in bdrv_freeze_backing_chain() et al
  block/file-posix: do not fail on unlock bytes
  tests/qemu-iotests: Remove redundant COPYING file
  block/gluster: limit the transfer size to 512 MiB
  qemu-img: Enable BDRV_REQ_MAY_UNMAP in convert
  iotests: Fix test 200 on s390x without virtio-pci

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/d61d1a1fb2f1...9a363f0bcc65