[Qemu-commits] [qemu/qemu] edafc7: qemu-img convert: Don't pre-zero images

[Peter Maydell]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: edafc70c0c8510862f2f213a3acf7067113bcd08
      
https://github.com/qemu/qemu/commit/edafc70c0c8510862f2f213a3acf7067113bcd08
  Author: Kevin Wolf <kwolf@redhat.com>
  Date:   2020-07-03 (Fri, 03 Jul 2020)

  Changed paths:
    M qemu-img.c

  Log Message:
  -----------
  qemu-img convert: Don't pre-zero images

Since commit 5a37b60a61c, qemu-img create will pre-zero the target image
if it isn't already zero-initialised (most importantly, for host block
devices, but also iscsi etc.), so that writing explicit zeros wouldn't
be necessary later.

This could speed up the operation significantly, in particular when the
source image file was only sparsely populated. However, it also means
that some block are written twice: Once when pre-zeroing them, and then
when they are overwritten with actual data. On a full image, the
pre-zeroing is wasted work because everything will be overwritten.

In practice, write_zeroes typically turns out faster than writing
explicit zero buffers, but slow enough that first zeroing everything and
then overwriting parts can be a significant net loss.

Meanwhile, qemu-img convert was rewritten in 690c7301600 and zero blocks
are now written to the target using bdrv_co_pwrite_zeroes() if the
target could be pre-zeroed. This way we already make use of the faster
write_zeroes operation, but avoid writing any blocks twice.

Remove the pre-zeroing because these days this former optimisation has
actually turned into a pessimisation in the common case.

Reported-by: Nir Soffer <nsoffer@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200622151203.35624-1-kwolf@redhat.com>
Tested-by:  Nir Soffer <nsoffer@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>


  Commit: 5b99bdea842d4190d2471769903cdcc1621eb493
      
https://github.com/qemu/qemu/commit/5b99bdea842d4190d2471769903cdcc1621eb493
  Author: Stefan Hajnoczi <stefanha@redhat.com>
  Date:   2020-07-03 (Fri, 03 Jul 2020)

  Changed paths:
    M qemu-storage-daemon.c

  Log Message:
  -----------
  qemu-storage-daemon: remember to add qemu_object_opts

The --object option is supported by qemu-storage-daemon but the
qemu_object_opts QemuOptsList wasn't being added. As a result calls to
qemu_find_opts("object") failed with "There is no option group
'object'".

This patch fixes the object-del QMP command.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200619101132.2401756-2-stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>


  Commit: f10802d2c9fd8bfd92c70f465da1a5992445157f
      
https://github.com/qemu/qemu/commit/f10802d2c9fd8bfd92c70f465da1a5992445157f
  Author: Stefan Hajnoczi <stefanha@redhat.com>
  Date:   2020-07-03 (Fri, 03 Jul 2020)

  Changed paths:
    M qemu-storage-daemon.c

  Log Message:
  -----------
  qemu-storage-daemon: add missing cleanup calls

Several components used by qemu-storage-daemon have cleanup functions
that aren't called. Keep the "valgrind --leak-check=full" as clean as
possible by invoking the necessary cleanup functions.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200619101132.2401756-3-stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>


  Commit: c79e243ed67683d6d06692bd7040f7394da178b0
      
https://github.com/qemu/qemu/commit/c79e243ed67683d6d06692bd7040f7394da178b0
  Author: Kevin Wolf <kwolf@redhat.com>
  Date:   2020-07-03 (Fri, 03 Jul 2020)

  Changed paths:
    M block/vvfat.c

  Log Message:
  -----------
  vvfat: Check that updated filenames are valid

FAT allows only a restricted set of characters in file names, and for
some of the illegal characters, it's actually important that we catch
them: If filenames can contain '/', the guest can construct filenames
containing "../" and escape from the assigned vvfat directory. The same
problem could arise if ".." was ever accepted as a literal filename.

Fix this by adding a check that all filenames are valid in
check_directory_consistency().

Reported-by: Nathan Huckleberry <nhuck15@gmail.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200623175534.38286-2-kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>


  Commit: 3dfa23b9eff4315817c1acf59711b8414347de31
      
https://github.com/qemu/qemu/commit/3dfa23b9eff4315817c1acf59711b8414347de31
  Author: Kevin Wolf <kwolf@redhat.com>
  Date:   2020-07-03 (Fri, 03 Jul 2020)

  Changed paths:
    M block/vvfat.c

  Log Message:
  -----------
  vvfat: Fix array_remove_slice()

array_remove_slice() calls array_roll() with array->next - 1 as the
destination index. This is only correct for count == 1, otherwise we're
writing past the end of the array. array->next - count would be correct.

However, this is the only place ever calling array_roll(), so this
rather complicated operation isn't even necessary.

Fix the problem and simplify the code by replacing it with a single
memmove() call. array_roll() can now be removed.

Reported-by: Nathan Huckleberry <nhuck15@gmail.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200623175534.38286-3-kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>


  Commit: 49438972b8c2e16cf1f98b1b2355926e8c633e63
      
https://github.com/qemu/qemu/commit/49438972b8c2e16cf1f98b1b2355926e8c633e63
  Author: Max Reitz <mreitz@redhat.com>
  Date:   2020-07-03 (Fri, 03 Jul 2020)

  Changed paths:
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests.py: Do not wait() before communicate()

Waiting on a process for which we have a pipe will stall if the process
outputs more data than fits into the OS-provided buffer.  We must use
communicate() before wait(), and in fact, communicate() perfectly
replaces wait() already.

We have to drop the stderr=subprocess.STDOUT parameter from
subprocess.Popen() in qemu_nbd_early_pipe(), because stderr is passed on
to the child process, so if we do not drop this parameter, communicate()
will hang (because the pipe is not closed).

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200630083711.40567-1-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>


  Commit: 4f071a9460886667fde061c05b79dc786cc22e3c
      
https://github.com/qemu/qemu/commit/4f071a9460886667fde061c05b79dc786cc22e3c
  Author: Philippe Mathieu-Daudé <philmd@redhat.com>
  Date:   2020-07-03 (Fri, 03 Jul 2020)

  Changed paths:
    M tests/qemu-iotests/051.pc.out

  Log Message:
  -----------
  iotests: Fix 051 output after qdev_init_nofail() removal

Commit 96927c744 replaced qdev_init_nofail() call by
isa_realize_and_unref() which has a different error
message. Update the test output accordingly.

Gitlab CI error after merging b77b5b3dc7:
https://gitlab.com/qemu-project/qemu/-/jobs/597414772#L4375

Reported-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200616154949.6586-1-philmd@redhat.com>
Message-Id: <20200624140446.15380-2-alex.bennee@linaro.org>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>


  Commit: 7b7515702012219410802a168ae4aa45b72a44df
      
https://github.com/qemu/qemu/commit/7b7515702012219410802a168ae4aa45b72a44df
  Author: Peter Maydell <peter.maydell@linaro.org>
  Date:   2020-07-03 (Fri, 03 Jul 2020)

  Changed paths:
    M block/vvfat.c
    M qemu-img.c
    M qemu-storage-daemon.c
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- qemu-img convert: Don't pre-zero images (removes nowadays
  counterproductive optimisation)
- qemu-storage-daemon: Fix object-del, cleaner shutdown
- vvfat: Check that the guest doesn't escape the given host directory
  with read-write vvfat drives
- vvfat: Fix crash by out-of-bounds array writes for read-write drives
- iotests fixes

# gpg: Signature made Fri 03 Jul 2020 10:20:46 BST
# gpg:                using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg:                issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
  iotests: Fix 051 output after qdev_init_nofail() removal
  iotests.py: Do not wait() before communicate()
  vvfat: Fix array_remove_slice()
  vvfat: Check that updated filenames are valid
  qemu-storage-daemon: add missing cleanup calls
  qemu-storage-daemon: remember to add qemu_object_opts
  qemu-img convert: Don't pre-zero images

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>


Compare: https://github.com/qemu/qemu/compare/5f42c3375d45...7b7515702012