qemu-commits
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Qemu-commits] [qemu/qemu] 672de7: LUKS: support preallocation


From: Peter Maydell
Subject: [Qemu-commits] [qemu/qemu] 672de7: LUKS: support preallocation
Date: Tue, 20 Aug 2019 05:40:17 -0700

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 672de729a1f93d84e7597652b1125ab5d62421d8
      
https://github.com/qemu/qemu/commit/672de729a1f93d84e7597652b1125ab5d62421d8
  Author: Maxim Levitsky <address@hidden>
  Date:   2019-08-19 (Mon, 19 Aug 2019)

  Changed paths:
    M block/crypto.c
    M qapi/block-core.json

  Log Message:
  -----------
  LUKS: support preallocation

preallocation=off and preallocation=metadata
both allocate luks header only, and preallocation=falloc/full
is passed to underlying file.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1534951

Signed-off-by: Maxim Levitsky <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 4d7c487eac1652dfe4498fe84f32900ad461d61b
      
https://github.com/qemu/qemu/commit/4d7c487eac1652dfe4498fe84f32900ad461d61b
  Author: Max Reitz <address@hidden>
  Date:   2019-08-19 (Mon, 19 Aug 2019)

  Changed paths:
    M qemu-img.c

  Log Message:
  -----------
  qemu-img: Fix bdrv_has_zero_init() use in convert

bdrv_has_zero_init() only has meaning for newly created images or image
areas.  If qemu-img convert did not create the image itself, it cannot
rely on bdrv_has_zero_init()'s result to carry any meaning.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Reviewed-by: Stefano Garzarella <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: cdf3bc934ad1e5319b03f2c85f381f5ffd2f8ca8
      
https://github.com/qemu/qemu/commit/cdf3bc934ad1e5319b03f2c85f381f5ffd2f8ca8
  Author: Max Reitz <address@hidden>
  Date:   2019-08-19 (Mon, 19 Aug 2019)

  Changed paths:
    M block/mirror.c
    M blockdev.c
    M include/block/block_int.h
    M tests/test-block-iothread.c

  Log Message:
  -----------
  mirror: Fix bdrv_has_zero_init() use

bdrv_has_zero_init() only has meaning for newly created images or image
areas.  If the mirror job itself did not create the image, it cannot
rely on bdrv_has_zero_init()'s result to carry any meaning.

This is the case for drive-mirror with mode=existing and always for
blockdev-mirror.

Note that we only have to zero-initialize the target with sync=full,
because other modes actually do not promise that the target will contain
the same data as the source after the job -- sync=top only promises to
copy anything allocated in the top layer, and sync=none will only copy
new I/O.  (Which is how mirror has always handled it.)

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: ceaca56feee6a1d682c882300855cad44a67ec8e
      
https://github.com/qemu/qemu/commit/ceaca56feee6a1d682c882300855cad44a67ec8e
  Author: Max Reitz <address@hidden>
  Date:   2019-08-19 (Mon, 19 Aug 2019)

  Changed paths:
    M block.c
    M include/block/block.h
    M include/block/block_int.h

  Log Message:
  -----------
  block: Add bdrv_has_zero_init_truncate()

No .bdrv_has_zero_init() implementation returns 1 if growing the file
would add non-zero areas (at least with PREALLOC_MODE_OFF), so using it
in lieu of this new function was always safe.

But on the other hand, it is possible that growing an image that is not
zero-initialized would still add a zero-initialized area, like when
using nonpreallocating truncation on a preallocated image.  For callers
that care only about truncation, not about creation with potential
preallocation, this new function is useful.

Alternatively, we could have added a PreallocMode parameter to
bdrv_has_zero_init().  But the only user would have been qemu-img
convert, which does not have a plain PreallocMode value right now -- it
would have to parse the creation option to obtain it.  Therefore, the
simpler solution is to let bdrv_has_zero_init() inquire the
preallocation status and add the new bdrv_has_zero_init_truncate() that
presupposes PREALLOC_MODE_OFF.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Reviewed-by: Stefano Garzarella <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 1dcaf52760ccaa2e019164c887b8436ac6c5d8ea
      
https://github.com/qemu/qemu/commit/1dcaf52760ccaa2e019164c887b8436ac6c5d8ea
  Author: Max Reitz <address@hidden>
  Date:   2019-08-19 (Mon, 19 Aug 2019)

  Changed paths:
    M block/file-posix.c
    M block/file-win32.c
    M block/gluster.c
    M block/nfs.c
    M block/qcow2.c
    M block/qed.c
    M block/raw-format.c
    M block/rbd.c
    M block/sheepdog.c
    M block/ssh.c

  Log Message:
  -----------
  block: Implement .bdrv_has_zero_init_truncate()

We need to implement .bdrv_has_zero_init_truncate() for every block
driver that supports truncation and has a .bdrv_has_zero_init()
implementation.

Implement it the same way each driver implements .bdrv_has_zero_init().
This is at least not any more unsafe than what we had before.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Reviewed-by: Stefano Garzarella <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: b647d69adc394312dedb56f6b3ce19b9c316931f
      
https://github.com/qemu/qemu/commit/b647d69adc394312dedb56f6b3ce19b9c316931f
  Author: Max Reitz <address@hidden>
  Date:   2019-08-19 (Mon, 19 Aug 2019)

  Changed paths:
    M block/parallels.c
    M block/vhdx.c

  Log Message:
  -----------
  block: Use bdrv_has_zero_init_truncate()

vhdx and parallels call bdrv_has_zero_init() when they do not really
care about an image's post-create state but only about what happens when
you grow an image.  That is a bit ugly, and also overly safe when
growing preallocated images without preallocating the new areas.

Let them use bdrv_has_zero_init_truncate() instead.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Reviewed-by: Stefano Garzarella <address@hidden>
[mreitz: Added commit message]
Signed-off-by: Max Reitz <address@hidden>


  Commit: 38841dcd27a41e2d8916b7cf4f508a5663f453ff
      
https://github.com/qemu/qemu/commit/38841dcd27a41e2d8916b7cf4f508a5663f453ff
  Author: Max Reitz <address@hidden>
  Date:   2019-08-19 (Mon, 19 Aug 2019)

  Changed paths:
    M block/qcow2.c

  Log Message:
  -----------
  qcow2: Fix .bdrv_has_zero_init()

If a qcow2 file is preallocated, it can no longer guarantee that it
initially appears as filled with zeroes.

So implement .bdrv_has_zero_init() by checking whether the file is
preallocated; if so, forward the call to the underlying storage node,
except for when it is encrypted: Encrypted preallocated images always
return effectively random data, so .bdrv_has_zero_init() must always
return 0 for them.

.bdrv_has_zero_init_truncate() can remain bdrv_has_zero_init_1(),
because it presupposes PREALLOC_MODE_OFF.

Reported-by: Stefano Garzarella <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 0a28bf2826ca027f0d993388f118b0bdbfca73a9
      
https://github.com/qemu/qemu/commit/0a28bf2826ca027f0d993388f118b0bdbfca73a9
  Author: Max Reitz <address@hidden>
  Date:   2019-08-19 (Mon, 19 Aug 2019)

  Changed paths:
    M block/vdi.c

  Log Message:
  -----------
  vdi: Fix .bdrv_has_zero_init()

Static VDI images cannot guarantee to be zero-initialized.  If the image
has been statically allocated, forward the call to the underlying
storage node.

Reported-by: Stefano Garzarella <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Stefan Weil <address@hidden>
Acked-by: Stefano Garzarella <address@hidden>
Tested-by: Stefano Garzarella <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 9956688a8f05ddafb36b9d50d1ff7fbf67464275
      
https://github.com/qemu/qemu/commit/9956688a8f05ddafb36b9d50d1ff7fbf67464275
  Author: Max Reitz <address@hidden>
  Date:   2019-08-19 (Mon, 19 Aug 2019)

  Changed paths:
    M block/vhdx.c

  Log Message:
  -----------
  vhdx: Fix .bdrv_has_zero_init()

Fixed VHDX images cannot guarantee to be zero-initialized.  If the image
has the "fixed" subformat, forward the call to the underlying storage
node.

Reported-by: Stefano Garzarella <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 5a840549ee68a6c2bbcee4c402793375646bdc92
      
https://github.com/qemu/qemu/commit/5a840549ee68a6c2bbcee4c402793375646bdc92
  Author: Max Reitz <address@hidden>
  Date:   2019-08-19 (Mon, 19 Aug 2019)

  Changed paths:
    M tests/qemu-iotests/188
    M tests/qemu-iotests/188.out

  Log Message:
  -----------
  iotests: Convert to preallocated encrypted qcow2

Add a test case for converting an empty image (which only returns zeroes
when read) to a preallocated encrypted qcow2 image.
qcow2_has_zero_init() should return 0 then, thus forcing qemu-img
convert to create zero clusters.

Signed-off-by: Max Reitz <address@hidden>
Acked-by: Stefano Garzarella <address@hidden>
Tested-by: Stefano Garzarella <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: c2acc95befc2e03d8395dead80024a49819b4a23
      
https://github.com/qemu/qemu/commit/c2acc95befc2e03d8395dead80024a49819b4a23
  Author: Max Reitz <address@hidden>
  Date:   2019-08-19 (Mon, 19 Aug 2019)

  Changed paths:
    M tests/qemu-iotests/122
    M tests/qemu-iotests/122.out

  Log Message:
  -----------
  iotests: Test convert -n to pre-filled image

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 9463ee1f5f4e0c496f2f1f3ae641b650f4836627
      
https://github.com/qemu/qemu/commit/9463ee1f5f4e0c496f2f1f3ae641b650f4836627
  Author: Max Reitz <address@hidden>
  Date:   2019-08-19 (Mon, 19 Aug 2019)

  Changed paths:
    M tests/qemu-iotests/041
    M tests/qemu-iotests/041.out

  Log Message:
  -----------
  iotests: Full mirror to existing non-zero image

The result of a sync=full mirror should always be the equal to the
input.  Therefore, existing images should be treated as potentially
non-zero and thus should be explicitly initialized to be zero
beforehand.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: ad6434dc620958503a0ad2ab92269805c26bb806
      
https://github.com/qemu/qemu/commit/ad6434dc620958503a0ad2ab92269805c26bb806
  Author: Max Reitz <address@hidden>
  Date:   2019-08-19 (Mon, 19 Aug 2019)

  Changed paths:
    M block/vdi.c

  Log Message:
  -----------
  vdi: Make block_status recurse for fixed images

Suggested-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Fixes: 69f47505ee66afaa513305de0c1895a224e52c45
Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 4dd84ac9a731f19576aec14d0f2f0037e81a5922
      
https://github.com/qemu/qemu/commit/4dd84ac9a731f19576aec14d0f2f0037e81a5922
  Author: Max Reitz <address@hidden>
  Date:   2019-08-19 (Mon, 19 Aug 2019)

  Changed paths:
    M block/vmdk.c

  Log Message:
  -----------
  vmdk: Make block_status recurse for flat extents

Fixes: 69f47505ee66afaa513305de0c1895a224e52c45
Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: fbc8e1b7e49dee9b137c1037dc72d8e6fd4aba3d
      
https://github.com/qemu/qemu/commit/fbc8e1b7e49dee9b137c1037dc72d8e6fd4aba3d
  Author: Max Reitz <address@hidden>
  Date:   2019-08-19 (Mon, 19 Aug 2019)

  Changed paths:
    M block/vpc.c

  Log Message:
  -----------
  vpc: Do not return RAW from block_status

vpc is not really a passthrough driver, even when using the fixed
subformat (where host and guest offsets are equal).  It should handle
preallocation like all other drivers do, namely by returning
DATA | RECURSE instead of RAW.

There is no tangible difference but the fact that bdrv_is_allocated() no
longer falls through to the protocol layer.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 9c46f4a06d9d57bc9f357cec3e37de94e6068413
      
https://github.com/qemu/qemu/commit/9c46f4a06d9d57bc9f357cec3e37de94e6068413
  Author: Max Reitz <address@hidden>
  Date:   2019-08-19 (Mon, 19 Aug 2019)

  Changed paths:
    M tests/qemu-iotests/141
    M tests/qemu-iotests/141.out
    M tests/qemu-iotests/common.filter

  Log Message:
  -----------
  iotests: Fix 141 when run with qed

69f47505ee has changed qcow2 in such a way that the commit job run in
test 141 (and 144[1]) returns before it emits the READY event.  However,
141 also runs with qed, where the order is still the other way around.
Just filter out the {"return": {}} so the test passes for qed again.

[1] 144 only runs with qcow2, so it is fine as it is.

Suggested-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Fixes: 69f47505ee66afaa513305de0c1895a224e52c45
Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: fa27c478102a6b5d1c6b02c005607ad9404b915f
      
https://github.com/qemu/qemu/commit/fa27c478102a6b5d1c6b02c005607ad9404b915f
  Author: Max Reitz <address@hidden>
  Date:   2019-08-19 (Mon, 19 Aug 2019)

  Changed paths:
    M docs/qemu-block-drivers.texi
    M qapi/block-core.json
    M qemu-img.texi

  Log Message:
  -----------
  doc: Preallocation does not require writing zeroes

When preallocating an encrypted qcow2 image, it just lets the protocol
driver write data and then does not mark the clusters as zero.
Therefore, reading this image will yield effectively random data.

As such, we have not fulfilled the promise of always writing zeroes when
preallocating an image in a while.  It seems that nobody has really
cared, so change the documentation to conform to qemu's actual behavior.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Daniel P. Berrangé <address@hidden>
Reviewed-by: Maxim Levitsky <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 156d320349df5d17e1c4fbf11fad70d2d93f5e26
      
https://github.com/qemu/qemu/commit/156d320349df5d17e1c4fbf11fad70d2d93f5e26
  Author: Peter Maydell <address@hidden>
  Date:   2019-08-20 (Tue, 20 Aug 2019)

  Changed paths:
    M block.c
    M block/crypto.c
    M block/file-posix.c
    M block/file-win32.c
    M block/gluster.c
    M block/mirror.c
    M block/nfs.c
    M block/parallels.c
    M block/qcow2.c
    M block/qed.c
    M block/raw-format.c
    M block/rbd.c
    M block/sheepdog.c
    M block/ssh.c
    M block/vdi.c
    M block/vhdx.c
    M block/vmdk.c
    M block/vpc.c
    M blockdev.c
    M docs/qemu-block-drivers.texi
    M include/block/block.h
    M include/block/block_int.h
    M qapi/block-core.json
    M qemu-img.c
    M qemu-img.texi
    M tests/qemu-iotests/041
    M tests/qemu-iotests/041.out
    M tests/qemu-iotests/122
    M tests/qemu-iotests/122.out
    M tests/qemu-iotests/141
    M tests/qemu-iotests/141.out
    M tests/qemu-iotests/188
    M tests/qemu-iotests/188.out
    M tests/qemu-iotests/common.filter
    M tests/test-block-iothread.c

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-08-19' 
into staging

Block patches:
- preallocation=falloc/full support for LUKS
- Various minor fixes

# gpg: Signature made Mon 19 Aug 2019 16:36:45 BST
# gpg:                using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg:                issuer "address@hidden"
# gpg: Good signature from "Max Reitz <address@hidden>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2019-08-19:
  doc: Preallocation does not require writing zeroes
  iotests: Fix 141 when run with qed
  vpc: Do not return RAW from block_status
  vmdk: Make block_status recurse for flat extents
  vdi: Make block_status recurse for fixed images
  iotests: Full mirror to existing non-zero image
  iotests: Test convert -n to pre-filled image
  iotests: Convert to preallocated encrypted qcow2
  vhdx: Fix .bdrv_has_zero_init()
  vdi: Fix .bdrv_has_zero_init()
  qcow2: Fix .bdrv_has_zero_init()
  block: Use bdrv_has_zero_init_truncate()
  block: Implement .bdrv_has_zero_init_truncate()
  block: Add bdrv_has_zero_init_truncate()
  mirror: Fix bdrv_has_zero_init() use
  qemu-img: Fix bdrv_has_zero_init() use in convert
  LUKS: support preallocation

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/6894576347a7...156d320349df



reply via email to

[Prev in Thread] Current Thread [Next in Thread]