[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PATCH 13/17] qcow2: Add new autoclear feature for all zero image
|
From: |
Eric Blake |
|
Subject: |
[PATCH 13/17] qcow2: Add new autoclear feature for all zero image |
|
Date: |
Fri, 31 Jan 2020 11:44:32 -0600 |
With the recent introduction of BDRV_ZERO_OPEN, we can optimize
various qemu-img operations if we know the destination starts life
with all zero content. For an image with no cluster allocations and
no backing file, this was already trivial with BDRV_ZERO_CREATE; but
for a fully preallocated image, it does not scale to crawl through the
entire L1/L2 tree to see if every cluster is currently marked as a
zero cluster. But it is quite easy to add an autoclear bit to the
qcow2 file itself: the bit will be set after newly creating an image
or after qcow2_make_empty, and cleared on any other modification
(including by an older qemu that doesn't recognize the bit).
This patch documents the new bit, independently of implementing the
places in code that should set it (which means that for bisection
purposes, it is safer to still mask the bit out when opening an image
with the bit set).
A few iotests have updated output due to the larger number of named
header features.
Signed-off-by: Eric Blake <address@hidden>
---
RFC: As defined in this patch, I defined the bit to be clear if any
cluster defers to a backing file. But the block layer would handle
things just fine if we instead allowed the bit to be set if all
clusters allocated in this image are zero, even if there are other
clusters not allocated. Or maybe we want TWO bits: one if all
clusters allocated here are known zero, and a second if we know that
there are any clusters that defer to a backing image.
---
block/qcow2.c | 9 +++++++++
block/qcow2.h | 3 +++
docs/interop/qcow2.txt | 12 +++++++++++-
qapi/block-core.json | 4 ++++
tests/qemu-iotests/031.out | 8 ++++----
tests/qemu-iotests/036.out | 4 ++--
tests/qemu-iotests/061.out | 14 +++++++-------
7 files changed, 40 insertions(+), 14 deletions(-)
diff --git a/block/qcow2.c b/block/qcow2.c
index 9f2371925737..20cce9410c84 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2859,6 +2859,11 @@ int qcow2_update_header(BlockDriverState *bs)
.bit = QCOW2_AUTOCLEAR_DATA_FILE_RAW_BITNR,
.name = "raw external data",
},
+ {
+ .type = QCOW2_FEAT_TYPE_AUTOCLEAR,
+ .bit = QCOW2_AUTOCLEAR_ALL_ZERO_BITNR,
+ .name = "all zero",
+ },
};
ret = header_ext_add(buf, QCOW2_EXT_MAGIC_FEATURE_TABLE,
@@ -4874,6 +4879,10 @@ static ImageInfoSpecific
*qcow2_get_specific_info(BlockDriverState *bs,
.corrupt = s->incompatible_features &
QCOW2_INCOMPAT_CORRUPT,
.has_corrupt = true,
+ .all_zero = s->autoclear_features &
+ QCOW2_AUTOCLEAR_ALL_ZERO,
+ .has_all_zero = s->autoclear_features &
+ QCOW2_AUTOCLEAR_ALL_ZERO,
.refcount_bits = s->refcount_bits,
.has_bitmaps = !!bitmaps,
.bitmaps = bitmaps,
diff --git a/block/qcow2.h b/block/qcow2.h
index 094212623257..6fc2d323d753 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -237,11 +237,14 @@ enum {
enum {
QCOW2_AUTOCLEAR_BITMAPS_BITNR = 0,
QCOW2_AUTOCLEAR_DATA_FILE_RAW_BITNR = 1,
+ QCOW2_AUTOCLEAR_ALL_ZERO_BITNR = 2,
QCOW2_AUTOCLEAR_BITMAPS = 1 << QCOW2_AUTOCLEAR_BITMAPS_BITNR,
QCOW2_AUTOCLEAR_DATA_FILE_RAW = 1 <<
QCOW2_AUTOCLEAR_DATA_FILE_RAW_BITNR,
+ QCOW2_AUTOCLEAR_ALL_ZERO = 1 << QCOW2_AUTOCLEAR_ALL_ZERO_BITNR,
QCOW2_AUTOCLEAR_MASK = QCOW2_AUTOCLEAR_BITMAPS
| QCOW2_AUTOCLEAR_DATA_FILE_RAW,
+ /* TODO: Add _ALL_ZERO to _MASK once it is handled correctly */
};
enum qcow2_discard_type {
diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt
index 8510d74c8079..d435363a413c 100644
--- a/docs/interop/qcow2.txt
+++ b/docs/interop/qcow2.txt
@@ -153,7 +153,17 @@ in the description of a field.
File bit (incompatible feature bit 1) is also
set.
- Bits 2-63: Reserved (set to 0)
+ Bit 2: All zero image bit
+ If this bit is set, the entire image reads
+ as all zeroes. This can be useful for
+ detecting just-created images even when
+ clusters are preallocated, which in turn
+ can be used to optimize image copying.
+
+ This bit should not be set if any cluster
+ in the image defers to a backing file.
+
+ Bits 3-63: Reserved (set to 0)
96 - 99: refcount_order
Describes the width of a reference count block entry (width
diff --git a/qapi/block-core.json b/qapi/block-core.json
index ef94a296868f..af837ed5af33 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -71,6 +71,9 @@
# @corrupt: true if the image has been marked corrupt; only valid for
# compat >= 1.1 (since 2.2)
#
+# @all-zero: present and true only if the image is known to read as all
+# zeroes (since 5.0)
+#
# @refcount-bits: width of a refcount entry in bits (since 2.3)
#
# @encrypt: details about encryption parameters; only set if image
@@ -87,6 +90,7 @@
'*data-file-raw': 'bool',
'*lazy-refcounts': 'bool',
'*corrupt': 'bool',
+ '*all-zero': 'bool',
'refcount-bits': 'int',
'*encrypt': 'ImageInfoSpecificQCow2Encryption',
'*bitmaps': ['Qcow2BitmapInfo']
diff --git a/tests/qemu-iotests/031.out b/tests/qemu-iotests/031.out
index 46f97c5a4ea4..bb1afa7b87f6 100644
--- a/tests/qemu-iotests/031.out
+++ b/tests/qemu-iotests/031.out
@@ -117,7 +117,7 @@ header_length 104
Header extension:
magic 0x6803f857
-length 288
+length 336
data <binary>
Header extension:
@@ -150,7 +150,7 @@ header_length 104
Header extension:
magic 0x6803f857
-length 288
+length 336
data <binary>
Header extension:
@@ -164,7 +164,7 @@ No errors were found on the image.
magic 0x514649fb
version 3
-backing_file_offset 0x1d8
+backing_file_offset 0x208
backing_file_size 0x17
cluster_bits 16
size 67108864
@@ -188,7 +188,7 @@ data 'host_device'
Header extension:
magic 0x6803f857
-length 288
+length 336
data <binary>
Header extension:
diff --git a/tests/qemu-iotests/036.out b/tests/qemu-iotests/036.out
index 23b699ce0622..e409acf60e2b 100644
--- a/tests/qemu-iotests/036.out
+++ b/tests/qemu-iotests/036.out
@@ -26,7 +26,7 @@ compatible_features []
autoclear_features [63]
Header extension:
magic 0x6803f857
-length 288
+length 336
data <binary>
@@ -38,7 +38,7 @@ compatible_features []
autoclear_features []
Header extension:
magic 0x6803f857
-length 288
+length 336
data <binary>
*** done
diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out
index 413cc4e0f4ab..d873f79bb606 100644
--- a/tests/qemu-iotests/061.out
+++ b/tests/qemu-iotests/061.out
@@ -26,7 +26,7 @@ header_length 104
Header extension:
magic 0x6803f857
-length 288
+length 336
data <binary>
magic 0x514649fb
@@ -84,7 +84,7 @@ header_length 104
Header extension:
magic 0x6803f857
-length 288
+length 336
data <binary>
magic 0x514649fb
@@ -140,7 +140,7 @@ header_length 104
Header extension:
magic 0x6803f857
-length 288
+length 336
data <binary>
ERROR cluster 5 refcount=0 reference=1
@@ -195,7 +195,7 @@ header_length 104
Header extension:
magic 0x6803f857
-length 288
+length 336
data <binary>
magic 0x514649fb
@@ -264,7 +264,7 @@ header_length 104
Header extension:
magic 0x6803f857
-length 288
+length 336
data <binary>
read 65536/65536 bytes at offset 44040192
@@ -298,7 +298,7 @@ header_length 104
Header extension:
magic 0x6803f857
-length 288
+length 336
data <binary>
ERROR cluster 5 refcount=0 reference=1
@@ -327,7 +327,7 @@ header_length 104
Header extension:
magic 0x6803f857
-length 288
+length 336
data <binary>
read 131072/131072 bytes at offset 0
--
2.24.1
- [PATCH 07/17] gluster: Drop useless has_zero_init callback, (continued)
- [PATCH 07/17] gluster: Drop useless has_zero_init callback, Eric Blake, 2020/01/31
- [PATCH 06/17] block: Improve bdrv_has_zero_init_truncate with backing file, Eric Blake, 2020/01/31
- [PATCH 05/17] block: Don't advertise zero_init_truncate with encryption, Eric Blake, 2020/01/31
- [PATCH 03/17] qcow2: Avoid feature name extension on small cluster size, Eric Blake, 2020/01/31
- [PATCH 08/17] sheepdog: Consistently set bdrv_has_zero_init_truncate, Eric Blake, 2020/01/31
- [PATCH 10/17] block: Add new BDRV_ZERO_OPEN flag, Eric Blake, 2020/01/31
- [PATCH 11/17] file-posix: Support BDRV_ZERO_OPEN, Eric Blake, 2020/01/31
- [PATCH 09/17] block: Refactor bdrv_has_zero_init{,_truncate}, Eric Blake, 2020/01/31
- [PATCH 12/17] gluster: Support BDRV_ZERO_OPEN, Eric Blake, 2020/01/31
- [PATCH 13/17] qcow2: Add new autoclear feature for all zero image,
Eric Blake <=
- [PATCH 15/17] qcow2: Implement all-zero autoclear bit, Eric Blake, 2020/01/31
- [PATCH 14/17] qcow2: Expose all zero bit through .bdrv_known_zeroes, Eric Blake, 2020/01/31
- [PATCH 16/17] iotests: Add new test for qcow2 all-zero bit, Eric Blake, 2020/01/31
- [PATCH 17/17] qcow2: Let qemu-img check cover all-zero bit, Eric Blake, 2020/01/31