[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-commits] [qemu/qemu] 3ae3fc: docs: improve qcow2 spec about extend
From: |
Peter Maydell |
Subject: |
[Qemu-commits] [qemu/qemu] 3ae3fc: docs: improve qcow2 spec about extending image header |
Date: |
Fri, 21 Feb 2020 06:30:12 -0800 |
Branch: refs/heads/master
Home: https://github.com/qemu/qemu
Commit: 3ae3fcfae533086d3e210589c10f317717c9d789
https://github.com/qemu/qemu/commit/3ae3fcfae533086d3e210589c10f317717c9d789
Author: Vladimir Sementsov-Ogievskiy <address@hidden>
Date: 2020-02-20 (Thu, 20 Feb 2020)
Changed paths:
M docs/interop/qcow2.txt
Log Message:
-----------
docs: improve qcow2 spec about extending image header
Make it more obvious how to add new fields to the version 3 header and
how to interpret them.
The specification is adjusted so that for new defined optional fields:
1. Software may support some of these optional fields and ignore the
others, which means that features may be backported to downstream
Qemu independently.
2. If we want to add incompatible field (or a field, for which some of
its values would be incompatible), it must be accompanied by
incompatible feature bit.
Also the concept of "default is zero" is clarified, as it's strange to
say that the value of the field is assumed to be zero for the software
version which don't know about the field at all and don't know how to
treat it be it zero or not.
Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>
[mreitz: s/some its/some of its/]
Signed-off-by: Max Reitz <address@hidden>
Commit: 66fcbca5a7202d2ff2973b61a30f915204917eb7
https://github.com/qemu/qemu/commit/66fcbca5a7202d2ff2973b61a30f915204917eb7
Author: Vladimir Sementsov-Ogievskiy <address@hidden>
Date: 2020-02-20 (Thu, 20 Feb 2020)
Changed paths:
M docs/interop/qcow2.txt
Log Message:
-----------
docs: qcow2: introduce compression type feature
The patch adds a new additional field to the qcow2 header: compression_type,
which specifies compression type. If field is absent or zero, default
compression type is set: ZLIB, which corresponds to current behavior.
New compression type (ZSTD) is to be added in further commit.
Suggested-by: Denis Plotnikov <address@hidden>
Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>
[mreitz: s/Bits 3-63: Reserved/Bits 4-63: Reserved/]
Signed-off-by: Max Reitz <address@hidden>
Commit: 6756696e38117f9e7c769dd33b339f5ab71f7d47
https://github.com/qemu/qemu/commit/6756696e38117f9e7c769dd33b339f5ab71f7d47
Author: Thomas Huth <address@hidden>
Date: 2020-02-20 (Thu, 20 Feb 2020)
Changed paths:
M tests/qemu-iotests/139
Log Message:
-----------
iotests: Remove the superfluous 2nd check for the availability of quorum
Commit d9df28e7b07 ("iotests: check whitelisted formats") added the
modern @iotests.skip_if_unsupported() to the functions in this test,
so we don't need the old explicit test here anymore.
Signed-off-by: Thomas Huth <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>
Reviewed-by: Andrey Shinkevich <address@hidden>
Tested-by: Andrey Shinkevich <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Commit: b0c4cf21b0f26533e4aa6ca1930b2c9f52371bfa
https://github.com/qemu/qemu/commit/b0c4cf21b0f26533e4aa6ca1930b2c9f52371bfa
Author: Max Reitz <address@hidden>
Date: 2020-02-20 (Thu, 20 Feb 2020)
Changed paths:
M tests/qemu-iotests/147
Log Message:
-----------
iotests/147: Fix drive parameters
8dff69b94 added an aio parameter to the drive parameter but forgot to
add a comma before, thus breaking the test. Fix it again.
Fixes: 8dff69b9415b4287e900358744b732195e1ab2e2
Signed-off-by: Max Reitz <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Tested-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Commit: facda5443f5a8676fb635b82ac1046ac6b6a67ce
https://github.com/qemu/qemu/commit/facda5443f5a8676fb635b82ac1046ac6b6a67ce
Author: Peter Krempa <address@hidden>
Date: 2020-02-20 (Thu, 20 Feb 2020)
Changed paths:
M block.c
M block/qapi.c
M blockdev.c
M include/block/block.h
M include/block/qapi.h
M monitor/hmp-cmds.c
M qapi/block-core.json
Log Message:
-----------
qapi: Allow getting flat output from 'query-named-block-nodes'
When a management application manages node names there's no reason to
recurse into backing images in the output of query-named-block-nodes.
Add a parameter to the command which will return just the top level
structs.
Signed-off-by: Peter Krempa <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
[mreitz: Fixed coding style]
Signed-off-by: Max Reitz <address@hidden>
Commit: 168468fe19c872d31d17581e6a6cf7b69bd105b1
https://github.com/qemu/qemu/commit/168468fe19c872d31d17581e6a6cf7b69bd105b1
Author: David Edmondson <address@hidden>
Date: 2020-02-20 (Thu, 20 Feb 2020)
Changed paths:
M docs/interop/qemu-img.rst
M qemu-img-cmds.hx
M qemu-img.c
Log Message:
-----------
qemu-img: Add --target-is-zero to convert
In many cases the target of a convert operation is a newly provisioned
target that the user knows is blank (reads as zero). In this situation
there is no requirement for qemu-img to wastefully zero out the entire
device.
Add a new option, --target-is-zero, allowing the user to indicate that
an existing target device will return zeros for all reads.
Signed-off-by: David Edmondson <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Commit: 087ab8e775f48766068e65de1bc99d03b40d1670
https://github.com/qemu/qemu/commit/087ab8e775f48766068e65de1bc99d03b40d1670
Author: Daniel P. Berrangé <address@hidden>
Date: 2020-02-20 (Thu, 20 Feb 2020)
Changed paths:
M block/qcow2.c
A tests/qemu-iotests/284
A tests/qemu-iotests/284.out
M tests/qemu-iotests/group
Log Message:
-----------
block: always fill entire LUKS header space with zeros
When initializing the LUKS header the size with default encryption
parameters will currently be 2068480 bytes. This is rounded up to
a multiple of the cluster size, 2081792, with 64k sectors. If the
end of the header is not the same as the end of the cluster we fill
the extra space with zeros. This was forgetting that not even the
space allocated for the header will be fully initialized, as we
only write key material for the first key slot. The space left
for the other 7 slots is never written to.
An optimization to the ref count checking code:
commit a5fff8d4b4d928311a5005efa12d0991fe3b66f9 (refs/bisect/bad)
Author: Vladimir Sementsov-Ogievskiy <address@hidden>
Date: Wed Feb 27 16:14:30 2019 +0300
qcow2-refcount: avoid eating RAM
made the assumption that every cluster which was allocated would
have at least some data written to it. This was violated by way
the LUKS header is only partially written, with much space simply
reserved for future use.
Depending on the cluster size this problem was masked by the
logic which wrote zeros between the end of the LUKS header and
the end of the cluster.
$ qemu-img create --object secret,id=cluster_encrypt0,data=123456 \
-f qcow2 -o cluster_size=2k,encrypt.iter-time=1,\
encrypt.format=luks,encrypt.key-secret=cluster_encrypt0 \
cluster_size_check.qcow2 100M
Formatting 'cluster_size_check.qcow2', fmt=qcow2 size=104857600
encrypt.format=luks encrypt.key-secret=cluster_encrypt0
encrypt.iter-time=1 cluster_size=2048 lazy_refcounts=off refcount_bits=16
$ qemu-img check --object secret,id=cluster_encrypt0,data=redhat \
'json:{"driver": "qcow2", "encrypt.format": "luks", \
"encrypt.key-secret": "cluster_encrypt0", \
"file.driver": "file", "file.filename": "cluster_size_check.qcow2"}'
ERROR: counting reference for region exceeding the end of the file by one
cluster or more: offset 0x2000 size 0x1f9000
Leaked cluster 4 refcount=1 reference=0
...snip...
Leaked cluster 130 refcount=1 reference=0
1 errors were found on the image.
Data may be corrupted, or further writes to the image may corrupt it.
127 leaked clusters were found on the image.
This means waste of disk space, but no harm to data.
Image end offset: 268288
The problem only exists when the disk image is entirely empty. Writing
data to the disk image payload will solve the problem by causing the
end of the file to be extended further.
The change fixes it by ensuring that the entire allocated LUKS header
region is fully initialized with zeros. The qemu-img check will still
fail for any pre-existing disk images created prior to this change,
unless at least 1 byte of the payload is written to.
Fully writing zeros to the entire LUKS header is a good idea regardless
as it ensures that space has been allocated on the host filesystem (or
whatever block storage backend is used).
Signed-off-by: Daniel P. Berrangé <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Commit: 4bc267a7c7cf204b05d3d1f01625691540a7b965
https://github.com/qemu/qemu/commit/4bc267a7c7cf204b05d3d1f01625691540a7b965
Author: Vladimir Sementsov-Ogievskiy <address@hidden>
Date: 2020-02-20 (Thu, 20 Feb 2020)
Changed paths:
M block/backup-top.c
Log Message:
-----------
block/backup-top: fix flags handling
backup-top "supports" write-unchanged, by skipping CBW operation in
backup_top_co_pwritev. But it forgets to do the same in
backup_top_co_pwrite_zeroes, as well as declare support for
BDRV_REQ_WRITE_UNCHANGED.
Fix this, and, while being here, declare also support for flags
supported by source child.
Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Commit: b7e9eae98c160f3e7eb9ffbedba8821e358d963c
https://github.com/qemu/qemu/commit/b7e9eae98c160f3e7eb9ffbedba8821e358d963c
Author: Max Reitz <address@hidden>
Date: 2020-02-20 (Thu, 20 Feb 2020)
Changed paths:
M tests/qemu-iotests/279
Log Message:
-----------
iotests/279: Fix for non-qcow2 formats
First, driver=qcow2 will not work so well with non-qcow2 formats (and
this test claims to support qcow, qed, and vmdk).
Second, vmdk will always report the backing file format to be vmdk.
Filter that out so the output looks like for all other formats.
Third, the flat vmdk subformats do not support backing files, so they
will not work with this test.
Signed-off-by: Max Reitz <address@hidden>
Message-Id: <address@hidden>
Tested-by: Thomas Huth <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Commit: 78c81a3f108870d325b0a39d88711366afe6f703
https://github.com/qemu/qemu/commit/78c81a3f108870d325b0a39d88711366afe6f703
Author: Max Reitz <address@hidden>
Date: 2020-02-20 (Thu, 20 Feb 2020)
Changed paths:
M block/nbd.c
Log Message:
-----------
block/nbd: Fix hang in .bdrv_close()
When nbd_close() is called from a coroutine, the connection_co never
gets to run, and thus nbd_teardown_connection() hangs.
This is because aio_co_enter() only puts the connection_co into the main
coroutine's wake-up queue, so this main coroutine needs to yield and
wait for connection_co to terminate.
Suggested-by: Kevin Wolf <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Maxim Levitsky <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Commit: fd17146cd93d1704cd96d7c2757b325fc7aac6fd
https://github.com/qemu/qemu/commit/fd17146cd93d1704cd96d7c2757b325fc7aac6fd
Author: Max Reitz <address@hidden>
Date: 2020-02-20 (Thu, 20 Feb 2020)
Changed paths:
M block.c
Log Message:
-----------
block: Generic file creation fallback
If a protocol driver does not support image creation, we can see whether
maybe the file exists already. If so, just truncating it will be
sufficient.
Signed-off-by: Max Reitz <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Commit: 87ca3b8fa615b278b33cabf9ed22b3f44b5214ba
https://github.com/qemu/qemu/commit/87ca3b8fa615b278b33cabf9ed22b3f44b5214ba
Author: Max Reitz <address@hidden>
Date: 2020-02-20 (Thu, 20 Feb 2020)
Changed paths:
M block/file-posix.c
Log Message:
-----------
file-posix: Drop hdev_co_create_opts()
The generic fallback implementation effectively does the same.
Reviewed-by: Maxim Levitsky <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Commit: 80f0900905b555f00d644894c786b6d66ac2e00e
https://github.com/qemu/qemu/commit/80f0900905b555f00d644894c786b6d66ac2e00e
Author: Max Reitz <address@hidden>
Date: 2020-02-20 (Thu, 20 Feb 2020)
Changed paths:
M block/iscsi.c
Log Message:
-----------
iscsi: Drop iscsi_co_create_opts()
The generic fallback implementation effectively does the same.
Reviewed-by: Maxim Levitsky <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Commit: 4dddeac115c5a2c5f74731fda0afd031a0b45490
https://github.com/qemu/qemu/commit/4dddeac115c5a2c5f74731fda0afd031a0b45490
Author: Max Reitz <address@hidden>
Date: 2020-02-20 (Thu, 20 Feb 2020)
Changed paths:
A tests/qemu-iotests/259
A tests/qemu-iotests/259.out
M tests/qemu-iotests/group
Log Message:
-----------
iotests: Add test for image creation fallback
Signed-off-by: Max Reitz <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Maxim Levitsky <address@hidden>
[mreitz: Added a note that NBD does not support resizing, which is why
the second case is expected to fail]
Signed-off-by: Max Reitz <address@hidden>
Commit: c69291e712ae4ef95f628424db6586473da61d43
https://github.com/qemu/qemu/commit/c69291e712ae4ef95f628424db6586473da61d43
Author: Max Reitz <address@hidden>
Date: 2020-02-20 (Thu, 20 Feb 2020)
Changed paths:
M qemu-img.c
Log Message:
-----------
qemu-img: Fix convert -n -B for backing-less targets
s.target_has_backing does not reflect whether the target BDS has a
backing file; it only tells whether we should use a backing file during
conversion (specified by -B).
As such, if you use convert -n, the target does not necessarily actually
have a backing file, and then dereferencing out_bs->backing fails here.
When converting to an existing file, we should set
target_backing_sectors to a negative value, because first, as the
comment explains, this value is only used for optimization, so it is
always fine to do that.
Second, we use this value to determine where the target must be
initialized to zeroes (overlays are initialized to zero after the end of
their backing file). When converting to an existing file, we cannot
assume that to be true.
Cc: address@hidden
Fixes: 351c8efff9ad809c822d55620df54d575d536f68
("qemu-img: Special post-backing convert handling")
Signed-off-by: Max Reitz <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Commit: f535cc90b50b1be2e6c204b844eab991c872b78b
https://github.com/qemu/qemu/commit/f535cc90b50b1be2e6c204b844eab991c872b78b
Author: Max Reitz <address@hidden>
Date: 2020-02-20 (Thu, 20 Feb 2020)
Changed paths:
M tests/qemu-iotests/122
M tests/qemu-iotests/122.out
Log Message:
-----------
iotests: Test convert -n -B to backing-less target
This must not crash.
Signed-off-by: Max Reitz <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Commit: 804359b8b90f76d9d8fbe8d85a6544b68f107f10
https://github.com/qemu/qemu/commit/804359b8b90f76d9d8fbe8d85a6544b68f107f10
Author: Max Reitz <address@hidden>
Date: 2020-02-20 (Thu, 20 Feb 2020)
Changed paths:
M block/qapi.c
Log Message:
-----------
block: Fix VM size field width in snapshot dump
When printing the snapshot list (e.g. with qemu-img snapshot -l), the VM
size field is only seven characters wide. As of de38b5005e9, this is
not necessarily sufficient: We generally print three digits, and this
may require a decimal point. Also, the unit field grew from something
as plain as "M" to " MiB". This means that number and unit may take up
eight characters in total; but we also want spaces in front.
Considering previously the maximum width was four characters and the
field width was chosen to be three characters wider, let us adjust the
field width to be eleven now.
Fixes: de38b5005e946aa3714963ea4c501e279e7d3666
Buglink: https://bugs.launchpad.net/qemu/+bug/1859989
Signed-off-by: Max Reitz <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Commit: dff8d44c96f128480430b0c59ed8760917dbd427
https://github.com/qemu/qemu/commit/dff8d44c96f128480430b0c59ed8760917dbd427
Author: Max Reitz <address@hidden>
Date: 2020-02-20 (Thu, 20 Feb 2020)
Changed paths:
A tests/qemu-iotests/286
A tests/qemu-iotests/286.out
M tests/qemu-iotests/group
Log Message:
-----------
iotests: Test snapshot -l field separation
Add a test that all fields in "qemu-img snapshot -l"s output are
separated by spaces.
Signed-off-by: Max Reitz <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
[mreitz: Renamed test from 284 to 286]
Signed-off-by: Max Reitz <address@hidden>
Commit: 9e6b7f7eb89580cd8dfc1dd4983db9c158517d0a
https://github.com/qemu/qemu/commit/9e6b7f7eb89580cd8dfc1dd4983db9c158517d0a
Author: Peter Maydell <address@hidden>
Date: 2020-02-21 (Fri, 21 Feb 2020)
Changed paths:
M block.c
M block/backup-top.c
M block/file-posix.c
M block/iscsi.c
M block/nbd.c
M block/qapi.c
M block/qcow2.c
M blockdev.c
M docs/interop/qcow2.txt
M docs/interop/qemu-img.rst
M include/block/block.h
M include/block/qapi.h
M monitor/hmp-cmds.c
M qapi/block-core.json
M qemu-img-cmds.hx
M qemu-img.c
M tests/qemu-iotests/122
M tests/qemu-iotests/122.out
M tests/qemu-iotests/139
M tests/qemu-iotests/147
A tests/qemu-iotests/259
A tests/qemu-iotests/259.out
M tests/qemu-iotests/279
A tests/qemu-iotests/284
A tests/qemu-iotests/284.out
A tests/qemu-iotests/286
A tests/qemu-iotests/286.out
M tests/qemu-iotests/group
Log Message:
-----------
Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-02-20'
into staging
Block patches:
- qemu-img convert: New --target-is-zero parameter
- qcow2: Specify non-default compression type flag
- optionally flat output for query-named-block-nodes
- some fixes
- pseudo-creation of images on block devices is now done by a generic
block layer function
# gpg: Signature made Thu 20 Feb 2020 16:05:34 GMT
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "address@hidden"
# gpg: Good signature from "Max Reitz <address@hidden>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40
* remotes/maxreitz/tags/pull-block-2020-02-20:
iotests: Test snapshot -l field separation
block: Fix VM size field width in snapshot dump
iotests: Test convert -n -B to backing-less target
qemu-img: Fix convert -n -B for backing-less targets
iotests: Add test for image creation fallback
iscsi: Drop iscsi_co_create_opts()
file-posix: Drop hdev_co_create_opts()
block: Generic file creation fallback
block/nbd: Fix hang in .bdrv_close()
iotests/279: Fix for non-qcow2 formats
block/backup-top: fix flags handling
block: always fill entire LUKS header space with zeros
qemu-img: Add --target-is-zero to convert
qapi: Allow getting flat output from 'query-named-block-nodes'
iotests/147: Fix drive parameters
iotests: Remove the superfluous 2nd check for the availability of quorum
docs: qcow2: introduce compression type feature
docs: improve qcow2 spec about extending image header
Signed-off-by: Peter Maydell <address@hidden>
Compare: https://github.com/qemu/qemu/compare/b651b80822fa...9e6b7f7eb895
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- [Qemu-commits] [qemu/qemu] 3ae3fc: docs: improve qcow2 spec about extending image header,
Peter Maydell <=