[Qemu-commits] [qemu/qemu] 572ad9: qcow2: introduce compression type fea

qemu-commits
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-commits] [qemu/qemu] 572ad9: qcow2: introduce compression type fea

From:	Peter Maydell
Subject:	[Qemu-commits] [qemu/qemu] 572ad9: qcow2: introduce compression type feature
Date:	Thu, 14 May 2020 03:00:35 -0700
  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 572ad9783f3e728a70617ce6bf144c09de3add50
      
https://github.com/qemu/qemu/commit/572ad9783f3e728a70617ce6bf144c09de3add50
  Author: Denis Plotnikov <address@hidden>
  Date:   2020-05-13 (Wed, 13 May 2020)

  Changed paths:
    M block/qcow2.c
    M block/qcow2.h
    M include/block/block_int.h
    M qapi/block-core.json
    M tests/qemu-iotests/031.out
    M tests/qemu-iotests/036.out
    M tests/qemu-iotests/049.out
    M tests/qemu-iotests/060.out
    M tests/qemu-iotests/061.out
    M tests/qemu-iotests/065
    M tests/qemu-iotests/080
    M tests/qemu-iotests/082.out
    M tests/qemu-iotests/085.out
    M tests/qemu-iotests/144.out
    M tests/qemu-iotests/182.out
    M tests/qemu-iotests/185.out
    M tests/qemu-iotests/198.out
    M tests/qemu-iotests/206.out
    M tests/qemu-iotests/242.out
    M tests/qemu-iotests/255.out
    M tests/qemu-iotests/274.out
    M tests/qemu-iotests/280.out
    M tests/qemu-iotests/common.filter

  Log Message:
  -----------
  qcow2: introduce compression type feature

The patch adds some preparation parts for incompatible compression type
feature to qcow2 allowing the use different compression methods for
image clusters (de)compressing.

It is implied that the compression type is set on the image creation and
can be changed only later by image conversion, thus compression type
defines the only compression algorithm used for the image, and thus,
for all image clusters.

The goal of the feature is to add support of other compression methods
to qcow2. For example, ZSTD which is more effective on compression than ZLIB.

The default compression is ZLIB. Images created with ZLIB compression type
are backward compatible with older qemu versions.

Adding of the compression type breaks a number of tests because now the
compression type is reported on image creation and there are some changes
in the qcow2 header in size and offsets.

The tests are fixed in the following ways:
    * filter out compression_type for many tests
    * fix header size, feature table size and backing file offset
      affected tests: 031, 036, 061, 080
      header_size +=8: 1 byte compression type
                       7 bytes padding
      feature_table += 48: incompatible feature compression type
      backing_file_offset += 56 (8 + 48 -> header_change + feature_table_change)
    * add "compression type" for test output matching when it isn't filtered
      affected tests: 049, 060, 061, 065, 082, 085, 144, 182, 185, 198, 206,
                      242, 255, 274, 280

Signed-off-by: Denis Plotnikov <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
QAPI part:
Acked-by: Markus Armbruster <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 25dd077d1d0aaef23f8608468cbef9a396583b1b
      
https://github.com/qemu/qemu/commit/25dd077d1d0aaef23f8608468cbef9a396583b1b
  Author: Denis Plotnikov <address@hidden>
  Date:   2020-05-13 (Wed, 13 May 2020)

  Changed paths:
    M block/qcow2-threads.c

  Log Message:
  -----------
  qcow2: rework the cluster compression routine

The patch enables processing the image compression type defined
for the image and chooses an appropriate method for image clusters
(de)compression.

Signed-off-by: Denis Plotnikov <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: d298ac10add95e2b7f8850332f3b755c8a23d925
      
https://github.com/qemu/qemu/commit/d298ac10add95e2b7f8850332f3b755c8a23d925
  Author: Denis Plotnikov <address@hidden>
  Date:   2020-05-13 (Wed, 13 May 2020)

  Changed paths:
    M block/qcow2-threads.c
    M block/qcow2.c
    M configure
    M docs/interop/qcow2.txt
    M qapi/block-core.json

  Log Message:
  -----------
  qcow2: add zstd cluster compression

zstd significantly reduces cluster compression time.
It provides better compression performance maintaining
the same level of the compression ratio in comparison with
zlib, which, at the moment, is the only compression
method available.

The performance test results:
Test compresses and decompresses qemu qcow2 image with just
installed rhel-7.6 guest.
Image cluster size: 64K. Image on disk size: 2.2G

The test was conducted with brd disk to reduce the influence
of disk subsystem to the test results.
The results is given in seconds.

compress cmd:
  time ./qemu-img convert -O qcow2 -c -o compression_type=[zlib|zstd]
                  src.img [zlib|zstd]_compressed.img
decompress cmd
  time ./qemu-img convert -O qcow2
                  [zlib|zstd]_compressed.img uncompressed.img

           compression               decompression
         zlib       zstd           zlib         zstd
------------------------------------------------------------
real     65.5       16.3 (-75 %)    1.9          1.6 (-16 %)
user     65.0       15.8            5.3          2.5
sys       3.3        0.2            2.0          2.0

Both ZLIB and ZSTD gave the same compression ratio: 1.57
compressed image size in both cases: 1.4G

Signed-off-by: Denis Plotnikov <address@hidden>
QAPI part:
Acked-by: Markus Armbruster <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: dd488fc1c000700741355426198d240c6f25ccb7
      
https://github.com/qemu/qemu/commit/dd488fc1c000700741355426198d240c6f25ccb7
  Author: Denis Plotnikov <address@hidden>
  Date:   2020-05-13 (Wed, 13 May 2020)

  Changed paths:
    A tests/qemu-iotests/287
    A tests/qemu-iotests/287.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  iotests: 287: add qcow2 compression type test

The test checks fulfilling qcow2 requirements for the compression
type feature and zstd compression type operability.

Signed-off-by: Denis Plotnikov <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Tested-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: fc9aefc8c0d3c6392656ea661ce72c1583b70bbd
      
https://github.com/qemu/qemu/commit/fc9aefc8c0d3c6392656ea661ce72c1583b70bbd
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2020-05-13 (Wed, 13 May 2020)

  Changed paths:
    M block/block-copy.c

  Log Message:
  -----------
  block/block-copy: fix use-after-free of task pointer

Obviously, we should g_free the task after trace point and offset
update.

Reported-by: Coverity (CID 1428756)
Fixes: 4ce5dd3e9b5ee0fac18625860eb3727399ee965e
Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: d8f9d57dbd0caf225c47f12e9faea9180e79fe2a
      
https://github.com/qemu/qemu/commit/d8f9d57dbd0caf225c47f12e9faea9180e79fe2a
  Author: Peter Maydell <address@hidden>
  Date:   2020-05-13 (Wed, 13 May 2020)

  Changed paths:
    M block/block-copy.c
    M block/qcow2-threads.c
    M block/qcow2.c
    M block/qcow2.h
    M configure
    M docs/interop/qcow2.txt
    M include/block/block_int.h
    M qapi/block-core.json
    M tests/qemu-iotests/031.out
    M tests/qemu-iotests/036.out
    M tests/qemu-iotests/049.out
    M tests/qemu-iotests/060.out
    M tests/qemu-iotests/061.out
    M tests/qemu-iotests/065
    M tests/qemu-iotests/080
    M tests/qemu-iotests/082.out
    M tests/qemu-iotests/085.out
    M tests/qemu-iotests/144.out
    M tests/qemu-iotests/182.out
    M tests/qemu-iotests/185.out
    M tests/qemu-iotests/198.out
    M tests/qemu-iotests/206.out
    M tests/qemu-iotests/242.out
    M tests/qemu-iotests/255.out
    M tests/qemu-iotests/274.out
    M tests/qemu-iotests/280.out
    A tests/qemu-iotests/287
    A tests/qemu-iotests/287.out
    M tests/qemu-iotests/common.filter
    M tests/qemu-iotests/group

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-05-13' 
into staging

Block patches:
- zstd compression for qcow2
- Fix use-after-free

# gpg: Signature made Wed 13 May 2020 15:14:06 BST
# gpg:                using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg:                issuer "address@hidden"
# gpg: Good signature from "Max Reitz <address@hidden>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2020-05-13:
  block/block-copy: fix use-after-free of task pointer
  iotests: 287: add qcow2 compression type test
  qcow2: add zstd cluster compression
  qcow2: rework the cluster compression routine
  qcow2: introduce compression type feature

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/d5c75ec500d9...d8f9d57dbd0c
[Prev in Thread]
Current Thread
[Next in Thread]
[Qemu-commits] [qemu/qemu] 572ad9: qcow2: introduce compression type feature, Peter Maydell <=
Prev by Date: [Qemu-commits] [qemu/qemu] aadad3: hw/tpm: fix usage of bool in tpm-tis.c
Next by Date: [Qemu-commits] [qemu/qemu] 233982: MAINTAINERS: Upgrade myself as 9pfs co-maintainer
Previous by thread: [Qemu-commits] [qemu/qemu] aadad3: hw/tpm: fix usage of bool in tpm-tis.c
Next by thread: [Qemu-commits] [qemu/qemu] 233982: MAINTAINERS: Upgrade myself as 9pfs co-maintainer
Index(es):
- Date
- Thread