[Qemu-commits] [qemu/qemu] bb87e4: block: add bdrv_refresh

qemu-commits
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-commits] [qemu/qemu] bb87e4: block: add bdrv_refresh_perms() helpe

From:	Peter Maydell
Subject:	[Qemu-commits] [qemu/qemu] bb87e4: block: add bdrv_refresh_perms() helper
Date:	Thu, 31 Dec 2020 15:53:11 -0800
  Branch: refs/heads/staging
  Home:   https://github.com/qemu/qemu
  Commit: bb87e4d1c0c34d55c6fdecaf20b6f9640498e1ad
      
https://github.com/qemu/qemu/commit/bb87e4d1c0c34d55c6fdecaf20b6f9640498e1ad
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: add bdrv_refresh_perms() helper

Make separate function for common pattern.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201106124241.16950-5-vsementsov@virtuozzo.com>
[mreitz: Squashed in
https://lists.nongnu.org/archive/html/qemu-block/2020-11/msg00299.html]
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: 74ad9a3b4da3bc2475c765d1926c655e9f8a0a20
      
https://github.com/qemu/qemu/commit/74ad9a3b4da3bc2475c765d1926c655e9f8a0a20
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: bdrv_set_perm() drop redundant parameters.

We should never set permissions other than cumulative permissions of
parents. During bdrv_reopen_multiple() we _check_ for synthetic
permissions but when we do _set_ the graph is already updated.
Add an assertion to bdrv_reopen_multiple(), other cases are more
obvious.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201106124241.16950-6-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: 6e0c916cc8f171ec69148d8e35d4435f14a5f9b6
      
https://github.com/qemu/qemu/commit/6e0c916cc8f171ec69148d8e35d4435f14a5f9b6
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: bdrv_child_set_perm() drop redundant parameters.

We must set the permission used for _check_.  Assert that we have
backup and drop extra arguments.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201106124241.16950-7-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: 071b474f5457c166f90a2c379508dc92b92dc1ae
      
https://github.com/qemu/qemu/commit/071b474f5457c166f90a2c379508dc92b92dc1ae
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: drop tighten_restrictions

The only users of this thing are:
 1. bdrv_child_try_set_perm, to ignore failures on loosen restrictions
 2. assertion in bdrv_replace_child
 3. assertion in bdrv_inactivate_recurse

Assertions are not enough reason for overcomplication the permission
update system. So, look at bdrv_child_try_set_perm.

We are interested in tighten_restrictions only on failure. But on
failure this field is not reliable: we may fail in the middle of
permission update, some nodes are not touched and we don't know should
their permissions be tighten or not. So, we rely on the fact that if we
loose restrictions on some node (or BdrvChild), we'll not tighten
restriction in the whole subtree as part of this update (assertions 2
and 3 rely on this fact as well). And, if we rely on this fact anyway,
we can just check it on top, and don't pass additional pointer through
the whole recursive infrastructure.

Note also, that further patches will fix real bugs in permission update
system, so now is good time to simplify it, as a help for further
refactorings.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201106124241.16950-8-vsementsov@virtuozzo.com>
[mreitz: Fixed rebase conflict]
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: 2153994e2efad92092a60bf6d1947f4a721ae46d
      
https://github.com/qemu/qemu/commit/2153994e2efad92092a60bf6d1947f4a721ae46d
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M include/block/block.h

  Log Message:
  -----------
  block: simplify comment to BDRV_REQ_SERIALISING

1. BDRV_REQ_NO_SERIALISING doesn't exist already, don't mention it.

2. We are going to add one more user of BDRV_REQ_SERIALISING, so
   comment about backup becomes a bit confusing here. The use case in
   backup is documented in block/backup.c, so let's just drop
   duplication here.

3. The fact that BDRV_REQ_SERIALISING is only for write requests is
   omitted. Add a note.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-Id: <20201021145859.11201-2-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: 2e36da62cf36d83897f922969e4523317e5f6ad3
      
https://github.com/qemu/qemu/commit/2e36da62cf36d83897f922969e4523317e5f6ad3
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M block/io.c

  Log Message:
  -----------
  block/io.c: drop assertion on double waiting for request serialisation

The comments states, that on misaligned request we should have already
been waiting. But for bdrv_padding_rmw_read, we called
bdrv_mark_request_serialising with align = request_alignment, and now
we serialise with align = cluster_size. So we may have to wait again
with larger alignment.

Note, that the only user of BDRV_REQ_SERIALISING is backup which issues
cluster-aligned requests, so seems the assertion should not fire for
now. But it's wrong anyway.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201021145859.11201-3-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: 3183937ff943f0bd0c43bb2625be1a040677ce25
      
https://github.com/qemu/qemu/commit/3183937ff943f0bd0c43bb2625be1a040677ce25
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M block/io.c

  Log Message:
  -----------
  block/io: split out bdrv_find_conflicting_request

To be reused in separate.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20201021145859.11201-4-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: ec1c8868316f9ef33baa695400cb13cf19d1dc78
      
https://github.com/qemu/qemu/commit/ec1c8868316f9ef33baa695400cb13cf19d1dc78
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M block/io.c

  Log Message:
  -----------
  block/io: bdrv_wait_serialising_requests_locked: drop extra bs arg

bs is linked in req, so no needs to pass it separately. Most of
tracked-requests API doesn't have bs argument. Actually, after this
patch only tracked_request_begin has it, but it's for purpose.

While being here, also add a comment about what "_locked" is.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20201021145859.11201-5-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: 8ac5aab255d5c55d21bb33f4f6dd6dc58319e512
      
https://github.com/qemu/qemu/commit/8ac5aab255d5c55d21bb33f4f6dd6dc58319e512
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M block/file-posix.c
    M block/io.c
    M include/block/block_int.h

  Log Message:
  -----------
  block: bdrv_mark_request_serialising: split non-waiting function

We'll need a separate function, which will only "mark" request
serialising with specified align but not wait for conflicting
requests. So, it will be like old bdrv_mark_request_serialising(),
before merging bdrv_wait_serialising_requests_locked() into it.

To reduce the possible mess, let's do the following:

Public function that does both marking and waiting will be called
bdrv_make_request_serialising, and private function which will only
"mark" will be called tracked_request_set_serialising().

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201021145859.11201-6-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: d1a764d126aa9bd1b519855607f55daf266b07bf
      
https://github.com/qemu/qemu/commit/d1a764d126aa9bd1b519855607f55daf266b07bf
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M block/io.c
    M include/block/block.h

  Log Message:
  -----------
  block: introduce BDRV_REQ_NO_WAIT flag

Add flag to make serialising request no wait: if there are conflicting
requests, just return error immediately. It's will be used in upcoming
preallocate filter.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201021145859.11201-7-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: 9530a25b8b7eb5cc1800b66ee617610cd43f0fad
      
https://github.com/qemu/qemu/commit/9530a25b8b7eb5cc1800b66ee617610cd43f0fad
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: bdrv_check_perm(): process children anyway

Do generic processing even for drivers which define .bdrv_check_perm
handler. It's needed for further preallocate filter: it will need to do
additional action on bdrv_check_perm, but don't want to reimplement
generic logic.

The patch doesn't change existing behaviour: the only driver that
implements bdrv_check_perm is file-posix, but it never has any
children.

Also, bdrv_set_perm() don't stop processing if driver has
.bdrv_set_perm handler as well.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-8-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: 33fa2222eb044147e75e5ec395e1fd53328bc9fb
      
https://github.com/qemu/qemu/commit/33fa2222eb044147e75e5ec395e1fd53328bc9fb
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M block/meson.build
    A block/preallocate.c
    M docs/system/qemu-block-drivers.rst.inc
    M qapi/block-core.json

  Log Message:
  -----------
  block: introduce preallocate filter

It's intended to be inserted between format and protocol nodes to
preallocate additional space (expanding protocol file) on writes
crossing EOF. It improves performance for file-systems with slow
allocation.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-9-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
[mreitz: Two comment fixes, and bumped the version from 5.2 to 6.0]
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: 42ba0225bdb133b63cb71f20f478ed14b49b1ede
      
https://github.com/qemu/qemu/commit/42ba0225bdb133b63cb71f20f478ed14b49b1ede
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M qemu-io-cmds.c

  Log Message:
  -----------
  qemu-io: add preallocate mode parameter for truncate command

This will be used in further test.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201021145859.11201-10-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: 5b66a8fc2012bdbe767aece12874fdbf3e9df50e
      
https://github.com/qemu/qemu/commit/5b66a8fc2012bdbe767aece12874fdbf3e9df50e
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests: qemu_io_silent: support --image-opts

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201021145859.11201-11-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: 18654716455b295a8bc18863da988f5ed7db259f
      
https://github.com/qemu/qemu/commit/18654716455b295a8bc18863da988f5ed7db259f
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests.py: execute_setup_common(): add required_fmts argument

Add a parameter to skip test if some needed additional formats are not
supported (for example filter drivers).

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-12-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: d2ace2b95ff35f71a532bc1f5c4cb60971feb4a8
      
https://github.com/qemu/qemu/commit/d2ace2b95ff35f71a532bc1f5c4cb60971feb4a8
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    A tests/qemu-iotests/298
    A tests/qemu-iotests/298.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  iotests: add 298 to test new preallocate filter driver

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-13-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: 270124e7efcaaef68c492d1293af975992138606
      
https://github.com/qemu/qemu/commit/270124e7efcaaef68c492d1293af975992138606
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M scripts/simplebench/simplebench.py

  Log Message:
  -----------
  scripts/simplebench: fix grammar: s/successed/succeeded/

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-14-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: 4a44554a65dfc5ccd5dad428981d0ab2959b4b8f
      
https://github.com/qemu/qemu/commit/4a44554a65dfc5ccd5dad428981d0ab2959b4b8f
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M scripts/simplebench/simplebench.py

  Log Message:
  -----------
  scripts/simplebench: support iops

Support benchmarks returning not seconds but iops. We'll use it for
further new test.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-15-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: f52e1af0b08af93b5354fe2648eccaec6bb8a2b2
      
https://github.com/qemu/qemu/commit/f52e1af0b08af93b5354fe2648eccaec6bb8a2b2
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M scripts/simplebench/simplebench.py

  Log Message:
  -----------
  scripts/simplebench: use standard deviation for +- error

Standard deviation is more usual to see after +- than current maximum
of deviations.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-16-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: bfccfa62ac771400a146dfe768a900f9f6e64467
      
https://github.com/qemu/qemu/commit/bfccfa62ac771400a146dfe768a900f9f6e64467
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M scripts/simplebench/bench-example.py
    M scripts/simplebench/bench_write_req.py
    M scripts/simplebench/simplebench.py

  Log Message:
  -----------
  simplebench: rename ascii() to results_to_text()

Next patch will use utf8 plus-minus symbol, let's use more generic (and
more readable) name.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-17-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: 8e979febb01222edb1e53fb61a93a4c803924869
      
https://github.com/qemu/qemu/commit/8e979febb01222edb1e53fb61a93a4c803924869
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M scripts/simplebench/bench-example.py
    M scripts/simplebench/bench_write_req.py
    A scripts/simplebench/results_to_text.py
    M scripts/simplebench/simplebench.py

  Log Message:
  -----------
  simplebench: move results_to_text() into separate file

Let's keep view part in separate: this way it's better to improve it in
the following commits.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-18-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: 96be1aeec73a53364a0a95cd24a9cb70a973a0fd
      
https://github.com/qemu/qemu/commit/96be1aeec73a53364a0a95cd24a9cb70a973a0fd
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M scripts/simplebench/results_to_text.py

  Log Message:
  -----------
  simplebench/results_to_text: improve view of the table

Move to generic format for floats and percentage for error.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-19-vsementsov@virtuozzo.com>
Acked-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: aa362403f46848c4377ffa9702008e6a2d5f876e
      
https://github.com/qemu/qemu/commit/aa362403f46848c4377ffa9702008e6a2d5f876e
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M scripts/simplebench/results_to_text.py

  Log Message:
  -----------
  simplebench/results_to_text: add difference line to the table

Performance improvements / degradations are usually discussed in
percentage. Let's make the script calculate it for us.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-20-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
[mreitz: 'seconds' instead of 'secs']
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: 181f60c8c73e60af89b42483b54c14dfebfbc384
      
https://github.com/qemu/qemu/commit/181f60c8c73e60af89b42483b54c14dfebfbc384
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M scripts/simplebench/results_to_text.py

  Log Message:
  -----------
  simplebench/results_to_text: make executable

Make results_to_text a tool to dump results saved in JSON file.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-21-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: cff6d3ca43cdc8da0104204a52b0e4bd644e16e1
      
https://github.com/qemu/qemu/commit/cff6d3ca43cdc8da0104204a52b0e4bd644e16e1
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    A scripts/simplebench/bench_prealloc.py

  Log Message:
  -----------
  scripts/simplebench: add bench_prealloc.py

Benchmark for new preallocate filter.

Example usage:
    ./bench_prealloc.py ../../build/qemu-img \
        ssd-ext4:/path/to/mount/point \
        ssd-xfs:/path2 hdd-ext4:/path3 hdd-xfs:/path4

The benchmark shows performance improvement (or degradation) when use
new preallocate filter with qcow2 image.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-22-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: ef9bba1484bb8fb5fda53a7bf90bf5e1a8e6a9f6
      
https://github.com/qemu/qemu/commit/ef9bba1484bb8fb5fda53a7bf90bf5e1a8e6a9f6
  Author: Alberto Garcia <berto@igalia.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M block/quorum.c
    A tests/qemu-iotests/312
    A tests/qemu-iotests/312.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  quorum: Implement bdrv_co_block_status()

The quorum driver does not implement bdrv_co_block_status() and
because of that it always reports to contain data even if all its
children are known to be empty.

One consequence of this is that if we for example create a quorum with
a size of 10GB and we mirror it to a new image the operation will
write 10GB of actual zeroes to the destination image wasting a lot of
time and disk space.

Since a quorum has an arbitrary number of children of potentially
different formats there is no way to report all possible allocation
status flags in a way that makes sense, so this implementation only
reports when a given region is known to contain zeroes
(BDRV_BLOCK_ZERO) or not (BDRV_BLOCK_DATA).

If all children agree that a region contains zeroes then we can return
BDRV_BLOCK_ZERO using the smallest size reported by the children
(because all agree that a region of at least that size contains
zeroes).

If at least one child disagrees we have to return BDRV_BLOCK_DATA.
In this case we use the largest of the sizes reported by the children
that didn't return BDRV_BLOCK_ZERO (because we know that there won't
be an agreement for at least that size).

Signed-off-by: Alberto Garcia <berto@igalia.com>
Tested-by: Tao Xu <tao3.xu@intel.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: 
<db83149afcf0f793effc8878089d29af4c46ffe1.1605286097.git.berto@igalia.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: 5cddb2e95f8d9e9ee535964df4136b562ce268e1
      
https://github.com/qemu/qemu/commit/5cddb2e95f8d9e9ee535964df4136b562ce268e1
  Author: Alberto Garcia <berto@igalia.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M block/quorum.c
    M tests/qemu-iotests/312
    M tests/qemu-iotests/312.out

  Log Message:
  -----------
  quorum: Implement bdrv_co_pwrite_zeroes()

This simply calls bdrv_co_pwrite_zeroes() in all children.

bs->supported_zero_flags is also set to the flags that are supported
by all children.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Message-Id: 
<2f09c842781fe336b4c2e40036bba577b7430190.1605286097.git.berto@igalia.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: c8807c5edcc8bd8917a5b7531d47ef6a99e07bd8
      
https://github.com/qemu/qemu/commit/c8807c5edcc8bd8917a5b7531d47ef6a99e07bd8
  Author: Philippe Mathieu-Daudé <philmd@redhat.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M block/nvme.c

  Log Message:
  -----------
  block/nvme: Implement fake truncate() coroutine

NVMe drive cannot be shrunk.

Since commit c80d8b06cfa we can use the @exact parameter (set
to false) to return success if the block device is larger than
the requested offset (even if we can not be shrunk).

Use this parameter to implement the NVMe truncate() coroutine,
similarly how it is done for the iscsi and file-posix drivers
(see commit 82325ae5f2f "Evaluate @exact in protocol drivers").

Reported-by: Xueqiang Wei <xuwei@redhat.com>
Suggested-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201210125202.858656-1-philmd@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>


  Commit: 0c8fbfb91536cd1505a2af88912e8e854d29fab0
      
https://github.com/qemu/qemu/commit/0c8fbfb91536cd1505a2af88912e8e854d29fab0
  Author: Max Reitz <mreitz@redhat.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M tests/qemu-iotests/102
    M tests/qemu-iotests/102.out

  Log Message:
  -----------
  iotests/102: Pass $QEMU_HANDLE to _send_qemu_cmd

The first parameter passed to _send_qemu_cmd is supposed to be the
$QEMU_HANDLE.  102 does not do so here, fix it.

As a result, the output changes: Now we see the prompt this command is
supposedly waiting for before the resize message - as it should be.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201217153803.101231-2-mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>


  Commit: 0e72078128229bf9efb542e396ab44bf91b91340
      
https://github.com/qemu/qemu/commit/0e72078128229bf9efb542e396ab44bf91b91340
  Author: Max Reitz <mreitz@redhat.com>
  Date:   2020-12-18 (Fri, 18 Dec 2020)

  Changed paths:
    M tests/qemu-iotests/085.out
    M tests/qemu-iotests/094.out
    M tests/qemu-iotests/095.out
    M tests/qemu-iotests/109.out
    M tests/qemu-iotests/117.out
    M tests/qemu-iotests/127.out
    M tests/qemu-iotests/140.out
    M tests/qemu-iotests/141.out
    M tests/qemu-iotests/143.out
    M tests/qemu-iotests/144.out
    M tests/qemu-iotests/153.out
    M tests/qemu-iotests/156.out
    M tests/qemu-iotests/161.out
    M tests/qemu-iotests/173.out
    M tests/qemu-iotests/182.out
    M tests/qemu-iotests/183.out
    M tests/qemu-iotests/185.out
    M tests/qemu-iotests/191.out
    M tests/qemu-iotests/223.out
    M tests/qemu-iotests/229.out
    M tests/qemu-iotests/249.out
    M tests/qemu-iotests/308.out
    M tests/qemu-iotests/312.out
    M tests/qemu-iotests/common.qemu

  Log Message:
  -----------
  iotests: Fix _send_qemu_cmd with bash 5.1

With bash 5.1, the output of the following script changes:

  a=("double  space")
  a=${a[@]:0:1}
  echo "$a"

from "double space" to "double  space", i.e. all white space is
preserved as-is.  This is probably what we actually want here (judging
from the "...to accommodate pathnames with spaces" comment), but before
5.1, we would have to quote the ${} slice to get the same behavior.

In any case, without quoting, the reference output of many iotests is
different between bash 5.1 and pre-5.1, which is not very good.  The
output of 5.1 is what we want, so whatever we do to get pre-5.1 to the
same result, it means we have to fix the reference output of basically
all tests that invoke _send_qemu_cmd (except the ones that only use
single spaces in the commands they invoke).

Instead of quoting the ${} slice (cmd="${$@: 1:...}"), we can also just
not use array slicing and replace the whole thing with a simple "cmd=$1;
shift", which works because all callers quote the whole $cmd argument
anyway.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201217153803.101231-3-mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>


  Commit: 26f6b15e2636eb20cb6757093170341b22fe6fbc
      
https://github.com/qemu/qemu/commit/26f6b15e2636eb20cb6757093170341b22fe6fbc
  Author: Peter Maydell <peter.maydell@linaro.org>
  Date:   2020-12-31 (Thu, 31 Dec 2020)

  Changed paths:
    M block.c
    M block/file-posix.c
    M block/io.c
    M block/meson.build
    M block/nvme.c
    A block/preallocate.c
    M block/quorum.c
    M docs/system/qemu-block-drivers.rst.inc
    M include/block/block.h
    M include/block/block_int.h
    M qapi/block-core.json
    M qemu-io-cmds.c
    M scripts/simplebench/bench-example.py
    A scripts/simplebench/bench_prealloc.py
    M scripts/simplebench/bench_write_req.py
    A scripts/simplebench/results_to_text.py
    M scripts/simplebench/simplebench.py
    M tests/qemu-iotests/085.out
    M tests/qemu-iotests/094.out
    M tests/qemu-iotests/095.out
    M tests/qemu-iotests/102
    M tests/qemu-iotests/102.out
    M tests/qemu-iotests/109.out
    M tests/qemu-iotests/117.out
    M tests/qemu-iotests/127.out
    M tests/qemu-iotests/140.out
    M tests/qemu-iotests/141.out
    M tests/qemu-iotests/143.out
    M tests/qemu-iotests/144.out
    M tests/qemu-iotests/153.out
    M tests/qemu-iotests/156.out
    M tests/qemu-iotests/161.out
    M tests/qemu-iotests/173.out
    M tests/qemu-iotests/182.out
    M tests/qemu-iotests/183.out
    M tests/qemu-iotests/185.out
    M tests/qemu-iotests/191.out
    M tests/qemu-iotests/223.out
    M tests/qemu-iotests/229.out
    M tests/qemu-iotests/249.out
    A tests/qemu-iotests/298
    A tests/qemu-iotests/298.out
    M tests/qemu-iotests/308.out
    A tests/qemu-iotests/312
    A tests/qemu-iotests/312.out
    M tests/qemu-iotests/common.qemu
    M tests/qemu-iotests/group
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-12-18' 
into staging

Block patches:
- New block filter: preallocate (which, on writes beyond an image file's
  end, allocates big chunks of data so that such post-EOF writes will
  occur less frequently)
- write-zeroes and block-status support for Quorum
- Implementation of truncate for the nvme block driver similarly to the
  existing implementations for host block devices and iscsi devices
- Block layer refactoring: Drop the tighten_restrictions concept in the
  block permission functions
- iotest fixes

# gpg: Signature made Fri 18 Dec 2020 14:45:30 GMT
# gpg:                using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg:                issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2020-12-18: (30 commits)
  iotests: Fix _send_qemu_cmd with bash 5.1
  iotests/102: Pass $QEMU_HANDLE to _send_qemu_cmd
  block/nvme: Implement fake truncate() coroutine
  quorum: Implement bdrv_co_pwrite_zeroes()
  quorum: Implement bdrv_co_block_status()
  scripts/simplebench: add bench_prealloc.py
  simplebench/results_to_text: make executable
  simplebench/results_to_text: add difference line to the table
  simplebench/results_to_text: improve view of the table
  simplebench: move results_to_text() into separate file
  simplebench: rename ascii() to results_to_text()
  scripts/simplebench: use standard deviation for +- error
  scripts/simplebench: support iops
  scripts/simplebench: fix grammar: s/successed/succeeded/
  iotests: add 298 to test new preallocate filter driver
  iotests.py: execute_setup_common(): add required_fmts argument
  iotests: qemu_io_silent: support --image-opts
  qemu-io: add preallocate mode parameter for truncate command
  block: introduce preallocate filter
  block: bdrv_check_perm(): process children anyway
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>


Compare: https://github.com/qemu/qemu/compare/c7e48f91653d...26f6b15e2636
[Prev in Thread]
Current Thread
[Next in Thread]
[Qemu-commits] [qemu/qemu] bb87e4: block: add bdrv_refresh_perms() helper, Peter Maydell <=
Prev by Date: [Qemu-commits] [qemu/qemu] 8ba9c4: iotests/210: Fix reference output
Previous by thread: [Qemu-commits] [qemu/qemu] 8ba9c4: iotests/210: Fix reference output
Index(es):
- Date
- Thread