qemu-commits
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Qemu-commits] [qemu/qemu] 780369: block-backend: simplify blk_get_aio_c


From: GitHub
Subject: [Qemu-commits] [qemu/qemu] 780369: block-backend: simplify blk_get_aio_context
Date: Tue, 15 May 2018 09:59:10 -0700

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 7803696d8557e02441a2781a19d4008b2f50925c
      
https://github.com/qemu/qemu/commit/7803696d8557e02441a2781a19d4008b2f50925c
  Author: Daniel Henrique Barboza <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/block-backend.c

  Log Message:
  -----------
  block-backend: simplify blk_get_aio_context

blk_get_aio_context verifies if BlockDriverState bs is not NULL,
return bdrv_get_aio_context(bs) if true or qemu_get_aio_context()
otherwise. However, bdrv_get_aio_context from block.c already does
this verification itself, also returning qemu_get_aio_context()
if bs is NULL:

AioContext *bdrv_get_aio_context(BlockDriverState *bs)
{
    return bs ? bs->aio_context : qemu_get_aio_context();
}

This patch simplifies blk_get_aio_context to simply call
bdrv_get_aio_context instead of replicating the same logic.

Signed-off-by: Daniel Henrique Barboza <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: e31f6864a6d6fa072a5d02e7679d0db6d0c22311
      
https://github.com/qemu/qemu/commit/e31f6864a6d6fa072a5d02e7679d0db6d0c22311
  Author: Eric Blake <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/io.c
    M include/block/block_int.h

  Log Message:
  -----------
  block: Support byte-based aio callbacks

We are gradually moving away from sector-based interfaces, towards
byte-based.  Add new sector-based aio callbacks for read and write,
to match the fact that bdrv_aio_pdiscard is already byte-based.

Ideally, drivers should be converted to use coroutine callbacks
rather than aio; but that is not quite as trivial (and if we were
to do that conversion, the null-aio driver would disappear), so for
the short term, converting the signature but keeping things with
aio is easier.  However, we CAN declare that a driver that uses
the byte-based aio interfaces now defaults to byte-based
operations, and must explicitly provide a refresh_limits override
to stick with larger alignments (making the alignment issues more
obvious directly in the drivers touched in the next few patches).

Once all drivers are converted, the sector-based aio callbacks will
be removed; in the meantime, a FIXME comment is added due to a
slight inefficiency that will be touched up as part of that later
cleanup.

Simplify some instances of 'bs->drv' into 'drv' while touching this,
since the local variable already exists to reduce typing.

Signed-off-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: de7056a3f95e85aa6bba30ad4b37a8dbc2b072ab
      
https://github.com/qemu/qemu/commit/de7056a3f95e85aa6bba30ad4b37a8dbc2b072ab
  Author: Eric Blake <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/file-win32.c
    M block/win32-aio.c
    M include/block/raw-aio.h

  Log Message:
  -----------
  file-win32: Switch to byte-based callbacks

We are gradually moving away from sector-based interfaces, towards
byte-based.  Make the change for the last few sector-based callbacks
in the file-win32 driver.

Note that the driver was already using byte-based calls for
performing actual I/O, so this just gets rid of a round trip
of scaling; however, as I don't know if Windows is tolerant of
non-sector AIO operations, I went with the conservative approach
of modifying .bdrv_refresh_limits to override the block layer
defaults back to the pre-patch value of 512.

Signed-off-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: b3241e9274ca1aafb94ad8683305bb383e11c766
      
https://github.com/qemu/qemu/commit/b3241e9274ca1aafb94ad8683305bb383e11c766
  Author: Eric Blake <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/null.c

  Log Message:
  -----------
  null: Switch to byte-based read/write

We are gradually moving away from sector-based interfaces, towards
byte-based.  Make the change for the last few sector-based callbacks
in the null-co and null-aio drivers.

Note that since the null driver does nothing on writes, it trivially
supports the BDRV_REQ_FUA flag (all writes have already landed to
the same bit-bucket without needing an extra flush call).  Also, since
the null driver does just as well with byte-based requests, we can
now avoid cycles wasted on read-modify-write by taking advantage of
the block layer now defaulting the alignment to 1 instead of 512.

Signed-off-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: e8e16d4baffc2d5734c1c54c28867860dd5a0074
      
https://github.com/qemu/qemu/commit/e8e16d4baffc2d5734c1c54c28867860dd5a0074
  Author: Eric Blake <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/rbd.c

  Log Message:
  -----------
  rbd: Switch to byte-based callbacks

We are gradually moving away from sector-based interfaces, towards
byte-based.  Make the change for the last few sector-based callbacks
in the rbd driver.

Note that the driver was already using byte-based calls for
performing actual I/O, so this just gets rid of a round trip
of scaling; however, as I don't know if RBD is tolerant of
non-sector AIO operations, I went with the conservate approach
of adding .bdrv_refresh_limits to override the block layer
defaults back to the pre-patch value of 512.

Signed-off-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 918889b291dfb8f55d537e01ecbb082d27c2737a
      
https://github.com/qemu/qemu/commit/918889b291dfb8f55d537e01ecbb082d27c2737a
  Author: Eric Blake <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/vxhs.c

  Log Message:
  -----------
  vxhs: Switch to byte-based callbacks

We are gradually moving away from sector-based interfaces, towards
byte-based.  Make the change for the last few sector-based callbacks
in the vxhs driver.

Note that the driver was already using byte-based calls for
performing actual I/O, so this just gets rid of a round trip
of scaling; however, as I don't know if VxHS is tolerant of
non-sector AIO operations, I went with the conservative approach
of adding .bdrv_refresh_limits to override the block layer
defaults back to the pre-patch value of 512.

Signed-off-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: edfab6a08b0dc240ac3a665adf4dc40db9b9ec7f
      
https://github.com/qemu/qemu/commit/edfab6a08b0dc240ac3a665adf4dc40db9b9ec7f
  Author: Eric Blake <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/io.c
    M include/block/block_int.h

  Log Message:
  -----------
  block: Drop last of the sector-based aio callbacks

We are gradually moving away from sector-based interfaces, towards
byte-based.  Now that all drivers with aio callbacks are using the
byte-based interfaces, we can remove the sector-based versions.

Signed-off-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: e18a58b4e3aa355a9b6eef1f39a980a4f18f1294
      
https://github.com/qemu/qemu/commit/e18a58b4e3aa355a9b6eef1f39a980a4f18f1294
  Author: Eric Blake <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/gluster.c
    M block/io.c
    M block/iscsi.c
    M block/parallels.c
    M block/qcow.c
    M block/qed.c
    M block/replication.c
    M block/sheepdog.c
    M block/ssh.c
    M block/vhdx.c
    M include/block/block_int.h

  Log Message:
  -----------
  block: Merge .bdrv_co_writev{,_flags} in drivers

We have too many driver callback interfaces; simplify the mess
somewhat by merging the flags parameter of .bdrv_co_writev_flags()
into .bdrv_co_writev().  Note that as long as a driver doesn't set
.supported_write_flags, the flags argument will be 0 and behavior is
identical.  Also note that the public function bdrv_co_writev() still
lacks a flags argument; so the driver signature is thus intentionally
slightly different.  But that's not the end of the world, nor the first
time that the driver interface differs slightly from the public
interface.

Ideally, we should be rewriting all of these drivers to use modern
byte-based interfaces.  But that's a more invasive patch to write
and audit, compared to the simplification done here.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Daniel P. Berrangé <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 83592184d8692c818e083c3ce43f63ca2533fa85
      
https://github.com/qemu/qemu/commit/83592184d8692c818e083c3ce43f63ca2533fa85
  Author: Alberto Garcia <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M hmp-commands.hx
    M hmp.c

  Log Message:
  -----------
  hmp: Allow using a qdev id in block_set_io_throttle

The QMP version of this command can take a qdev ID since 7a9877a02635,
but the HMP version is still using the deprecated block device name so
there's no way to refer to a block device added like this:

  -blockdev node-name=disk0,driver=qcow2,file.driver=file,file.filename=hd.qcow2
  -device virtio-blk-pci,id=virtio-blk-pci0,drive=disk0

This patch works around this problem by using the specified name as a
qdev ID if the block device name is not found.

Signed-off-by: Alberto Garcia <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: ab9ba614556ac5b0f8d96b99e0dba19f1e28d6c2
      
https://github.com/qemu/qemu/commit/ab9ba614556ac5b0f8d96b99e0dba19f1e28d6c2
  Author: John Snow <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M blockjob.c
    M qapi/block-core.json

  Log Message:
  -----------
  blockjob: expose error string via query

When we've reached the concluded state, we need to expose the error
state if applicable. Add the new field.

This should be sufficient for determining if a job completed
successfully or not after concluding; if we want to discriminate
based on how it failed more mechanically, we can always add an
explicit return code enumeration later.

I didn't bother to make it only show up if we are in the concluded
state; I don't think it's necessary.

Cc: address@hidden
Signed-off-by: John Snow <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 37aa19b63c46d933f1e4ea944cfccee54e2caf4a
      
https://github.com/qemu/qemu/commit/37aa19b63c46d933f1e4ea944cfccee54e2caf4a
  Author: Kevin Wolf <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M blockjob.c

  Log Message:
  -----------
  blockjob: Fix assertion in block_job_finalize()

Every job gets a non-NULL job->txn on creation, but it doesn't
necessarily keep it until it is decommissioned: Finalising a job removes
it from its transaction. Therefore, calling 'blockdev-job-finalize' a
second time on an already concluded job causes an assertion failure.

Remove job->txn from the assertion in block_job_finalize() to fix this.
block_job_do_finalize() still has the same assertion, but if a job is
already removed from its transaction, block_job_apply_verb() will
already error out before we run into that assertion.

Cc: address@hidden
Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Reviewed-by: John Snow <address@hidden>


  Commit: 05df8a6a2b4e36e8d69de2130e616d5ac28e8837
      
https://github.com/qemu/qemu/commit/05df8a6a2b4e36e8d69de2130e616d5ac28e8837
  Author: Kevin Wolf <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/backup.c
    M block/commit.c
    M block/mirror.c
    M block/stream.c
    M blockjob.c
    M include/block/blockjob.h

  Log Message:
  -----------
  blockjob: Wrappers for progress counter access

Block job drivers are not expected to mess with the internals of the
BlockJob object, so provide wrapper functions for one of the cases where
they still do it: Updating the progress counter.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Reviewed-by: John Snow <address@hidden>


  Commit: f05fee508f538ca262d2ab19bcd8772196efe848
      
https://github.com/qemu/qemu/commit/f05fee508f538ca262d2ab19bcd8772196efe848
  Author: Kevin Wolf <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/backup.c
    M block/commit.c
    M block/mirror.c
    M block/stream.c
    M include/block/blockjob.h

  Log Message:
  -----------
  blockjob: Move RateLimit to BlockJob

Every block job has a RateLimit, and they all do the exact same thing
with it, so it should be common infrastructure. Move the struct field
for a start.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Reviewed-by: John Snow <address@hidden>


  Commit: 18bb69287ea522ab696e1bea818b93e5eaa85745
      
https://github.com/qemu/qemu/commit/18bb69287ea522ab696e1bea818b93e5eaa85745
  Author: Kevin Wolf <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/backup.c
    M block/commit.c
    M block/mirror.c
    M block/stream.c
    M blockjob.c
    M include/block/blockjob.h
    M include/block/blockjob_int.h

  Log Message:
  -----------
  blockjob: Implement block_job_set_speed() centrally

All block job drivers support .set_speed and all of them duplicate the
same code to implement it. Move that code to blockjob.c and remove the
now useless callback.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Reviewed-by: John Snow <address@hidden>


  Commit: dee81d5111ff0e24ac63ab0dbbd19e84c2f87904
      
https://github.com/qemu/qemu/commit/dee81d5111ff0e24ac63ab0dbbd19e84c2f87904
  Author: Kevin Wolf <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/backup.c
    M block/commit.c
    M block/mirror.c
    M block/stream.c
    M blockjob.c
    M include/block/blockjob_int.h

  Log Message:
  -----------
  blockjob: Introduce block_job_ratelimit_get_delay()

This gets us rid of more direct accesses to BlockJob fields from the
job drivers.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Reviewed-by: John Snow <address@hidden>


  Commit: bd21935b50d100d8da8c05cd3c2009f0f3432cb4
      
https://github.com/qemu/qemu/commit/bd21935b50d100d8da8c05cd3c2009f0f3432cb4
  Author: Kevin Wolf <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/backup.c
    M blockjob.c
    M include/block/blockjob.h

  Log Message:
  -----------
  blockjob: Add block_job_driver()

The backup block job directly accesses the driver field in BlockJob. Add
a wrapper for getting it.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Reviewed-by: John Snow <address@hidden>


  Commit: 6cba5377f54d7ea859a29c1877785e7101794683
      
https://github.com/qemu/qemu/commit/6cba5377f54d7ea859a29c1877785e7101794683
  Author: Max Reitz <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M tests/qemu-iotests/122
    M tests/qemu-iotests/122.out
    A tests/qemu-iotests/214
    A tests/qemu-iotests/214.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  iotests: Split 214 off of 122

Commit abd3622cc03cf41ed542126a540385f30a4c0175 added a case to 122
regarding how the qcow2 driver handles an incorrect compressed data
length value.  This does not really fit into 122, as that file is
supposed to contain qemu-img convert test cases, which this case is not.
So this patch splits it off into its own file; maybe we will even get
more qcow2-only compression tests in the future.

Also, that test case does not work with refcount_bits=1, so mark that
option as unsupported.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Alberto Garcia <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 74c44a59348f7fac96c32621e37ee636546f26f8
      
https://github.com/qemu/qemu/commit/74c44a59348f7fac96c32621e37ee636546f26f8
  Author: Alberto Garcia <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/qcow2-refcount.c

  Log Message:
  -----------
  Fix error message about compressed clusters with OFLAG_COPIED

Compressed clusters are not supposed to have the COPIED bit set.
"qemu-img check" detects that and prints an error message reporting
the number of the affected host cluster. This doesn't make much sense
because compressed clusters are not aligned to host clusters, so it
would be better to report the offset instead. Plus, the calculation is
wrong and it uses the raw L2 entry as if it was simply an offset.

This patch fixes the error message and reports the offset of the
compressed cluster.

Signed-off-by: Alberto Garcia <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 3c7d14b201ee4eeec2ca259b5a071a4599aa8847
      
https://github.com/qemu/qemu/commit/3c7d14b201ee4eeec2ca259b5a071a4599aa8847
  Author: Alberto Garcia <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M docs/interop/qcow2.txt

  Log Message:
  -----------
  specs/qcow2: Clarify that compressed clusters have the COPIED bit reset

Compressed clusters are not supposed to have the COPIED bit set, but
this is not made explicit in the specs, so let's document it.

Signed-off-by: Alberto Garcia <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 52253998ec3e523c9e20ae81e2a6431d8ff733ba
      
https://github.com/qemu/qemu/commit/52253998ec3e523c9e20ae81e2a6431d8ff733ba
  Author: Alberto Garcia <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/qcow2.c
    M block/qcow2.h
    M tests/qemu-iotests/137.out

  Log Message:
  -----------
  qcow2: Give the refcount cache the minimum possible size by default

The L2 and refcount caches have default sizes that can be overridden
using the l2-cache-size and refcount-cache-size (an additional
parameter named cache-size sets the combined size of both caches).

Unless forced by one of the aforementioned parameters, QEMU will set
the unspecified sizes so that the L2 cache is 4 times larger than the
refcount cache.

This is based on the premise that the refcount metadata needs to be
only a fourth of the L2 metadata to cover the same amount of disk
space. This is incorrect for two reasons:

 a) The amount of disk covered by an L2 table depends solely on the
    cluster size, but in the case of a refcount block it depends on
    the cluster size *and* the width of each refcount entry.
    The 4/1 ratio is only valid with 16-bit entries (the default).

 b) When we talk about disk space and L2 tables we are talking about
    guest space (L2 tables map guest clusters to host clusters),
    whereas refcount blocks are used for host clusters (including
    L1/L2 tables and the refcount blocks themselves). On a fully
    populated (and uncompressed) qcow2 file, image size > virtual size
    so there are more refcount entries than L2 entries.

Problem (a) could be fixed by adjusting the algorithm to take into
account the refcount entry width. Problem (b) could be fixed by
increasing a bit the refcount cache size to account for the clusters
used for qcow2 metadata.

However this patch takes a completely different approach and instead
of keeping a ratio between both cache sizes it assigns as much as
possible to the L2 cache and the remainder to the refcount cache.

The reason is that L2 tables are used for every single I/O request
from the guest and the effect of increasing the cache is significant
and clearly measurable. Refcount blocks are however only used for
cluster allocation and internal snapshots and in practice are accessed
sequentially in most cases, so the effect of increasing the cache is
negligible (even when doing random writes from the guest).

So, make the refcount cache as small as possible unless the user
explicitly asks for a larger one.

Signed-off-by: Alberto Garcia <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 603790ef3aec6a19b1c095188a1d2171934a27de
      
https://github.com/qemu/qemu/commit/603790ef3aec6a19b1c095188a1d2171934a27de
  Author: Alberto Garcia <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M docs/qcow2-cache.txt

  Log Message:
  -----------
  docs: Document the new default sizes of the qcow2 caches

We have just reduced the refcount cache size to the minimum unless
the user explicitly requests a larger one, so we have to update the
documentation to reflect this change.

Signed-off-by: Alberto Garcia <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 81c6ddf49a76a663cea16c07a07d51b67c853209
      
https://github.com/qemu/qemu/commit/81c6ddf49a76a663cea16c07a07d51b67c853209
  Author: Max Reitz <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M tests/qemu-iotests/common.qemu

  Log Message:
  -----------
  iotests: Add failure matching to common.qemu

Currently, common.qemu only allows to match for results indicating
success.  The only way to fail is by provoking a timeout.  However,
sometimes we do have a defined failure output and can match for that,
which saves us from having to wait for the timeout in case of failure.
Because failure can sometimes just result in a _notrun in the test, it
is actually important to care about being able to fail quickly.

Also, sometimes we simply do not get any specific output in case of
success.  The only way to handle this currently would be to define an
error message as the string to look for, which means that actual success
results in a timeout.  This is really bad because it unnecessarily slows
down a succeeding test.

Therefore, this patch adds a new parameter $success_or_failure to
_timed_wait_for and _send_qemu_cmd.  Setting this to a non-empty string
makes both commands expect two match parameters: If the first matches,
the function succeeds.  If the second matches, the function fails.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: b05a2225d2e87a04697509219d00ced7c46ed34d
      
https://github.com/qemu/qemu/commit/b05a2225d2e87a04697509219d00ced7c46ed34d
  Author: Max Reitz <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M tests/qemu-iotests/181
    M tests/qemu-iotests/201

  Log Message:
  -----------
  iotests: Skip 181 and 201 without userfaultfd

userfaultfd support depends on the host kernel, so it may not be
available.  If so, 181 and 201 should be skipped.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 6c6f24fd84895d03baa898bbc4324dd4ccc97071
      
https://github.com/qemu/qemu/commit/6c6f24fd84895d03baa898bbc4324dd4ccc97071
  Author: Max Reitz <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/Makefile.objs
    A block/copy-on-read.c
    M qapi/block-core.json

  Log Message:
  -----------
  block: Add COR filter driver

This adds a simple copy-on-read filter driver.  It relies on the already
existing COR functionality in the central block layer code, which may be
moved here once we no longer need it there.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Alberto Garcia <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 24b7c538fea86b598e2a335f4805a0ab50a30e98
      
https://github.com/qemu/qemu/commit/24b7c538fea86b598e2a335f4805a0ab50a30e98
  Author: Max Reitz <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M include/block/block.h

  Log Message:
  -----------
  block: BLK_PERM_WRITE includes ..._UNCHANGED

Currently we never actually check whether the WRITE_UNCHANGED
permission has been taken for unchanging writes.  But the one check that
is commented out checks both WRITE and WRITE_UNCHANGED; and considering
that WRITE_UNCHANGED is already documented as being weaker than WRITE,
we should probably explicitly document WRITE to include WRITE_UNCHANGED.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>
Message-id: address@hidden
Reviewed-by: Kevin Wolf <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: c6035964f8316b504060618d05b5dd434f18595b
      
https://github.com/qemu/qemu/commit/c6035964f8316b504060618d05b5dd434f18595b
  Author: Max Reitz <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/io.c
    M include/block/block.h

  Log Message:
  -----------
  block: Add BDRV_REQ_WRITE_UNCHANGED flag

This flag signifies that a write request will not change the visible
disk content.  With this flag set, it is sufficient to have the
BLK_PERM_WRITE_UNCHANGED permission instead of BLK_PERM_WRITE.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>
Message-id: address@hidden
Reviewed-by: Kevin Wolf <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 7adcf59fecf3c8ce9330430187350b53f9e50cf7
      
https://github.com/qemu/qemu/commit/7adcf59fecf3c8ce9330430187350b53f9e50cf7
  Author: Max Reitz <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/io.c

  Log Message:
  -----------
  block: Set BDRV_REQ_WRITE_UNCHANGED for COR writes

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>
Message-id: address@hidden
Reviewed-by: Kevin Wolf <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 1b1a920b713af6af795d49d0e3d2a8a65020bf82
      
https://github.com/qemu/qemu/commit/1b1a920b713af6af795d49d0e3d2a8a65020bf82
  Author: Max Reitz <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/quorum.c

  Log Message:
  -----------
  block/quorum: Support BDRV_REQ_WRITE_UNCHANGED

We just need to forward it to quorum's children (except in case of a
rewrite because of corruption), but for that we first have to support
flags in child requests at all.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>
Message-id: address@hidden
Reviewed-by: Kevin Wolf <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 228345bf5db8bc97d1c64f062e138d389065d1ab
      
https://github.com/qemu/qemu/commit/228345bf5db8bc97d1c64f062e138d389065d1ab
  Author: Max Reitz <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/blkdebug.c
    M block/blkreplay.c
    M block/blkverify.c
    M block/copy-on-read.c
    M block/mirror.c
    M block/raw-format.c
    M block/throttle.c

  Log Message:
  -----------
  block: Support BDRV_REQ_WRITE_UNCHANGED in filters

Update the rest of the filter drivers to support
BDRV_REQ_WRITE_UNCHANGED.  They already forward write request flags to
their children, so we just have to announce support for it.

This patch does not cover the replication driver because that currently
does not support flags at all, and because it just grabs the WRITE
permission for its children when it can, so we should be fine just
submitting the incoming WRITE_UNCHANGED requests as normal writes.

It also does not cover format drivers for similar reasons.  They all use
bdrv_format_default_perms() as their .bdrv_child_perm() implementation
so they just always grab the WRITE permission for their file children
whenever possible.  In addition, it often would be difficult to
ascertain whether incoming unchanging writes end up as unchanging writes
in their files.  So we just leave them as normal potentially changing
writes.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>
Message-id: address@hidden
Reviewed-by: Kevin Wolf <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 5fdc0b73eb68d107944cfa65185fb155b511e496
      
https://github.com/qemu/qemu/commit/5fdc0b73eb68d107944cfa65185fb155b511e496
  Author: Max Reitz <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M tests/qemu-iotests/197

  Log Message:
  -----------
  iotests: Clean up wrap image in 197

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>
Message-id: address@hidden
Reviewed-by: Kevin Wolf <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: a62cbac4ce2db79c14ff299e98ee556b57467c19
      
https://github.com/qemu/qemu/commit/a62cbac4ce2db79c14ff299e98ee556b57467c19
  Author: Max Reitz <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    A tests/qemu-iotests/215
    A tests/qemu-iotests/215.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  iotests: Copy 197 for COR filter driver

iotest 197 tests copy-on-read using the (now old) copy-on-read flag.
Copy it to 215 and modify it to use the COR filter driver instead.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Kevin Wolf <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 3e7a95feb9b5d66cff7fee38b3c423135ed245f6
      
https://github.com/qemu/qemu/commit/3e7a95feb9b5d66cff7fee38b3c423135ed245f6
  Author: Max Reitz <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    A tests/qemu-iotests/216
    A tests/qemu-iotests/216.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  iotests: Add test for COR across nodes

COR across nodes (that is, you have some filter node between the
actually COR target and the node that performs the COR) cannot reliably
work together with the permission system when there is no explicit COR
node that can request the WRITE_UNCHANGED permission for its child.
This is because COR (currently) sneaks its requests by the usual
permission checks, so it can work without a WRITE* permission; but if
there is a filter node in between, that will re-issue the request, which
then passes through the usual check -- and if nobody has requested a
WRITE_UNCHANGED permission, that check will fail.

There is no real direct fix apart from hoping that there is someone who
has requested that permission; in case of just the qemu-io HMP command
(and no guest device), however, that is not the case.  The real real fix
is to implement the copy-on-read flag through an implicitly added COR
node.  Such a node can request the necessary permissions as shown in
this test.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Kevin Wolf <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 5279b30392da7a3248b320c75f20c61e3a95863c
      
https://github.com/qemu/qemu/commit/5279b30392da7a3248b320c75f20c61e3a95863c
  Author: Max Reitz <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M qemu-img.c

  Log Message:
  -----------
  qemu-img: Check post-truncation size

Some block drivers (iscsi and file-posix when dealing with device files)
do not actually support truncation, even though they provide a
.bdrv_truncate() method and will happily return success when providing a
new size that does not exceed the current size.  This is because these
drivers expect the user to resize the image outside of qemu and then
provide qemu with that information through the block_resize command
(compare cb1b83e740384b4e0d950f3d7c81c02b8ce86c2e).

Of course, anyone using qemu-img resize will find that behavior useless.
So we should check the actual size of the image after the supposedly
successful truncation took place, emit an error if nothing changed and
emit a warning if the target size was not met.

Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1523065
Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: c1e3489dfaa01d215e37b1412759b856c33d44ed
      
https://github.com/qemu/qemu/commit/c1e3489dfaa01d215e37b1412759b856c33d44ed
  Author: Max Reitz <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M include/block/block_int.h

  Log Message:
  -----------
  block: Document BDRV_REQ_WRITE_UNCHANGED support

Add BDRV_REQ_WRITE_UNCHANGED to the list of flags honored during pwrite
and pwrite_zeroes, and also add a note on when you absolutely need to
support it.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 2a01c01f9ecb43af4c0a85fe6adc429ffc9c31b5
      
https://github.com/qemu/qemu/commit/2a01c01f9ecb43af4c0a85fe6adc429ffc9c31b5
  Author: Max Reitz <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M qemu-io.c

  Log Message:
  -----------
  qemu-io: Use purely string blockdev options

Currently, qemu-io only uses string-valued blockdev options (as all are
converted directly from QemuOpts) -- with one exception: -U adds the
force-share option as a boolean.  This in itself is already a bit
questionable, but a real issue is that it also assumes the value already
existing in the options QDict would be a boolean, which is wrong.

That has the following effect:

$ ./qemu-io -r -U --image-opts \
    driver=file,filename=/dev/null,force-share=off
[1]    15200 segmentation fault (core dumped)  ./qemu-io -r -U
--image-opts driver=file,filename=/dev/null,force-share=off

Since @opts is converted from QemuOpts, the value must be a string, and
we have to compare it as such.  Consequently, it makes sense to also set
it as a string instead of a boolean.

Cc: address@hidden
Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 4615f87832d2fcb7a544bedeece2741bf8c21f94
      
https://github.com/qemu/qemu/commit/4615f87832d2fcb7a544bedeece2741bf8c21f94
  Author: Max Reitz <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M qemu-img.c

  Log Message:
  -----------
  qemu-img: Use only string options in img_open_opts

img_open_opts() takes a QemuOpts and converts them to a QDict, so all
values therein are strings.  Then it may try to call qdict_get_bool(),
however, which will fail with a segmentation fault every time:

$ ./qemu-img info -U --image-opts \
    driver=file,filename=/dev/null,force-share=off
[1]    27869 segmentation fault (core dumped)  ./qemu-img info -U
--image-opts driver=file,filename=/dev/null,force-share=off

Fix this by using qdict_get_str() and comparing the value as a string.
Also, when adding a force-share value to the QDict, add it as a string
so it fits the rest of the dict.

Cc: address@hidden
Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 4e7d73c5fbd97e55ffe5af02f24d1f7dbe3bbf20
      
https://github.com/qemu/qemu/commit/4e7d73c5fbd97e55ffe5af02f24d1f7dbe3bbf20
  Author: Max Reitz <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M tests/qemu-iotests/153
    M tests/qemu-iotests/153.out

  Log Message:
  -----------
  iotests: Add test for -U/force-share conflicts

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 1fce860ea5eba1ca00a67911fc0b8a5d80009514
      
https://github.com/qemu/qemu/commit/1fce860ea5eba1ca00a67911fc0b8a5d80009514
  Author: Kevin Wolf <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/Makefile.objs
    M block/blkdebug.c
    M block/blkreplay.c
    M block/blkverify.c
    A block/copy-on-read.c
    M block/io.c
    M block/mirror.c
    M block/qcow2-refcount.c
    M block/qcow2.c
    M block/qcow2.h
    M block/quorum.c
    M block/raw-format.c
    M block/throttle.c
    M docs/interop/qcow2.txt
    M docs/qcow2-cache.txt
    M include/block/block.h
    M include/block/block_int.h
    M qapi/block-core.json
    M qemu-img.c
    M qemu-io.c
    M tests/qemu-iotests/122
    M tests/qemu-iotests/122.out
    M tests/qemu-iotests/137.out
    M tests/qemu-iotests/153
    M tests/qemu-iotests/153.out
    M tests/qemu-iotests/181
    M tests/qemu-iotests/197
    M tests/qemu-iotests/201
    A tests/qemu-iotests/214
    A tests/qemu-iotests/214.out
    A tests/qemu-iotests/215
    A tests/qemu-iotests/215.out
    A tests/qemu-iotests/216
    A tests/qemu-iotests/216.out
    M tests/qemu-iotests/common.qemu
    M tests/qemu-iotests/group

  Log Message:
  -----------
  Merge remote-tracking branch 'mreitz/tags/pull-block-2018-05-15' into 
queue-block

- Copy-on-read block driver
- The qcow2 default refcount cache size has been decreased
- Various bug fixes

# gpg: Signature made Tue May 15 16:18:25 2018 CEST
# gpg:                using RSA key F407DB0061D5CF40
# gpg: Good signature from "Max Reitz <address@hidden>"
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* mreitz/tags/pull-block-2018-05-15: (21 commits)
  iotests: Add test for -U/force-share conflicts
  qemu-img: Use only string options in img_open_opts
  qemu-io: Use purely string blockdev options
  block: Document BDRV_REQ_WRITE_UNCHANGED support
  qemu-img: Check post-truncation size
  iotests: Add test for COR across nodes
  iotests: Copy 197 for COR filter driver
  iotests: Clean up wrap image in 197
  block: Support BDRV_REQ_WRITE_UNCHANGED in filters
  block/quorum: Support BDRV_REQ_WRITE_UNCHANGED
  block: Set BDRV_REQ_WRITE_UNCHANGED for COR writes
  block: Add BDRV_REQ_WRITE_UNCHANGED flag
  block: BLK_PERM_WRITE includes ..._UNCHANGED
  block: Add COR filter driver
  iotests: Skip 181 and 201 without userfaultfd
  iotests: Add failure matching to common.qemu
  docs: Document the new default sizes of the qcow2 caches
  qcow2: Give the refcount cache the minimum possible size by default
  specs/qcow2: Clarify that compressed clusters have the COPIED bit reset
  Fix error message about compressed clusters with OFLAG_COPIED
  ...

Signed-off-by: Kevin Wolf <address@hidden>


  Commit: c416eecea5f3aea863ab8fda5a36a24157b8f704
      
https://github.com/qemu/qemu/commit/c416eecea5f3aea863ab8fda5a36a24157b8f704
  Author: Peter Maydell <address@hidden>
  Date:   2018-05-15 (Tue, 15 May 2018)

  Changed paths:
    M block/Makefile.objs
    M block/backup.c
    M block/blkdebug.c
    M block/blkreplay.c
    M block/blkverify.c
    M block/block-backend.c
    M block/commit.c
    A block/copy-on-read.c
    M block/file-win32.c
    M block/gluster.c
    M block/io.c
    M block/iscsi.c
    M block/mirror.c
    M block/null.c
    M block/parallels.c
    M block/qcow.c
    M block/qcow2-refcount.c
    M block/qcow2.c
    M block/qcow2.h
    M block/qed.c
    M block/quorum.c
    M block/raw-format.c
    M block/rbd.c
    M block/replication.c
    M block/sheepdog.c
    M block/ssh.c
    M block/stream.c
    M block/throttle.c
    M block/vhdx.c
    M block/vxhs.c
    M block/win32-aio.c
    M blockjob.c
    M docs/interop/qcow2.txt
    M docs/qcow2-cache.txt
    M hmp-commands.hx
    M hmp.c
    M include/block/block.h
    M include/block/block_int.h
    M include/block/blockjob.h
    M include/block/blockjob_int.h
    M include/block/raw-aio.h
    M qapi/block-core.json
    M qemu-img.c
    M qemu-io.c
    M tests/qemu-iotests/122
    M tests/qemu-iotests/122.out
    M tests/qemu-iotests/137.out
    M tests/qemu-iotests/153
    M tests/qemu-iotests/153.out
    M tests/qemu-iotests/181
    M tests/qemu-iotests/197
    M tests/qemu-iotests/201
    A tests/qemu-iotests/214
    A tests/qemu-iotests/214.out
    A tests/qemu-iotests/215
    A tests/qemu-iotests/215.out
    A tests/qemu-iotests/216
    A tests/qemu-iotests/216.out
    M tests/qemu-iotests/common.qemu
    M tests/qemu-iotests/group

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Switch AIO/callback based block drivers to a byte-based interface
- Block jobs: Expose error string via query-block-jobs
- Block job cleanups and fixes
- hmp: Allow using a qdev id in block_set_io_throttle

# gpg: Signature made Tue 15 May 2018 16:33:10 BST
# gpg:                using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <address@hidden>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (37 commits)
  iotests: Add test for -U/force-share conflicts
  qemu-img: Use only string options in img_open_opts
  qemu-io: Use purely string blockdev options
  block: Document BDRV_REQ_WRITE_UNCHANGED support
  qemu-img: Check post-truncation size
  iotests: Add test for COR across nodes
  iotests: Copy 197 for COR filter driver
  iotests: Clean up wrap image in 197
  block: Support BDRV_REQ_WRITE_UNCHANGED in filters
  block/quorum: Support BDRV_REQ_WRITE_UNCHANGED
  block: Set BDRV_REQ_WRITE_UNCHANGED for COR writes
  block: Add BDRV_REQ_WRITE_UNCHANGED flag
  block: BLK_PERM_WRITE includes ..._UNCHANGED
  block: Add COR filter driver
  iotests: Skip 181 and 201 without userfaultfd
  iotests: Add failure matching to common.qemu
  docs: Document the new default sizes of the qcow2 caches
  qcow2: Give the refcount cache the minimum possible size by default
  specs/qcow2: Clarify that compressed clusters have the COPIED bit reset
  Fix error message about compressed clusters with OFLAG_COPIED
  ...

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/0275e6b66c09...c416eecea5f3
      **NOTE:** This service been marked for deprecation: 
https://developer.github.com/changes/2018-04-25-github-services-deprecation/

      Functionality will be removed from GitHub.com on January 31st, 2019.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]