[Qemu-commits] [qemu/qemu] 26d49c: qcow2-refcount: Sanitize refcount table entry

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 26d49c46750aff62c8c6e7a8b14c79d4114a8e53
      
https://github.com/qemu/qemu/commit/26d49c46750aff62c8c6e7a8b14c79d4114a8e53
  Author: Max Reitz <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M block/qcow2-refcount.c

  Log Message:
  -----------
  qcow2-refcount: Sanitize refcount table entry

When reading the refcount table entry in get_refcount(), only bits which
are actually significant for the refcount block offset should be taken
into account.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Laszlo Ersek <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 3456a8d1852e970688b73d03fdc44dde851759e1
      
https://github.com/qemu/qemu/commit/3456a8d1852e970688b73d03fdc44dde851759e1
  Author: Kevin Wolf <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M block.c
    M block/qcow2.c
    M block/qed.c

  Log Message:
  -----------
  block: Update image size in bdrv_invalidate_cache()

After migration has completed, we call bdrv_invalidate_cache() so that
drivers which cache some data drop their stale copy of the data and
reread it from the image file to get a new version of data that the
source modified while the migration was running.

Reloading metadata from the image file is useless, though, if the size
of the image file stays stale (this is a value that is cached for all
image formats in block.c). Reads from (meta)data after the old EOF
return only zeroes, causing image corruption.

We need to update bs->total_sectors in all layers that could potentially
have changed their size (i.e. backing files are not a concern - if they
are changed, we're in bigger trouble)

Signed-off-by: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: dba2855572c746836ad90ce9154403b5929d996b
      
https://github.com/qemu/qemu/commit/dba2855572c746836ad90ce9154403b5929d996b
  Author: Max Reitz <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M block/qcow2-cluster.c

  Log Message:
  -----------
  qcow2: Check bs->drv in copy_sectors()

Before dereferencing bs->drv for a call to its member bdrv_co_readv(),
copy_sectors() should check whether that pointer is indeed valid, since
it may have been set to NULL by e.g. a concurrent write triggering the
corruption prevention mechanism.

Signed-off-by: Max Reitz <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 938789ea92b3073ad1113b3e1bdf90d469ea4bf4
      
https://github.com/qemu/qemu/commit/938789ea92b3073ad1113b3e1bdf90d469ea4bf4
  Author: Max Reitz <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: bs->drv may be NULL in bdrv_debug_resume()

Currently, bdrv_debug_resume() requires every bs->drv in the BDS stack
to be NULL until a bs->drv with an implementation of bdrv_debug_resume()
is found. For a normal function, this would be fine, but this is a
function for debugging purposes and should therefore allow intermediate
BDS not to have a driver (i.e., be "ejected"). Otherwise, it is hard to
debug such situations.

Signed-off-by: Max Reitz <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 98d39e34fe95f8609be3ccbd1b67926631d2c7c0
      
https://github.com/qemu/qemu/commit/98d39e34fe95f8609be3ccbd1b67926631d2c7c0
  Author: Max Reitz <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M tests/qemu-iotests/060
    M tests/qemu-iotests/060.out

  Log Message:
  -----------
  iotests: Test corruption during COW request

Extend test file 060 by a test case for corruption occuring concurrently
to a COW request. QEMU should not crash but rather return an appropriate
error message.

Signed-off-by: Max Reitz <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: b5042a36229b4fa5eeb66bbcde78f704975aec00
      
https://github.com/qemu/qemu/commit/b5042a36229b4fa5eeb66bbcde78f704975aec00
  Author: Benoît Canet <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M block.c
    M block/blkverify.c
    M block/quorum.c
    M include/block/block.h
    M include/block/block_int.h

  Log Message:
  -----------
  block: Rewrite the snapshot authorization mechanism for block filters.

This patch keep the recursive way of doing things but simplify it by giving
two responsabilities to all block filters implementors.

They will need to do two things:

-Set the is_filter field of their block driver to true.

-Implement the bdrv_recurse_is_first_non_filter method of their block driver 
like
it is done on the Quorum block driver. (block/quorum.c)

[Paolo Bonzini <address@hidden> pointed out that this patch changes
the semantics of blkverify, which now recurses down both bs->file and
s->test_file.
-- Stefan]

Reported-by: Paolo Bonzini <address@hidden>
Signed-off-by: Benoit Canet <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 11f590b1a242492a0108da42f40f0e2b20f0a778
      
https://github.com/qemu/qemu/commit/11f590b1a242492a0108da42f40f0e2b20f0a778
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M include/qom/object.h
    M qom/object.c

  Log Message:
  -----------
  object: add object_get_canonical_path_component()

It is often useful to find an object's child property name.  Also use
this new function to simplify the implementation of
object_get_canonical_path().

Reviewed-by: Andreas Färber <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 2da61b671eb89fcaa306738f44eed472977d6587
      
https://github.com/qemu/qemu/commit/2da61b671eb89fcaa306738f44eed472977d6587
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    A include/qemu/rfifolock.h
    M tests/Makefile
    A tests/test-rfifolock.c
    M util/Makefile.objs
    A util/rfifolock.c

  Log Message:
  -----------
  rfifolock: add recursive FIFO lock

QemuMutex does not guarantee fairness and cannot be acquired
recursively:

Fairness means each locker gets a turn and the scheduler cannot cause
starvation.

Recursive locking is useful for composition, it allows a sequence of
locking operations to be invoked atomically by acquiring the lock around
them.

This patch adds RFifoLock, a recursive lock that guarantees FIFO order.
Its first user is added in the next patch.

RFifoLock has one additional feature: it can be initialized with an
optional contention callback.  The callback is invoked whenever a thread
must wait for the lock.  For example, it can be used to poke the current
owner so that they release the lock soon.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 98563fc3ec44c1becce6f1720ad6b0a82ed101b4
      
https://github.com/qemu/qemu/commit/98563fc3ec44c1becce6f1720ad6b0a82ed101b4
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M async.c
    M include/block/aio.h
    M tests/test-aio.c

  Log Message:
  -----------
  aio: add aio_context_acquire() and aio_context_release()

It can be useful to run an AioContext from a thread which normally does
not "own" the AioContext.  For example, request draining can be
implemented by acquiring the AioContext and looping aio_poll() until all
requests have been completed.

The following pattern should work:

  /* Event loop thread */
  while (running) {
      aio_context_acquire(ctx);
      aio_poll(ctx, true);
      aio_context_release(ctx);
  }

  /* Another thread */
  aio_context_acquire(ctx);
  bdrv_read(bs, 0x1000, buf, 1);
  aio_context_release(ctx);

This patch implements aio_context_acquire() and aio_context_release().

Note that existing aio_poll() callers do not need to worry about
acquiring and releasing - it is only needed when multiple threads will
call aio_poll() on the same AioContext.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: be8d8537668c9be7a8dee6aed94b2b3f9fcd4a9f
      
https://github.com/qemu/qemu/commit/be8d8537668c9be7a8dee6aed94b2b3f9fcd4a9f
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M Makefile.objs
    A include/sysemu/iothread.h
    A iothread.c

  Log Message:
  -----------
  iothread: add I/O thread object

This is a stand-in for Michael Roth's QContext.  I expect this to be
replaced once QContext is completed.

The IOThread object is an AioContext event loop thread.  This patch adds
the concept of multiple event loop threads, allowing users to define
them.

When SMP guests run on SMP hosts it makes sense to instantiate multiple
IOThreads.  This spreads event loop processing across multiple cores.
Note that additional patches are required to actually bind a device to
an IOThread.

[Andreas Färber <address@hidden> pointed out that the embedded parent
object instance should be called "parent_obj" and have a newline
afterwards.  This patch has been changed to reflect this.
-- Stefan]

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 7d1de46448d0d52183c397d76fbc86cb614ed21b
      
https://github.com/qemu/qemu/commit/7d1de46448d0d52183c397d76fbc86cb614ed21b
  Author: Igor Mammedov <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M hw/core/qdev-properties-system.c

  Log Message:
  -----------
  qdev: make get_pointer() handle temporary strings

get_pointer()'s print() callback might return a heap allocated
string, to avoid adding dedicated get_pointer_foo for this case
convert current print() callbacks to return temporary heap
allocated string and make get_pointer() free it.

Reviewed-by: Andreas Färber <address@hidden>
Signed-off-by: Igor Mammedov <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 6e4a876b433f78f72724f45ae3f9e26596da1b4d
      
https://github.com/qemu/qemu/commit/6e4a876b433f78f72724f45ae3f9e26596da1b4d
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M hw/core/qdev-properties-system.c
    M include/hw/qdev-properties.h

  Log Message:
  -----------
  iothread: add "iothread" qdev property type

Add a "iothread" qdev property type so devices can be hooked up to an
IOThread from the comand-line:

  qemu -object iothread,id=iothread0 \
       -device some-device,x-iothread=iothread0

Note that Paolo Bonzini <address@hidden> has suggested using QOM
links instead.  This way the relationship between the objects is
reflected in QOM.  There are currently shortcomings of
object_property_add_link() which prevent this use case.  I will attempt
to fix them and move to QOM links in a separate series.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 48ff269272f18d2b8fa53cb08365df417588f585
      
https://github.com/qemu/qemu/commit/48ff269272f18d2b8fa53cb08365df417588f585
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M hw/block/dataplane/virtio-blk.c
    M include/hw/virtio/virtio-blk.h

  Log Message:
  -----------
  dataplane: replace internal thread with IOThread

Today virtio-blk dataplane uses a 1:1 device-per-thread model.  Now that
IOThreads have been introduced we can generalize this to N:M devices per
threads.

This patch drops thread code from dataplane in favor of running inside
an IOThread AioContext.

As a bonus we solve the case where a guest keeps submitting I/O requests
while dataplane is trying to stop.  Previously the dataplane thread
would continue to process requests until the request gave it a break.
Now we can shut down in bounded time thanks to
aio_context_acquire/release.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 88eb7c29e4320597d2f246adf731f0aac97cfbcc
      
https://github.com/qemu/qemu/commit/88eb7c29e4320597d2f246adf731f0aac97cfbcc
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M iothread.c

  Log Message:
  -----------
  iothread: stash thread ID away

Keep the thread ID around so we can report it via QMP.

There's only one problem: qemu_get_thread_id() (gettid() wrapper on
Linux) must be called from the thread itself.  There is no way to get
the thread ID outside the thread.

This patch uses a condvar to wait for iothread_run() to populate the
thread_id inside the thread.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: dc3dd0d2bed6edf3b60041f31200c674348168e9
      
https://github.com/qemu/qemu/commit/dc3dd0d2bed6edf3b60041f31200c674348168e9
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M iothread.c
    M qapi-schema.json
    M qmp-commands.hx

  Log Message:
  -----------
  qmp: add query-iothreads command

The "query-iothreads" command returns a list of information about
iothreads.  See the patch for API documentation.

Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: d475e5acd2f4679d6ce458369ee658dbd60227e9
      
https://github.com/qemu/qemu/commit/d475e5acd2f4679d6ce458369ee658dbd60227e9
  Author: Kevin Wolf <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M block/qcow2.c

  Log Message:
  -----------
  qcow2: Keep option in qcow2_invalidate_cache()

Instead of manually building a list of all options from BDRVQcowState
values just reuse the options that were used to open the image.
qcow2_open() won't fully use all of the options in the QDict, but that's
okay.

This fixes all of the driver-specific options in qcow2, except for
lazy-refcounts, which was special cased before.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 27eb6c097c132bf9fc49d73554b0160293b630cd
      
https://github.com/qemu/qemu/commit/27eb6c097c132bf9fc49d73554b0160293b630cd
  Author: Kevin Wolf <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M block/qcow2.c

  Log Message:
  -----------
  qcow2: Don't write with BDRV_O_INCOMING

qcow2_open() causes writes when repairing an image with the dirty flag
set and when clearing autoclear flags. It shouldn't do this when another
qemu instance is still actively working on this image file.

One effect of the bug is that images may have a cleared dirty flag while
the migration source host still has it in use with lazy refcounts
enabled, so refcounts are not accurate and the dirty flag must remain
set.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 9562f69cfdc55c7c1625bb88df1637fed182e68b
      
https://github.com/qemu/qemu/commit/9562f69cfdc55c7c1625bb88df1637fed182e68b
  Author: Max Reitz <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: Unlink temporary file

If the image file cannot be opened and was created as a temporary file,
it should be deleted; thus, in this case, we should jump to the
"unlink_and_fail" label and not just to "fail".

Reported-by: Benoît Canet <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Acked-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: f988388025c230ef3293cc0c3820cb40e03adfbf
      
https://github.com/qemu/qemu/commit/f988388025c230ef3293cc0c3820cb40e03adfbf
  Author: Stefan Weil <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M include/qemu-io.h
    M qemu-io-cmds.c
    M qemu-io.c

  Log Message:
  -----------
  qemu-io: Fix warnings from static code analysis

Smatch complains about several global symbols which should be local.

Add the missing 'static' attributes and move the 'extern' declaration
of variable qemuio_misalign to qemu-io.h. This variable also changes
the type from 'int' to 'bool' which better fits documents its use.

Signed-off-by: Stefan Weil <address@hidden>
Acked-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 7af803d4f80efb56c250f16409501994e60adf60
      
https://github.com/qemu/qemu/commit/7af803d4f80efb56c250f16409501994e60adf60
  Author: Max Reitz <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M block/raw-posix.c

  Log Message:
  -----------
  block/raw-posix: bdrv_parse_filename() for hdev

The "host_device" protocol driver should strip the "host_device:" prefix
from filenames if present.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Benoit Canet <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: d3f49845831d75fa2e2a5dab04cb78fb12f8a2bc
      
https://github.com/qemu/qemu/commit/d3f49845831d75fa2e2a5dab04cb78fb12f8a2bc
  Author: Max Reitz <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M block/raw-posix.c

  Log Message:
  -----------
  block/raw-posix: bdrv_parse_filename() for floppy

The "host_floppy" protocol driver should strip the "host_floppy:" prefix
from filenames if present.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Benoit Canet <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 18fa1c42a314a3e18f097b28ec5189a144d36f99
      
https://github.com/qemu/qemu/commit/18fa1c42a314a3e18f097b28ec5189a144d36f99
  Author: Max Reitz <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M block/raw-posix.c

  Log Message:
  -----------
  block/raw-posix: bdrv_parse_filename() for cdrom

The "host_cdrom" protocol drivers should strip the "host_cdrom:" prefix
from filenames if present.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Benoit Canet <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: cc28c6aa46334be01f3e65a74601ae38f94bd602
      
https://github.com/qemu/qemu/commit/cc28c6aa46334be01f3e65a74601ae38f94bd602
  Author: Max Reitz <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M block/raw-posix.c

  Log Message:
  -----------
  block/raw-posix: Strip protocol prefix on creation

The hdev_create() implementation in block/raw-posix.c is used by the
"host_device", "host_cdrom" and "host_floppy" protocol block drivers
together. Thus, any of the associated prefixes may occur and exactly one
should should be stripped, if it does (thus,
"host_device:host_cdrom:/dev/cdrom" is not shortened to "/dev/cdrom").

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Benoit Canet <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 57ed25b1b08a43f29326df064d43b6420a23b5ba
      
https://github.com/qemu/qemu/commit/57ed25b1b08a43f29326df064d43b6420a23b5ba
  Author: Max Reitz <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M block/raw-win32.c

  Log Message:
  -----------
  block/raw-win32: bdrv_parse_filename() for hdev

The "host_device" protocol driver should strip the "host_device:" prefix
from filenames if present.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Benoit Canet <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 57fac92c2d4487d5c45e1ca96df6790f96c9e64c
      
https://github.com/qemu/qemu/commit/57fac92c2d4487d5c45e1ca96df6790f96c9e64c
  Author: Peter Maydell <address@hidden>
  Date:   2014-03-13 (Thu, 13 Mar 2014)

  Changed paths:
    M Makefile.objs
    M async.c
    M block.c
    M block/blkverify.c
    M block/qcow2-cluster.c
    M block/qcow2-refcount.c
    M block/qcow2.c
    M block/qed.c
    M block/quorum.c
    M block/raw-posix.c
    M block/raw-win32.c
    M hw/block/dataplane/virtio-blk.c
    M hw/core/qdev-properties-system.c
    M include/block/aio.h
    M include/block/block.h
    M include/block/block_int.h
    M include/hw/qdev-properties.h
    M include/hw/virtio/virtio-blk.h
    M include/qemu-io.h
    A include/qemu/rfifolock.h
    M include/qom/object.h
    A include/sysemu/iothread.h
    A iothread.c
    M qapi-schema.json
    M qemu-io-cmds.c
    M qemu-io.c
    M qmp-commands.hx
    M qom/object.c
    M tests/Makefile
    M tests/qemu-iotests/060
    M tests/qemu-iotests/060.out
    M tests/test-aio.c
    A tests/test-rfifolock.c
    M util/Makefile.objs
    A util/rfifolock.c

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into 
staging

Block pull request

# gpg: Signature made Thu 13 Mar 2014 13:50:49 GMT using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan Hajnoczi <address@hidden>"
# gpg:                 aka "Stefan Hajnoczi <address@hidden>"
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/block-pull-request: (24 commits)
  block/raw-win32: bdrv_parse_filename() for hdev
  block/raw-posix: Strip protocol prefix on creation
  block/raw-posix: bdrv_parse_filename() for cdrom
  block/raw-posix: bdrv_parse_filename() for floppy
  block/raw-posix: bdrv_parse_filename() for hdev
  qemu-io: Fix warnings from static code analysis
  block: Unlink temporary file
  qcow2: Don't write with BDRV_O_INCOMING
  qcow2: Keep option in qcow2_invalidate_cache()
  qmp: add query-iothreads command
  iothread: stash thread ID away
  dataplane: replace internal thread with IOThread
  iothread: add "iothread" qdev property type
  qdev: make get_pointer() handle temporary strings
  iothread: add I/O thread object
  aio: add aio_context_acquire() and aio_context_release()
  rfifolock: add recursive FIFO lock
  object: add object_get_canonical_path_component()
  block: Rewrite the snapshot authorization mechanism for block filters.
  iotests: Test corruption during COW request
  ...

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/41975b269cf1...57fac92c2d44