[Qemu-commits] [qemu/qemu] 406574: raw-posix: add raw_get_aio_fd() for virtio-blk-dat...

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 4065742ac0f3c84abdd8d718b44a88f3ac56015a
      
https://github.com/qemu/qemu/commit/4065742ac0f3c84abdd8d718b44a88f3ac56015a
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2013-01-02 (Wed, 02 Jan 2013)

  Changed paths:
    M block/raw-posix.c
    M include/block/block.h

  Log Message:
  -----------
  raw-posix: add raw_get_aio_fd() for virtio-blk-data-plane

The raw_get_aio_fd() function allows virtio-blk-data-plane to get the
file descriptor of a raw image file with Linux AIO enabled.  This
interface is really a layering violation that can be resolved once the
block layer is able to run outside the global mutex - at that point
virtio-blk-data-plane will switch from custom Linux AIO code to using
the block layer.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 583f6e7bbd24b31f7eecd5c21ba0a5a5a77f52f1
      
https://github.com/qemu/qemu/commit/583f6e7bbd24b31f7eecd5c21ba0a5a5a77f52f1
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2013-01-02 (Wed, 02 Jan 2013)

  Changed paths:
    M configure

  Log Message:
  -----------
  configure: add CONFIG_VIRTIO_BLK_DATA_PLANE

The virtio-blk-data-plane feature only works with Linux AIO.  Therefore
add a ./configure option and necessary checks to implement this
dependency.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 185ecf40e3589fc2717b0856ee1df05dd63a46dd
      
https://github.com/qemu/qemu/commit/185ecf40e3589fc2717b0856ee1df05dd63a46dd
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2013-01-02 (Wed, 02 Jan 2013)

  Changed paths:
    M hw/Makefile.objs
    A hw/dataplane/Makefile.objs
    A hw/dataplane/hostmem.c
    A hw/dataplane/hostmem.h

  Log Message:
  -----------
  dataplane: add host memory mapping code

The data plane thread needs to map guest physical addresses to host
pointers.  Normally this is done with cpu_physical_memory_map() but the
function assumes the global mutex is held.  The data plane thread does
not touch the global mutex and therefore needs a thread-safe memory
mapping mechanism.

Hostmem registers a MemoryListener similar to how vhost collects and
pushes memory region information into the kernel.  There is a
fine-grained lock on the regions list which is held during lookup and
when installing a new regions list.

When the physical memory map changes the MemoryListener callbacks are
invoked.  They build up a new list of memory regions which is finally
installed when the list has been completed.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 88807f89d945acad54c8365ff7b6ef0f0d0ddd56
      
https://github.com/qemu/qemu/commit/88807f89d945acad54c8365ff7b6ef0f0d0ddd56
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2013-01-02 (Wed, 02 Jan 2013)

  Changed paths:
    M hw/dataplane/Makefile.objs
    A hw/dataplane/vring.c
    A hw/dataplane/vring.h
    M trace-events

  Log Message:
  -----------
  dataplane: add virtqueue vring code

The virtio-blk-data-plane cannot access memory using the usual QEMU
functions since it executes outside the global mutex and the memory APIs
are this time are not thread-safe.

This patch introduces a virtqueue module based on the kernel's vhost
vring code.  The trick is that we map guest memory ahead of time and
access it cheaply outside the global mutex.

Once the hardware emulation code can execute outside the global mutex it
will be possible to drop this code.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 71973b046120a13df4eaa9143bed5ba8a67abc7f
      
https://github.com/qemu/qemu/commit/71973b046120a13df4eaa9143bed5ba8a67abc7f
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2013-01-02 (Wed, 02 Jan 2013)

  Changed paths:
    M hw/dataplane/Makefile.objs
    A hw/dataplane/event-poll.c
    A hw/dataplane/event-poll.h

  Log Message:
  -----------
  dataplane: add event loop

Outside the safety of the global mutex we need to poll on file
descriptors.  I found epoll(2) is a convenient way to do that, although
other options could replace this module in the future (such as an
AioContext-based loop or glib's GMainLoop).

One important feature of this small event loop implementation is that
the loop can be terminated in a thread-safe way.  This allows QEMU to
stop the data plane thread cleanly.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 3e9ec521711ed033476098cfc7f23c992cc606a2
      
https://github.com/qemu/qemu/commit/3e9ec521711ed033476098cfc7f23c992cc606a2
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2013-01-02 (Wed, 02 Jan 2013)

  Changed paths:
    M hw/dataplane/Makefile.objs
    A hw/dataplane/ioq.c
    A hw/dataplane/ioq.h

  Log Message:
  -----------
  dataplane: add Linux AIO request queue

The IOQueue has a pool of iocb structs and a function to add new
read/write requests.  Multiple requests can be added before calling the
submit function to actually tell the host kernel to begin I/O.  This
allows callers to batch requests and submit them in one go.

The actual I/O is performed using Linux AIO.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: d02776350d9c76348988fc9e58a64a4f6b1a9f61
      
https://github.com/qemu/qemu/commit/d02776350d9c76348988fc9e58a64a4f6b1a9f61
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2013-01-02 (Wed, 02 Jan 2013)

  Changed paths:
    M include/qemu/iov.h
    M iov.c

  Log Message:
  -----------
  iov: add iov_discard_front/back() to remove data

The iov_discard_front/back() functions remove data from the front or
back of the vector.  This is useful when peeling off header/footer
structs.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 8962e44fe438a051aff9f43209363f599be33624
      
https://github.com/qemu/qemu/commit/8962e44fe438a051aff9f43209363f599be33624
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2013-01-02 (Wed, 02 Jan 2013)

  Changed paths:
    M tests/test-iov.c

  Log Message:
  -----------
  test-iov: add iov_discard_front/back() testcases

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 530c0bbd73e1b658c9266582072847de1fbdff10
      
https://github.com/qemu/qemu/commit/530c0bbd73e1b658c9266582072847de1fbdff10
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2013-01-02 (Wed, 02 Jan 2013)

  Changed paths:
    M include/qemu-common.h
    M iov.c

  Log Message:
  -----------
  iov: add qemu_iovec_concat_iov()

The qemu_iovec_concat() function copies a subset of a QEMUIOVector.  The
new qemu_iovec_concat_iov() function does the same for a iov/cnt pair.

It is easy to define qemu_iovec_concat() in terms of
qemu_iovec_concat_iov().  The existing code is mostly unchanged, except
for the assertion src->size >= soffset, which cannot be efficiently
checked upfront on a iov/cnt pair.  Instead we assert upon hitting the
end of src with an unsatisfied soffset.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 8a873ba78069ef81c4ef073a0bd703172c8b3312
      
https://github.com/qemu/qemu/commit/8a873ba78069ef81c4ef073a0bd703172c8b3312
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2013-01-02 (Wed, 02 Jan 2013)

  Changed paths:
    M hw/virtio-blk.c
    M hw/virtio-blk.h
    M hw/virtio-pci.c

  Log Message:
  -----------
  virtio-blk: restore VirtIOBlkConf->config_wce flag

Two slightly different versions of a patch to conditionally set
VIRTIO_BLK_F_CONFIG_WCE through the "config-wce" qdev property have been
applied (ea776abca and eec7f96c2).  David Gibson
<address@hidden> noticed that the "config-wce"
property is broken as a result and fixed it recently.

The fix sets the host_features VIRTIO_BLK_F_CONFIG_WCE bit from a qdev
property.  Unfortunately, the virtio device then has no chance to test
for the presence of the feature bit during virtio_blk_init().

Therefore, reinstate the VirtIOBlkConf->config_wce flag.  Drop the
duplicate qdev property to set the host_features bit.  The
VirtIOBlkConf->config_wce flag will be used by virtio-blk-data-plane in
a later patch.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: e72f66a0a20f38d0c7576f6c0aec0ca644976e35
      
https://github.com/qemu/qemu/commit/e72f66a0a20f38d0c7576f6c0aec0ca644976e35
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2013-01-02 (Wed, 02 Jan 2013)

  Changed paths:
    M hw/dataplane/Makefile.objs
    A hw/dataplane/virtio-blk.c
    A hw/dataplane/virtio-blk.h
    M hw/virtio-blk.h
    M trace-events

  Log Message:
  -----------
  dataplane: add virtio-blk data plane code

virtio-blk-data-plane is a subset implementation of virtio-blk.  It only
handles read, write, and flush requests.  It does this using a dedicated
thread that executes an epoll(2)-based event loop and processes I/O
using Linux AIO.

This approach performs very well but can be used for raw image files
only.  The number of IOPS achieved has been reported to be several times
higher than the existing virtio-blk implementation.

Eventually it should be possible to unify virtio-blk-data-plane with the
main body of QEMU code once the block layer and hardware emulation is
able to run outside the global mutex.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 392808b49b6aee066d0c1d200e72fc3dc11c9d0f
      
https://github.com/qemu/qemu/commit/392808b49b6aee066d0c1d200e72fc3dc11c9d0f
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2013-01-02 (Wed, 02 Jan 2013)

  Changed paths:
    M hw/virtio-blk.c
    M hw/virtio-pci.c

  Log Message:
  -----------
  virtio-blk: add x-data-plane=on|off performance feature

The virtio-blk-data-plane feature is easy to integrate into
hw/virtio-blk.c.  The data plane can be started and stopped similar to
vhost-net.

Users can take advantage of the virtio-blk-data-plane feature using the
new -device virtio-blk-pci,x-data-plane=on property.

The x-data-plane name was chosen because at this stage the feature is
experimental and likely to see changes in the future.

If the VM configuration does not support virtio-blk-data-plane an error
message is printed.  Although we could fall back to regular virtio-blk,
I prefer the explicit approach since it prompts the user to fix their
configuration if they want the performance benefit of
virtio-blk-data-plane.

Limitations:
 * Only format=raw is supported
 * Live migration is not supported
 * Block jobs, hot unplug, and other operations fail with -EBUSY
 * I/O throttling limits are ignored
 * Only Linux hosts are supported due to Linux AIO usage

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 9e72c45033770b81b536ac6091e91807247cc25a
      
https://github.com/qemu/qemu/commit/9e72c45033770b81b536ac6091e91807247cc25a
  Author: Alexey Zaytsev <address@hidden>
  Date:   2013-01-02 (Wed, 02 Jan 2013)

  Changed paths:
    M hw/virtio-blk.c

  Log Message:
  -----------
  virtio-blk: Return UNSUPP for unknown request types

Currently, all unknown requests are treated as VIRTIO_BLK_T_IN

Signed-off-by: Alexey Zaytsev <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 37edbf7ea8067262a5c3d8bbe4786139348c8311
      
https://github.com/qemu/qemu/commit/37edbf7ea8067262a5c3d8bbe4786139348c8311
  Author: liguang <address@hidden>
  Date:   2013-01-02 (Wed, 02 Jan 2013)

  Changed paths:
    M cutils.c

  Log Message:
  -----------
  cutils: change strtosz_suffix_unit function

if value to be translated is larger than INT64_MAX,
this function will not be convenient for caller to
be aware of it, so change a little for this.

Signed-off-by: liguang <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 7944339726b4582b67fd94085c21c33636e8f973
      
https://github.com/qemu/qemu/commit/7944339726b4582b67fd94085c21c33636e8f973
  Author: liguang <address@hidden>
  Date:   2013-01-02 (Wed, 02 Jan 2013)

  Changed paths:
    M qemu-img.c

  Log Message:
  -----------
  qemu-img: report size overflow error message

qemu-img will complain when qcow or qcow2
size overflow for 64 bits, report the right
message in this condition.

$./qemu-img create -f qcow2 /tmp/foo 0x10000000000000000
before change:
qemu-img: Invalid image size specified! You may use k, M, G or T suffixes for
qemu-img: kilobytes, megabytes, gigabytes and terabytes.

after change:
qemu-img: Image size must be less than 8 EiB!

[Resolved conflict with a9300911 goto removal -- Stefan]

Signed-off-by: liguang <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: fccedc624c425e3acb1557f9f9b13104427ec5ce
      
https://github.com/qemu/qemu/commit/fccedc624c425e3acb1557f9f9b13104427ec5ce
  Author: Stefan Weil <address@hidden>
  Date:   2013-01-02 (Wed, 02 Jan 2013)

  Changed paths:
    M block/raw-win32.c

  Log Message:
  -----------
  block/raw-win32: Fix compiler warnings (wrong format specifiers)

Commit fbcad04d6bfdff937536eb23088a01a280a1a3af added fprintf statements
with wrong format specifiers.

GetLastError() returns a DWORD which is unsigned long, so %lu must be used.

Signed-off-by: Stefan Weil <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: bd751f2204a03d6fcd47a4b4b12ac534d2ecbea7
      
https://github.com/qemu/qemu/commit/bd751f2204a03d6fcd47a4b4b12ac534d2ecbea7
  Author: Liu Yuan <address@hidden>
  Date:   2013-01-02 (Wed, 02 Jan 2013)

  Changed paths:
    M block/sheepdog.c

  Log Message:
  -----------
  sheepdog: don't update inode when create_and_write fails

For the error case such as SD_RES_NO_SPACE, we shouldn't update the inode bitmap
to avoid the scenario that the object is allocated but wasn't created at the
server side. This will result in VM's IO error on the failed object.

Cc: MORITA Kazutaka <address@hidden>
Cc: Kevin Wolf <address@hidden>
Signed-off-by: Liu Yuan <address@hidden>
Reviewed-by: MORITA Kazutaka <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: d6b1ef89a1ede41334e4d0fa27e600e0b4d4f209
      
https://github.com/qemu/qemu/commit/d6b1ef89a1ede41334e4d0fa27e600e0b4d4f209
  Author: Liu Yuan <address@hidden>
  Date:   2013-01-02 (Wed, 02 Jan 2013)

  Changed paths:
    M block/sheepdog.c

  Log Message:
  -----------
  sheepdog: pass oid directly to send_pending_req()

Cc: MORITA Kazutaka <address@hidden>
Cc: Kevin Wolf <address@hidden>
Signed-off-by: Liu Yuan <address@hidden>
Reviewed-by: MORITA Kazutaka <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 217da7fdeb2a4c99c49f22f9dc64c8df2e3a4387
      
https://github.com/qemu/qemu/commit/217da7fdeb2a4c99c49f22f9dc64c8df2e3a4387
  Author: Anthony Liguori <address@hidden>
  Date:   2013-01-02 (Wed, 02 Jan 2013)

  Changed paths:
    M block/raw-posix.c
    M block/raw-win32.c
    M block/sheepdog.c
    M configure
    M cutils.c
    M hw/Makefile.objs
    A hw/dataplane/Makefile.objs
    A hw/dataplane/event-poll.c
    A hw/dataplane/event-poll.h
    A hw/dataplane/hostmem.c
    A hw/dataplane/hostmem.h
    A hw/dataplane/ioq.c
    A hw/dataplane/ioq.h
    A hw/dataplane/virtio-blk.c
    A hw/dataplane/virtio-blk.h
    A hw/dataplane/vring.c
    A hw/dataplane/vring.h
    M hw/virtio-blk.c
    M hw/virtio-blk.h
    M hw/virtio-pci.c
    M include/block/block.h
    M include/qemu-common.h
    M include/qemu/iov.h
    M iov.c
    M qemu-img.c
    M tests/test-iov.c
    M trace-events

  Log Message:
  -----------
  Merge remote-tracking branch 'stefanha/block' into staging

* stefanha/block:
  sheepdog: pass oid directly to send_pending_req()
  sheepdog: don't update inode when create_and_write fails
  block/raw-win32: Fix compiler warnings (wrong format specifiers)
  qemu-img: report size overflow error message
  cutils: change strtosz_suffix_unit function
  virtio-blk: Return UNSUPP for unknown request types
  virtio-blk: add x-data-plane=on|off performance feature
  dataplane: add virtio-blk data plane code
  virtio-blk: restore VirtIOBlkConf->config_wce flag
  iov: add qemu_iovec_concat_iov()
  test-iov: add iov_discard_front/back() testcases
  iov: add iov_discard_front/back() to remove data
  dataplane: add Linux AIO request queue
  dataplane: add event loop
  dataplane: add virtqueue vring code
  dataplane: add host memory mapping code
  configure: add CONFIG_VIRTIO_BLK_DATA_PLANE
  raw-posix: add raw_get_aio_fd() for virtio-blk-data-plane

Signed-off-by: Anthony Liguori <address@hidden>


Compare: https://github.com/qemu/qemu/compare/9a8a5ae69d3a...217da7fdeb2a