qemu-commits
[Top][All Lists]
Advanced
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Qemu-commits] [qemu/qemu] fcc676: block: Introduce API for copy offload

From: GitHub
Subject: [Qemu-commits] [qemu/qemu] fcc676: block: Introduce API for copy offloading
Date: Tue, 05 Jun 2018 02:38:13 -0700 
  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: fcc6767836efe1b160289905dce7228d594c123c
      
https://github.com/qemu/qemu/commit/fcc6767836efe1b160289905dce7228d594c123c
  Author: Fam Zheng <address@hidden>
  Date:   2018-06-01 (Fri, 01 Jun 2018)

  Changed paths:
    M block/io.c
    M include/block/block.h
    M include/block/block_int.h

  Log Message:
  -----------
  block: Introduce API for copy offloading

Introduce the bdrv_co_copy_range() API for copy offloading.  Block
drivers implementing this API support efficient copy operations that
avoid reading each block from the source device and writing it to the
destination devices.  Examples of copy offload primitives are SCSI
EXTENDED COPY and Linux copy_file_range(2).

Signed-off-by: Fam Zheng <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 384455385248762e74a080978f18f0c8f74757fe
      
https://github.com/qemu/qemu/commit/384455385248762e74a080978f18f0c8f74757fe
  Author: Fam Zheng <address@hidden>
  Date:   2018-06-01 (Fri, 01 Jun 2018)

  Changed paths:
    M block/raw-format.c

  Log Message:
  -----------
  raw: Check byte range uniformly

We don't verify the request range against s->size in the I/O callbacks
except for raw_co_pwritev. This is inconsistent (especially for
raw_co_pwrite_zeroes and raw_co_pdiscard), so fix them, in the meanwhile
make the helper reusable by the coming new callbacks.

Note that in most cases the block layer already verifies the request
byte range against our reported image length, before invoking the driver
callbacks.  The exception is during image creating, after
blk_set_allow_write_beyond_eof(blk, true) is called. But in that case,
the requests are not directly from the user or guest. So there is no
visible behavior change in adding the check code.

The int64_t -> uint64_t inconsistency, as shown by the type casting, is
pre-existing due to the interface.

Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Fam Zheng <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 72d219e2f916adeec9845473d239571a267f3314
      
https://github.com/qemu/qemu/commit/72d219e2f916adeec9845473d239571a267f3314
  Author: Fam Zheng <address@hidden>
  Date:   2018-06-01 (Fri, 01 Jun 2018)

  Changed paths:
    M block/raw-format.c

  Log Message:
  -----------
  raw: Implement copy offloading

Just pass down to ->file.

Signed-off-by: Fam Zheng <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: fd9fcd37a8645efe322956d94f76e90135522a16
      
https://github.com/qemu/qemu/commit/fd9fcd37a8645efe322956d94f76e90135522a16
  Author: Fam Zheng <address@hidden>
  Date:   2018-06-01 (Fri, 01 Jun 2018)

  Changed paths:
    M block/qcow2.c

  Log Message:
  -----------
  qcow2: Implement copy offloading

The two callbacks are implemented quite similarly to the read/write
functions: bdrv_co_copy_range_from maps for read and calls into bs->file
or bs->backing depending on the allocation status; bdrv_co_copy_range_to
maps for write and calls into bs->file.

Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Fam Zheng <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 1efad060d7e131dd52ecd1e038a6ddd37a3940c8
      
https://github.com/qemu/qemu/commit/1efad060d7e131dd52ecd1e038a6ddd37a3940c8
  Author: Fam Zheng <address@hidden>
  Date:   2018-06-01 (Fri, 01 Jun 2018)

  Changed paths:
    M block/file-posix.c
    M configure
    M include/block/raw-aio.h

  Log Message:
  -----------
  file-posix: Implement bdrv_co_copy_range

With copy_file_range(2), we can implement the bdrv_co_copy_range
semantics.

Signed-off-by: Fam Zheng <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: cc9743c236cce8a35449e3ef67140287b68bb705
      
https://github.com/qemu/qemu/commit/cc9743c236cce8a35449e3ef67140287b68bb705
  Author: Fam Zheng <address@hidden>
  Date:   2018-06-01 (Fri, 01 Jun 2018)

  Changed paths:
    M block/iscsi.c

  Log Message:
  -----------
  iscsi: Query and save device designator when opening

The device designator data returned in INQUIRY command will be useful to
fill in source/target fields during copy offloading. Do this when
connecting to the target and save the data for later use.

Signed-off-by: Fam Zheng <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 66e75c03b2247bda6dcaa883b700291bc0f7f0ef
      
https://github.com/qemu/qemu/commit/66e75c03b2247bda6dcaa883b700291bc0f7f0ef
  Author: Fam Zheng <address@hidden>
  Date:   2018-06-01 (Fri, 01 Jun 2018)

  Changed paths:
    M block/iscsi.c

  Log Message:
  -----------
  iscsi: Create and use iscsi_co_wait_for_task

This loop is repeated a growing number times. Make a helper.

Signed-off-by: Fam Zheng <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 604dfaaa3270081da689991afe83d94d3e8231df
      
https://github.com/qemu/qemu/commit/604dfaaa3270081da689991afe83d94d3e8231df
  Author: Fam Zheng <address@hidden>
  Date:   2018-06-01 (Fri, 01 Jun 2018)

  Changed paths:
    M block/iscsi.c
    M include/scsi/constants.h

  Log Message:
  -----------
  iscsi: Implement copy offloading

Issue EXTENDED COPY (LID1) command to implement the copy_range API.

The parameter data construction code is modified from libiscsi's
iscsi-dd.c.

Signed-off-by: Fam Zheng <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: b5679fa49c9a70efa7bf01f6efad1a65e2349a0b
      
https://github.com/qemu/qemu/commit/b5679fa49c9a70efa7bf01f6efad1a65e2349a0b
  Author: Fam Zheng <address@hidden>
  Date:   2018-06-01 (Fri, 01 Jun 2018)

  Changed paths:
    M block/block-backend.c
    M include/sysemu/block-backend.h

  Log Message:
  -----------
  block-backend: Add blk_co_copy_range

It's a BlockBackend wrapper of the BDS interface.

Signed-off-by: Fam Zheng <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: ee5306d0923377439776e8a30b9fd2de34b5cbfb
      
https://github.com/qemu/qemu/commit/ee5306d0923377439776e8a30b9fd2de34b5cbfb
  Author: Fam Zheng <address@hidden>
  Date:   2018-06-01 (Fri, 01 Jun 2018)

  Changed paths:
    M qemu-img.c

  Log Message:
  -----------
  qemu-img: Convert with copy offloading

The new blk_co_copy_range interface offers a more efficient way in the
case of network based storage. Make use of it to allow faster convert
operation.

Since copy offloading cannot do zero detection ('-S') and compression
(-c), only try it when these options are not used.

Signed-off-by: Fam Zheng <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 21891a5a3011608845b5d7f1f9cce60cdc2bcc62
      
https://github.com/qemu/qemu/commit/21891a5a3011608845b5d7f1f9cce60cdc2bcc62
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2018-06-01 (Fri, 01 Jun 2018)

  Changed paths:
    M tests/qemu-iotests/common.filter
    M util/main-loop.c

  Log Message:
  -----------
  main-loop: drop spin_counter

Commit d759c951f3287fad04210a52f2dc93f94cf58c7f ("replay: push
replay_mutex_lock up the call tree") removed the !timeout lock
optimization in the main loop.

The idea of the optimization was to avoid ping-pongs between threads by
keeping the Big QEMU Lock held across non-blocking (!timeout) main loop
iterations.

A warning is printed when the main loop spins without releasing BQL for
long periods of time.  These warnings were supposed to aid debugging but
in practice they just alarm users.  They are considered noise because
the cause of spinning is not shown and is hard to find.

Now that the lock optimization has been removed, there is no danger of
hogging the BQL.  Drop the spin counter and the infamous warning.

Signed-off-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Jeff Cody <address@hidden>


  Commit: 0d514fa23402ab7b4f1c965e0631d953bbe4d3b7
      
https://github.com/qemu/qemu/commit/0d514fa23402ab7b4f1c965e0631d953bbe4d3b7
  Author: Peter Maydell <address@hidden>
  Date:   2018-06-04 (Mon, 04 Jun 2018)

  Changed paths:
    M block/block-backend.c
    M block/file-posix.c
    M block/io.c
    M block/iscsi.c
    M block/qcow2.c
    M block/raw-format.c
    M configure
    M include/block/block.h
    M include/block/block_int.h
    M include/block/raw-aio.h
    M include/scsi/constants.h
    M include/sysemu/block-backend.h
    M qemu-img.c
    M tests/qemu-iotests/common.filter
    M util/main-loop.c

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into 
staging

Pull request

 * Copy offloading for qemu-img convert (iSCSI, raw, and qcow2)

   If the underlying storage supports copy offloading, qemu-img convert will
   use it instead of performing reads and writes.  This avoids data transfers
   and thus frees up storage bandwidth for other purposes.  SCSI EXTENDED COPY
   and Linux copy_file_range(2) are used to implement this optimization.

 * Drop spurious "WARNING: I\/O thread spun for 1000 iterations" warning

# gpg: Signature made Mon 04 Jun 2018 12:20:08 BST
# gpg:                using RSA key 9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <address@hidden>"
# gpg:                 aka "Stefan Hajnoczi <address@hidden>"
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/block-pull-request:
  main-loop: drop spin_counter
  qemu-img: Convert with copy offloading
  block-backend: Add blk_co_copy_range
  iscsi: Implement copy offloading
  iscsi: Create and use iscsi_co_wait_for_task
  iscsi: Query and save device designator when opening
  file-posix: Implement bdrv_co_copy_range
  qcow2: Implement copy offloading
  raw: Implement copy offloading
  raw: Check byte range uniformly
  block: Introduce API for copy offloading

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/5d7ad3ce103a...0d514fa23402
      **NOTE:** This service been marked for deprecation: 
https://developer.github.com/changes/2018-04-25-github-services-deprecation/

      Functionality will be removed from GitHub.com on January 31st, 2019.
reply via email to
[Prev in Thread] Current Thread [Next in Thread]