[Qemu-commits] [qemu/qemu] d134cf: iotests, parallels: quote TEST_IMG in 076 test to ...

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: d134cf73b10e9d0283e1d2531299c8f9ab13b5eb
      
https://github.com/qemu/qemu/commit/d134cf73b10e9d0283e1d2531299c8f9ab13b5eb
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M tests/qemu-iotests/076

  Log Message:
  -----------
  iotests, parallels: quote TEST_IMG in 076 test to be path-safe

suggested by Jeff Cody

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 0789890467d30e2ab10d84b5398bdc903db8cb91
      
https://github.com/qemu/qemu/commit/0789890467d30e2ab10d84b5398bdc903db8cb91
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: rename parallels_header to ParallelsHeader

this follows QEMU coding convention

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 2944256997dd8b080f8b0cc062e4b663cb2ec09c
      
https://github.com/qemu/qemu/commit/2944256997dd8b080f8b0cc062e4b663cb2ec09c
  Author: Roman Kagan <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: switch to bdrv_read

Switch the .bdrv_read method implementation from using bdrv_pread() to
bdrv_read() on the underlying file, since the latter is subject to i/o
throttling while the former is not.

Besides, since bdrv_read() operates in sectors rather than bytes, adjust
the helper functions to do so too.

Signed-off-by: Roman Kagan <address@hidden>
Reviewed-by: Denis V. Lunev <address@hidden>
Signed-off-by: Denis V. Lunev <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
CC: Stefan Hajnoczi <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 9de9da17d8304576e8402697bcee72c88ce499b8
      
https://github.com/qemu/qemu/commit/9de9da17d8304576e8402697bcee72c88ce499b8
  Author: Roman Kagan <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: read up to cluster end in one go

Teach parallels_read() to do reads in coarser granularity than just a
single sector: if requested, read up to the cluster end in one go.

Signed-off-by: Roman Kagan <address@hidden>
Reviewed-by: Denis V. Lunev <address@hidden>
Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: dd3bed16ff229496b30cc77224b0c0ae645c4dae
      
https://github.com/qemu/qemu/commit/dd3bed16ff229496b30cc77224b0c0ae645c4dae
  Author: Roman Kagan <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: add get_block_status

Implement VFS method for get_block_status to Parallels format driver.

qemu_co_mutex_lock is not necessary yet (the driver is read-only) but
will be necessary very soon when write will be supported.

Signed-off-by: Roman Kagan <address@hidden>
Reviewed-by: Denis V. Lunev <address@hidden>
Signed-off-by: Denis V. Lunev <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
CC: Stefan Hajnoczi <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 481fb9cf18925eab19e4af8a44bd86b82eb897ad
      
https://github.com/qemu/qemu/commit/481fb9cf18925eab19e4af8a44bd86b82eb897ad
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: provide _co_readv routine for parallels format driver

Main approach is taken from qcow2_co_readv.

The patch drops coroutine lock for the duration of IO operation and
peforms normal scatter-gather IO using standard QEMU backend.

The patch also adds comment about locking considerations in the driver.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Roman Kagan <address@hidden>
CC: Kevin Wolf <address@hidden>
CC: Stefan Hajnoczi <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 912f31281a683a24b552a8cc6c293ab389b62013
      
https://github.com/qemu/qemu/commit/912f31281a683a24b552a8cc6c293ab389b62013
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: replace magic constants 4, 64 with proper sizeofs

simple purification..

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: d0e61ce56d1520cade573eb344fdb136993d2279
      
https://github.com/qemu/qemu/commit/d0e61ce56d1520cade573eb344fdb136993d2279
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: mark parallels format driver as zero inited

>From the guest point of view unallocated blocks are zeroed.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Roman Kagan <address@hidden>
CC: Kevin Wolf <address@hidden>
CC: Stefan Hajnoczi <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 5a41e1fa95f379e236883f38dacda292f6c48e6f
      
https://github.com/qemu/qemu/commit/5a41e1fa95f379e236883f38dacda292f6c48e6f
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: _co_writev callback for Parallels format

Support write on Parallels images. The code is almost the same as one
in the previous patch implemented scatter-gather IO for read.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Roman Kagan <address@hidden>
CC: Kevin Wolf <address@hidden>
CC: Stefan Hajnoczi <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 50ffd8fd3cfceede87cec1f7f9a04cd7b9147271
      
https://github.com/qemu/qemu/commit/50ffd8fd3cfceede87cec1f7f9a04cd7b9147271
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M tests/qemu-iotests/076
    M tests/qemu-iotests/076.out

  Log Message:
  -----------
  iotests, parallels: test for write into Parallels image

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 74cf6c5026fef6e327f09786445f626df02cbdf0
      
https://github.com/qemu/qemu/commit/74cf6c5026fef6e327f09786445f626df02cbdf0
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: support parallels image creation

Do not even care to create WithoutFreeSpace image, it is obsolete.
Always create WithouFreSpacExt one.

The code also does not spend a lot of efforts to fill cylinders and
heads fields, they are not used actually in a real life neither in
QEMU nor in Parallels products.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: ca9c4e0675f9cb98138e1069605114f45746d985
      
https://github.com/qemu/qemu/commit/ca9c4e0675f9cb98138e1069605114f45746d985
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    A tests/qemu-iotests/131
    A tests/qemu-iotests/131.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  iotests, parallels: test for newly created parallels image via qemu-img

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: cc5690f20fcc075940a213380b362ae2054c03ba
      
https://github.com/qemu/qemu/commit/cc5690f20fcc075940a213380b362ae2054c03ba
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  parallels: change copyright information in the image header

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 369f7de9d57e4dd2f312255fc12271d5749c0a4e
      
https://github.com/qemu/qemu/commit/369f7de9d57e4dd2f312255fc12271d5749c0a4e
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: rename catalog_ names to bat_

BAT means 'block allocation table'. Thus this name is clean and shorter
on writing.

Some obvious formatting fixes in the old code were made to make checkpatch
happy.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 555cc9d9fc5c71be6bd3f288eaf1e5628732088f
      
https://github.com/qemu/qemu/commit/555cc9d9fc5c71be6bd3f288eaf1e5628732088f
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: create bat2sect helper

deduplicate copy/paste arithmetcs

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: dd97cdc064f24484a2ebc141a4ec6bba35f56007
      
https://github.com/qemu/qemu/commit/dd97cdc064f24484a2ebc141a4ec6bba35f56007
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: keep BAT bitmap data in little endian in memory

This will allow to use this data as buffer to BAT update directly
without any intermediate buffers.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 9eae9cca95e76afc2f2288a665e08a64953f2820
      
https://github.com/qemu/qemu/commit/9eae9cca95e76afc2f2288a665e08a64953f2820
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: read parallels image header and BAT into single buffer

This metadata cache would allow to properly batch BAT updates to disk
in next patches. These updates will be properly aligned to avoid
read-modify-write transactions on block level.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 23d6bd3bd1225e8c8ade6ed829eabcf90ddfa6f7
      
https://github.com/qemu/qemu/commit/23d6bd3bd1225e8c8ade6ed829eabcf90ddfa6f7
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: move parallels_open/probe to the very end of the file

This will help to avoid forward declarations for upcoming parallels_check

Some very obvious formatting fixes were made to the moved code to make
checkpatch happy.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 49ad6467313d17486af9029413debb709dc971a8
      
https://github.com/qemu/qemu/commit/49ad6467313d17486af9029413debb709dc971a8
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: implement parallels_check method of block driver

The check is very simple at the moment. It calculates necessary stats
and fix only the following errors:
- space leak at the end of the image. This would happens due to
  preallocation
- clusters outside the image are zeroed. Nothing else could be done here

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 6dd6b9f1440c37811ad963b49a48bf80a8bde377
      
https://github.com/qemu/qemu/commit/6dd6b9f1440c37811ad963b49a48bf80a8bde377
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: implement incorrect close detection

The software driver must set inuse field in Parallels header to
0x746F6E59 when the image is opened in read-write mode. The presence of
this magic in the header on open forces image consistency check.

There is an unfortunate trick here. We can not check for inuse in
parallels_check as this will happen too late. It is possible to do
that for simple check, but during the fix this would always report
an error as the image was opened in BDRV_O_RDWR mode. Thus we save
the flag in BDRVParallelsState for this.

On the other hand, nothing should be done to clear inuse in
parallels_check. Generic close will do the job right.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: a6be831e99f89d72a8c4a114347d5512c326af29
      
https://github.com/qemu/qemu/commit/a6be831e99f89d72a8c4a114347d5512c326af29
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M tests/qemu-iotests/131
    M tests/qemu-iotests/131.out

  Log Message:
  -----------
  iotests, parallels: check for incorrectly closed image in tests

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 6953d920784466dfaea77f7cfd23df2ad8b772a0
      
https://github.com/qemu/qemu/commit/6953d920784466dfaea77f7cfd23df2ad8b772a0
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: improve image reading performance

Try to perform IO for the biggest continuous block possible.
The performance for sequential read is increased from 220 Mb/sec to
360 Mb/sec for continous image on my SSD HDD.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 2d68e22e94e8bc5a0d32a38b53c592c320db8261
      
https://github.com/qemu/qemu/commit/2d68e22e94e8bc5a0d32a38b53c592c320db8261
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: create bat_entry_off helper

calculate offset of the BAT entry in the image file.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 0d31c7c200b3dca2aeeaa6f74ff3fd539aad803a
      
https://github.com/qemu/qemu/commit/0d31c7c200b3dca2aeeaa6f74ff3fd539aad803a
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: delay writing to BAT till bdrv_co_flush_to_os

The idea is that we do not need to immediately sync BAT to the image as
from the guest point of view there is a possibility that IO is lost
even in the physical controller until flush command was finished.
bdrv_co_flush_to_os is exactly the right place for this purpose.

Technically the patch uses loaded BAT data as a cache and performs
actual on-disk metadata updates in parallels_co_flush_to_os callback.

This patch speed ups
  qemu-img create -f parallels -o cluster_size=64k ./1.hds 64G
  qemu-io -f parallels -c "write -P 0x11 0 1024k" 1.hds
writing from 50-60 Mb/sec to 80-90 Mb/sec on rotational media and
from 160 Mb/sec to 190 Mb/sec on SSD disk.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: d61790112fa861fbbbb02b53f9c3beb9ca7f8419
      
https://github.com/qemu/qemu/commit/d61790112fa861fbbbb02b53f9c3beb9ca7f8419
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: add prealloc-mode and prealloc-size open paramemets

This is preparational commit for tweaks in Parallels image expansion.
The idea is that enlarge via truncate by one data block is slow. It
would be much better to use fallocate via bdrv_write_zeroes and
expand by some significant amount at once.

Original idea with sequential file writing to the end of the file without
fallocate/truncate would be slower than this approach if the image is
expanded with several operations:
- each image expanding means file metadata update, i.e. filesystem
  journal write. Truncate/write to newly truncated space update file
  metadata twice thus truncate removal helps. With fallocate call
  inside bdrv_write_zeroes file metadata is updated only once and
  this should happen infrequently thus this approach is the best one
  for the image expansion
- tail writes are ordered, i.e. the guest IO queue could not be sent
  immediately to the host introducing additional IO delays

This patch just adds proper parameters into BDRVParallelsState and
performs options parsing in parallels_open.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Roman Kagan <address@hidden>
CC: Kevin Wolf <address@hidden>
CC: Stefan Hajnoczi <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 19f5dc15912dfb6af06c97e4975023e545e85c72
      
https://github.com/qemu/qemu/commit/19f5dc15912dfb6af06c97e4975023e545e85c72
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: optimize linear image expansion

Plain image expansion spends a lot of time to update image file size.
This seriously affects the performance. The following simple test
  qemu_img create -f parallels -o cluster_size=64k ./1.hds 64G
  qemu_io -n -c "write -P 0x11 0 1024M" ./1.hds
could be improved if the format driver will pre-allocate some space
in the image file with a reasonable chunk.

This patch preallocates 128 Mb using bdrv_write_zeroes, which should
normally use fallocate() call inside. Fallback to older truncate()
could be used as a fallback using image open options thanks to the
previous patch.

The benefit is around 15%.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Karan <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
CC: Stefan Hajnoczi <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: ddd2ef2ce8d693b6e2635a0c20f65744641ff8df
      
https://github.com/qemu/qemu/commit/ddd2ef2ce8d693b6e2635a0c20f65744641ff8df
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  block/parallels: improve image writing performance further

Try to perform IO for the biggest continuous block possible.
All blocks abscent in the image are accounted in the same type
and preallocation is made for all of them at once.

The performance for sequential write is increased from 200 Mb/sec to
235 Mb/sec on my SSD HDD.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Roman Kagan <address@hidden>
Signed-off-by: Roman Kagan <address@hidden>
Message-id: address@hidden
CC: Kevin Wolf <address@hidden>
CC: Stefan Hajnoczi <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: e4a7b344df40b1f4b2e732ddb0d68079ce658d89
      
https://github.com/qemu/qemu/commit/e4a7b344df40b1f4b2e732ddb0d68079ce658d89
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M configure

  Log Message:
  -----------
  configure: handle clang -nopie argument warning

gcc 4.9.2 treats -nopie as an error:

  cc: error: unrecognized command line option ‘-nopie’

clang 3.5.0 treats -nopie as a warning:

  clang: warning: argument unused during compilation: '-nopie'

The causes ./configure to fail with clang:

  ERROR: configure test passed without -Werror but failed with -Werror.

Make the -nopie test use -Werror so that compile_prog works for both gcc
and clang.

Signed-off-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: John Snow <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 93b25869228a3c0c632a6aa66624cc4e549ba14a
      
https://github.com/qemu/qemu/commit/93b25869228a3c0c632a6aa66624cc4e549ba14a
  Author: John Snow <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M configure

  Log Message:
  -----------
  configure: factor out supported flag check

Factor out the function that checks if a compiler
flag is supported or not.

Signed-off-by: John Snow <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: bbbf2e04e5ea347d877c7fa8ee02e4bb647a48fc
      
https://github.com/qemu/qemu/commit/bbbf2e04e5ea347d877c7fa8ee02e4bb647a48fc
  Author: John Snow <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M configure

  Log Message:
  -----------
  configure: silence glib unknown attribute __alloc_size__

The glib headers use GCC attributes.  Unfortunately the __GNUC__ and
__GNUC_MINOR__ version macros are also defined by clang, but clang
doesn't support the same attributes as GCC.

clang 3.5.0 does not support the __alloc_size__ attribute:

  
https://github.com/llvm-mirror/clang/commit/c047507a9a79e89fc8339e074fa72822a7e7ea73

The following warning is produced:

  gstrfuncs.h:257:44: warning: unknown attribute '__alloc_size__' ignored 
[-Wunknown-attributes]
  G_GNUC_MALLOC G_GNUC_ALLOC_SIZE(2);
    gmacros.h:67:45: note: expanded from macro 'G_GNUC_ALLOC_SIZE'
          #define G_GNUC_ALLOC_SIZE(x) __attribute__((__alloc_size__(x)))

This patch checks whether glib headers cause warnings and disables
-Wunknown-attributes if it is able to.

Signed-off-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: John Snow <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: fd0e60530f10078f488fa3e9591cc7db5732989c
      
https://github.com/qemu/qemu/commit/fd0e60530f10078f488fa3e9591cc7db5732989c
  Author: John Snow <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M configure

  Log Message:
  -----------
  configure: Add workaround for ccache and clang

Test if ccache is interfering with semantic analysis of macros,
disable its habit of trying to compile already pre-processed
versions of code if so. ccache attempts to save time by compiling
pre-processed versions of code, but this disturbs clang's static
analysis enough to produce false positives.

ccache allows us to disable this feature, opting instead to
compile the original version instead of its preprocessed version.
This makes ccache much slower for cache misses, but at least it
becomes usable with QEMU/clang.

This workaround only activates for users using ccache AND clang,
and only if their configuration is observed to be producing warnings.
You may need to clear your ccache for builds started without -Werror,
as those may continue to produce warnings from the cache.

Thanks to Peter Eisentraut for his writeup on the issue:
http://peter.eisentraut.org/blog/2014/12/01/ccache-and-clang-part-3/

Signed-off-by: John Snow <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: eaf5fe2dd4ec001d645ff3b343f466457badaa64
      
https://github.com/qemu/qemu/commit/eaf5fe2dd4ec001d645ff3b343f466457badaa64
  Author: Paolo Bonzini <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/io.c

  Log Message:
  -----------
  block: return EPERM on writes or discards to read-only devices

This is the behavior in the operating system, for example Linux's
blkdev_write_iter has the following:
   if (bdev_read_only(I_BDEV(bd_inode)))
          return -EPERM;

This does not apply to opening a device for read/write, when the
device only supports read-only operation.  In this case any of
EACCES, EPERM or EROFS is acceptable depending on why writing is
not possible.

Signed-off-by: Paolo Bonzini <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 4196d2f0308cb1ae13ed450424ab7dfe154acda9
      
https://github.com/qemu/qemu/commit/4196d2f0308cb1ae13ed450424ab7dfe154acda9
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block.c
    M block/io.c
    M block/raw-posix.c
    M include/block/block.h
    M include/block/block_int.h

  Log Message:
  -----------
  block: minimal bounce buffer alignment

The patch introduces new concept: minimal memory alignment for bounce
buffers. Original so called "optimal" value is actually minimal required
value for aligment. It should be used for validation that the IOVec
is properly aligned and bounce buffer is not required.

Though, from the performance point of view, it would be better if
bounce buffer or IOVec allocated by QEMU will be aligned stricter.

The patch does not change any alignment value yet.

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Message-id: address@hidden
CC: Paolo Bonzini <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
CC: Stefan Hajnoczi <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 459b4e66129d091a11e9886ecc15a8bf9f7f3d92
      
https://github.com/qemu/qemu/commit/459b4e66129d091a11e9886ecc15a8bf9f7f3d92
  Author: Denis V. Lunev <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block.c
    M block/io.c
    M block/raw-posix.c

  Log Message:
  -----------
  block: align bounce buffers to page

The following sequence
    int fd = open(argv[1], O_RDWR | O_CREAT | O_DIRECT, 0644);
    for (i = 0; i < 100000; i++)
      write(fd, buf, 4096);
performs 5% better if buf is aligned to 4096 bytes.

The difference is quite reliable.

On the other hand we do not want at the moment to enforce bounce
buffering if guest request is aligned to 512 bytes.

The patch changes default bounce buffer optimal alignment to
MAX(page size, 4k). 4k is chosen as maximal known sector size on real
HDD.

The justification of the performance improve is quite interesting.
>From the kernel point of view each request to the disk was split
by two. This could be seen by blktrace like this:
  9,0   11  1     0.000000000 11151  Q  WS 312737792 + 1023 [qemu-img]
  9,0   11  2     0.000007938 11151  Q  WS 312738815 + 8 [qemu-img]
  9,0   11  3     0.000030735 11151  Q  WS 312738823 + 1016 [qemu-img]
  9,0   11  4     0.000032482 11151  Q  WS 312739839 + 8 [qemu-img]
  9,0   11  5     0.000041379 11151  Q  WS 312739847 + 1016 [qemu-img]
  9,0   11  6     0.000042818 11151  Q  WS 312740863 + 8 [qemu-img]
  9,0   11  7     0.000051236 11151  Q  WS 312740871 + 1017 [qemu-img]
  9,0    5  1     0.169071519 11151  Q  WS 312741888 + 1023 [qemu-img]
After the patch the pattern becomes normal:
  9,0    6  1     0.000000000 12422  Q  WS 314834944 + 1024 [qemu-img]
  9,0    6  2     0.000038527 12422  Q  WS 314835968 + 1024 [qemu-img]
  9,0    6  3     0.000072849 12422  Q  WS 314836992 + 1024 [qemu-img]
  9,0    6  4     0.000106276 12422  Q  WS 314838016 + 1024 [qemu-img]
and the amount of requests sent to disk (could be calculated counting
number of lines in the output of blktrace) is reduced about 2 times.

Both qemu-img and qemu-io are affected while qemu-kvm is not. The guest
does his job well and real requests comes properly aligned (to page).

Signed-off-by: Denis V. Lunev <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Message-id: address@hidden
CC: Paolo Bonzini <address@hidden>
CC: Kevin Wolf <address@hidden>
CC: Stefan Hajnoczi <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: d01c07f2221ca39ab2dd9e55932d99db98103b30
      
https://github.com/qemu/qemu/commit/d01c07f2221ca39ab2dd9e55932d99db98103b30
  Author: Fam Zheng <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/io.c

  Log Message:
  -----------
  Revert "block: Fix unaligned zero write"

This reverts commit fc3959e4669a1c2149b91ccb05101cfc7ae1fc05.

The core write code already handles the case, so remove this
duplication.

Because commit 61007b316 moved the touched code from block.c to
block/io.c, the change is manually reverted.

Signed-off-by: Fam Zheng <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 9eeb6dd1b27bd57eb4e3869290e87feac8e8b226
      
https://github.com/qemu/qemu/commit/9eeb6dd1b27bd57eb4e3869290e87feac8e8b226
  Author: Fam Zheng <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/io.c

  Log Message:
  -----------
  block: Fix NULL deference for unaligned write if qiov is NULL

For zero write, callers pass in NULL qiov (qemu-io "write -z" or
scsi-disk "write same").

Commit fc3959e466 fixed bdrv_co_write_zeroes which is the common case
for this bug, but it still exists in bdrv_aio_write_zeroes. A simpler
fix would be in bdrv_co_do_pwritev which is the NULL dereference point
and covers both cases.

So don't access it in bdrv_co_do_pwritev in this case, use three aligned
writes.

[Initialize ret to 0 in bdrv_co_do_zero_pwritev() to avoid uninitialized
variable warning with gcc 4.9.2.
--Stefan]

Signed-off-by: Fam Zheng <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: ab53c44718305d3fde3d9d2251889f1cab694be2
      
https://github.com/qemu/qemu/commit/ab53c44718305d3fde3d9d2251889f1cab694be2
  Author: Fam Zheng <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M tests/qemu-iotests/033
    M tests/qemu-iotests/033.out

  Log Message:
  -----------
  qemu-iotests: Test unaligned sub-block zero write

Test zero write in byte range 512~1024 for 4k alignment.

Signed-off-by: Fam Zheng <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: a53f1a95f9605f300fbafbc8b60b8a8c67e9c4b4
      
https://github.com/qemu/qemu/commit/a53f1a95f9605f300fbafbc8b60b8a8c67e9c4b4
  Author: Paolo Bonzini <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block/io.c

  Log Message:
  -----------
  block: get_block_status: use "else" when testing the opposite condition

A bit of Boolean algebra (and common sense) tells us that the
second "if" here is looking for blocks that are not allocated.
This is the opposite of the "if" that sets BDRV_BLOCK_ALLOCATED,
and thus it can use an "else".

Signed-off-by: Paolo Bonzini <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 8b6db32a4ec47d1171ccfa21d557096b99f4eef0
      
https://github.com/qemu/qemu/commit/8b6db32a4ec47d1171ccfa21d557096b99f4eef0
  Author: Peter Maydell <address@hidden>
  Date:   2015-05-22 (Fri, 22 May 2015)

  Changed paths:
    M block.c
    M block/io.c
    M block/parallels.c
    M block/raw-posix.c
    M configure
    M include/block/block.h
    M include/block/block_int.h
    M tests/qemu-iotests/033
    M tests/qemu-iotests/033.out
    M tests/qemu-iotests/076
    M tests/qemu-iotests/076.out
    A tests/qemu-iotests/131
    A tests/qemu-iotests/131.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into 
staging

# gpg: Signature made Fri May 22 10:00:53 2015 BST using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan Hajnoczi <address@hidden>"
# gpg:                 aka "Stefan Hajnoczi <address@hidden>"

* remotes/stefanha/tags/block-pull-request: (38 commits)
  block: get_block_status: use "else" when testing the opposite condition
  qemu-iotests: Test unaligned sub-block zero write
  block: Fix NULL deference for unaligned write if qiov is NULL
  Revert "block: Fix unaligned zero write"
  block: align bounce buffers to page
  block: minimal bounce buffer alignment
  block: return EPERM on writes or discards to read-only devices
  configure: Add workaround for ccache and clang
  configure: silence glib unknown attribute __alloc_size__
  configure: factor out supported flag check
  configure: handle clang -nopie argument warning
  block/parallels: improve image writing performance further
  block/parallels: optimize linear image expansion
  block/parallels: add prealloc-mode and prealloc-size open paramemets
  block/parallels: delay writing to BAT till bdrv_co_flush_to_os
  block/parallels: create bat_entry_off helper
  block/parallels: improve image reading performance
  iotests, parallels: check for incorrectly closed image in tests
  block/parallels: implement incorrect close detection
  block/parallels: implement parallels_check method of block driver
  ...

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/f5790c3bc817...8b6db32a4ec4