[Qemu-commits] [qemu/qemu] 0c3041: iotests: Update 051's reference output

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 0c304110bd7092cbabff55b45193e13549b53e40
      
https://github.com/qemu/qemu/commit/0c304110bd7092cbabff55b45193e13549b53e40
  Author: Markus Armbruster <address@hidden>
  Date:   2015-03-16 (Mon, 16 Mar 2015)

  Changed paths:
    M tests/qemu-iotests/051.out

  Log Message:
  -----------
  iotests: Update 051's reference output

Commit c4bacaf improved error reporting, but neglected to update
051.out.  Commit 2726958 tried to redress, but didn't get it quite
right (punctuation difference), and shortly after commit
ae071cc..master improved error reporting some more, neglecting 051.out
some more.  Sorry!

Signed-off-by: Markus Armbruster <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 97a2ca7ae6bccb78b2acf10d6d3eebd1e9f3bbdc
      
https://github.com/qemu/qemu/commit/97a2ca7ae6bccb78b2acf10d6d3eebd1e9f3bbdc
  Author: Markus Armbruster <address@hidden>
  Date:   2015-03-16 (Mon, 16 Mar 2015)

  Changed paths:
    M qemu-img.c
    M tests/qemu-iotests/061.out

  Log Message:
  -----------
  qemu-img: Fix convert, amend error messages for unknown options

Message quality regressed in commit dc523cd.

Signed-off-by: Markus Armbruster <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: a1f688f4152e65260b94f37543521ceff8bfebe4
      
https://github.com/qemu/qemu/commit/a1f688f4152e65260b94f37543521ceff8bfebe4
  Author: Markus Armbruster <address@hidden>
  Date:   2015-03-16 (Mon, 16 Mar 2015)

  Changed paths:
    M block.c
    M qemu-doc.texi
    M tests/qemu-iotests/049.out
    M tests/qemu-iotests/087.out

  Log Message:
  -----------
  block: Deprecate QCOW/QCOW2 encryption

We've steered users away from QCOW/QCOW2 encryption for a while,
because it's a flawed design (commit 136cd19 Describe flaws in
qcow/qcow2 encryption in the docs).

In addition to flawed crypto, we have comically bad usability, and
plain old bugs.  Let me show you.

= Example images =

I'm going to use a raw image as backing file, and two QCOW2 images,
one encrypted, and one not:

    $ qemu-img create -f raw backing.img 4m
    Formatting 'backing.img', fmt=raw size=4194304
    $ qemu-img create -f qcow2 -o 
encryption,backing_file=backing.img,backing_fmt=raw geheim.qcow2 4m
    Formatting 'geheim.qcow2', fmt=qcow2 size=4194304 
backing_file='backing.img' backing_fmt='raw' encryption=on cluster_size=65536 
lazy_refcounts=off
    $ qemu-img create -f qcow2 -o backing_file=backing.img,backing_fmt=raw 
normal.qcow2 4m
    Formatting 'normal.qcow2', fmt=qcow2 size=4194304 
backing_file='backing.img' backing_fmt='raw' encryption=off cluster_size=65536 
lazy_refcounts=off

= Usability issues =

== Confusing startup ==

When no image is encrypted, and you don't give -S, QEMU starts the
guest immediately:

    $ qemu-system-x86_64 -nodefaults -display none -monitor stdio normal.qcow2
    QEMU 2.2.50 monitor - type 'help' for more information
    (qemu) info status
    VM status: running

But as soon as there's an encrypted image in play, the guest is *not*
started, with no notification whatsoever:

    $ qemu-system-x86_64 -nodefaults -display none -monitor stdio geheim.qcow2
    QEMU 2.2.50 monitor - type 'help' for more information
    (qemu) info status
    VM status: paused (prelaunch)

If the user figured out that he needs to type "cont" to enter his
keys, the confusion enters the next level: "cont" asks for at most
*one* key.  If more are needed, it then silently does nothing.  The
user has to type "cont" once per encrypted image:

    $ qemu-system-x86_64 -nodefaults -display none -monitor stdio -drive 
if=none,file=geheim.qcow2 -drive if=none,file=geheim.qcow2
    QEMU 2.2.50 monitor - type 'help' for more information
    (qemu) info status
    VM status: paused (prelaunch)
    (qemu) c
    none0 (geheim.qcow2) is encrypted.
    Password: ******
    (qemu) info status
    VM status: paused (prelaunch)
    (qemu) c
    none1 (geheim.qcow2) is encrypted.
    Password: ******
    (qemu) info status
    VM status: running

== Incorrect passwords not caught ==

All existing encryption schemes give you the GIGO treatment: garbage
password in, garbage data out.  Guests usually refuse to mount
garbage, but other usage is prone to data loss.

== Need to stop the guest to add an encrypted image ==

    $ qemu-system-x86_64 -nodefaults -display none -monitor stdio
    QEMU 2.2.50 monitor - type 'help' for more information
    (qemu) info status
    VM status: running
    (qemu) drive_add "" if=none,file=geheim.qcow2
    Guest must be stopped for opening of encrypted image
    (qemu) stop
    (qemu) drive_add "" if=none,file=geheim.qcow2
    OK

Commit c3adb58 added this restriction.  Before, we could expose images
lacking an encryption key to guests, with potentially catastrophic
results.  See also "Use without key is not always caught".

= Bugs =

== Use without key is not always caught ==

Encrypted images can be in an intermediate state "opened, but no key".
The weird startup behavior and the need to stop the guest are there to
ensure the guest isn't exposed to that state.  But other things still
are!

* drive_backup

    $ qemu-system-x86_64 -nodefaults -display none -monitor stdio geheim.qcow2
    QEMU 2.2.50 monitor - type 'help' for more information
    (qemu) drive_backup -f ide0-hd0 out.img raw
    Formatting 'out.img', fmt=raw size=4194304

  I guess this writes encrypted data to raw image out.img.  Good luck
  with figuring out how to decrypt that again.

* commit

    $ qemu-system-x86_64 -nodefaults -display none -monitor stdio geheim.qcow2
    QEMU 2.2.50 monitor - type 'help' for more information
    (qemu) commit ide0-hd0

  I guess this writes encrypted data into the unencrypted raw backing
  image, effectively destroying it.

== QMP device_add of usb-storage fails when it shouldn't ==

When the image is encrypted, device_add creates the device, defers
actually attaching it to when the key becomes available, then fails.
This is wrong.  device_add must either create the device and succeed,
or do nothing and fail.

    $ qemu-system-x86_64 -nodefaults -display none -usb -qmp stdio -drive 
if=none,id=foo,file=geheim.qcow2
    {"QMP": {"version": {"qemu": {"micro": 50, "minor": 2, "major": 2}, 
"package": ""}, "capabilities": []}}
    { "execute": "qmp_capabilities" }
    {"return": {}}
    { "execute": "device_add", "arguments": { "driver": "usb-storage", "id": 
"bar", "drive": "foo" } }
    {"error": {"class": "DeviceEncrypted", "desc": "'foo' (geheim.qcow2) is 
encrypted"}}
    {"execute":"device_del","arguments": { "id": "bar" } }
    {"timestamp": {"seconds": 1426003440, "microseconds": 237181}, "event": 
"DEVICE_DELETED", "data": {"path": "/machine/peripheral/bar/bar.0/legacy[0]"}}
    {"timestamp": {"seconds": 1426003440, "microseconds": 238231}, "event": 
"DEVICE_DELETED", "data": {"device": "bar", "path": "/machine/peripheral/bar"}}
    {"return": {}}

This stuff is worse than useless, it's a trap for users.

If people become sufficiently interested in encrypted images to
contribute a cryptographically sane implementation for QCOW2 (or
whatever other format), then rewriting the necessary support around it
from scratch will likely be easier and yield better results than
fixing up the existing mess.

Let's deprecate the mess now, drop it after a grace period, and move
on.

Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 6ec46ad541b20a1530cedc741d19eea9ffc39ac3
      
https://github.com/qemu/qemu/commit/6ec46ad541b20a1530cedc741d19eea9ffc39ac3
  Author: Markus Armbruster <address@hidden>
  Date:   2015-03-16 (Mon, 16 Mar 2015)

  Changed paths:
    M block/write-threshold.c
    M qapi/block-core.json

  Log Message:
  -----------
  block: Fix block-set-write-threshold not to use funky error class

Error classes are a leftover from the days of "rich" error objects.
New code should always use ERROR_CLASS_GENERIC_ERROR.  Commit e246211
added a use of ERROR_CLASS_DEVICE_NOT_FOUND.  Replace it.

Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 2867ce4ab86c77579e94c6bb2b6e44ddfcf67f5d
      
https://github.com/qemu/qemu/commit/2867ce4ab86c77579e94c6bb2b6e44ddfcf67f5d
  Author: Markus Armbruster <address@hidden>
  Date:   2015-03-16 (Mon, 16 Mar 2015)

  Changed paths:
    M qemu-img.c

  Log Message:
  -----------
  qemu-img: Avoid qerror_report_err() outside QMP handlers, again

qerror_report_err() is a transitional interface to help with
converting existing monitor commands to QMP.  It should not be used
elsewhere.  Replace by error_report_err().

Commit 6936f29 cleaned that up in qemu-img.c, but two calls have crept
in since.  Take care of them the same way.

Signed-off-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 14a58a4e0c2e98a7d9232e1c229a531ca231133b
      
https://github.com/qemu/qemu/commit/14a58a4e0c2e98a7d9232e1c229a531ca231133b
  Author: Max Reitz <address@hidden>
  Date:   2015-03-16 (Mon, 16 Mar 2015)

  Changed paths:
    M block/qcow2-refcount.c

  Log Message:
  -----------
  qcow2: Respect new_block in alloc_refcount_block()

When choosing a new place for the refcount table, alloc_refcount_block()
tries to infer the number of clusters used so far from its argument
cluster_index (which comes from the idea that if any cluster with an
index greater than cluster_index was in use, the refcount table would
have to be big enough already to describe cluster_index).

However, there is a cluster that may be at or after cluster_index, and
which is not covered by the refcount structures, and that is the new
refcount block new_block. Therefore, it should be taken into account for
the blocks_used calculation.

Also, because new_block already describes (or is intended to describe)
cluster_index, we may not put the new refcount structures there.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 0e8a371468ce24513b15a9ae362f12822e1973a3
      
https://github.com/qemu/qemu/commit/0e8a371468ce24513b15a9ae362f12822e1973a3
  Author: Max Reitz <address@hidden>
  Date:   2015-03-16 (Mon, 16 Mar 2015)

  Changed paths:
    A tests/qemu-iotests/121
    A tests/qemu-iotests/121.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  iotests: Add tests for refcount table growth

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 4b4d7b072f0faf9008ca835af101338857b28394
      
https://github.com/qemu/qemu/commit/4b4d7b072f0faf9008ca835af101338857b28394
  Author: Max Reitz <address@hidden>
  Date:   2015-03-16 (Mon, 16 Mar 2015)

  Changed paths:
    A tests/qemu-iotests/115
    A tests/qemu-iotests/115.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  iotests: Test non-self-referential qcow2 refblocks

It is easy to create only self-referential refblocks, but there are
cases where that is impossible. This adds a test for two of those cases
(combined in a single test case).

Suggested-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 5560625badb9e710577986de65ff4e4b413729a0
      
https://github.com/qemu/qemu/commit/5560625badb9e710577986de65ff4e4b413729a0
  Author: Fam Zheng <address@hidden>
  Date:   2015-03-16 (Mon, 16 Mar 2015)

  Changed paths:
    M monitor.c

  Log Message:
  -----------
  monitor: Convert bdrv_find to blk_by_name

Signed-off-by: Fam Zheng <address@hidden>
Message-id: address@hidden
Reviewed-by: Max Reitz <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: c9ebaf744ea7785776c358400eb13c256a3c9db6
      
https://github.com/qemu/qemu/commit/c9ebaf744ea7785776c358400eb13c256a3c9db6
  Author: Fam Zheng <address@hidden>
  Date:   2015-03-16 (Mon, 16 Mar 2015)

  Changed paths:
    M migration/block.c

  Log Message:
  -----------
  migration: Convert bdrv_find to blk_by_name

Signed-off-by: Fam Zheng <address@hidden>
Message-id: address@hidden
Reviewed-by: Max Reitz <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: a0e8544cf84cf193e455a9a3b7d9997dbc268b4b
      
https://github.com/qemu/qemu/commit/a0e8544cf84cf193e455a9a3b7d9997dbc268b4b
  Author: Fam Zheng <address@hidden>
  Date:   2015-03-16 (Mon, 16 Mar 2015)

  Changed paths:
    M blockdev.c

  Log Message:
  -----------
  blockdev: Convert bdrv_find to blk_by_name

Signed-off-by: Fam Zheng <address@hidden>
Message-id: address@hidden
Reviewed-by: Max Reitz <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: d51a2427f68a312b676edd0e9efce881649d1767
      
https://github.com/qemu/qemu/commit/d51a2427f68a312b676edd0e9efce881649d1767
  Author: Fam Zheng <address@hidden>
  Date:   2015-03-16 (Mon, 16 Mar 2015)

  Changed paths:
    M block.c
    M include/block/block.h

  Log Message:
  -----------
  block: Drop bdrv_find

All callers are converted, so drop it.

Signed-off-by: Fam Zheng <address@hidden>
Message-id: address@hidden
Reviewed-by: Max Reitz <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 2ec711dcd45effc8d583dee6ff92d94573aad75b
      
https://github.com/qemu/qemu/commit/2ec711dcd45effc8d583dee6ff92d94573aad75b
  Author: Peter Lieven <address@hidden>
  Date:   2015-03-16 (Mon, 16 Mar 2015)

  Changed paths:
    M block/vpc.c

  Log Message:
  -----------
  block/vpc: optimize vpc_co_get_block_status

*pnum can't be greater than s->block_size / BDRV_SECTOR_SIZE for allocated
sectors since there is always a bitmap in between.

Signed-off-by: Peter Lieven <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 0444dceee48fed54e8334428fa57f9ff997736e8
      
https://github.com/qemu/qemu/commit/0444dceee48fed54e8334428fa57f9ff997736e8
  Author: Kevin Wolf <address@hidden>
  Date:   2015-03-16 (Mon, 16 Mar 2015)

  Changed paths:
    M block/vpc.c

  Log Message:
  -----------
  vpc: Ignore geometry for large images

The CHS calculation as done per the VHD spec imposes a maximum image
size of ~127 GB. Real VHD images exist that are larger than that.

Apparently there are two separate non-standard ways to achieve this:
You could use more heads than the spec does - this is the option that
qemu-img create chooses.

However, other images exist where the geometry is set to the maximum
(65535/16/255), but the actual image size is larger. Until now, such
images are truncated at 127 GB when opening them with qemu.

This patch changes the vpc driver to ignore geometry in this case and
only trust the size field in the header.

Signed-off-by: Kevin Wolf <address@hidden>
[PL: Fixed maximum geometry in the commit msg]
Signed-off-by: Peter Lieven <address@hidden>
Message-id: address@hidden
Reviewed-by: Max Reitz <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 690cbb095a17c429513890d991bc57c98dd83912
      
https://github.com/qemu/qemu/commit/690cbb095a17c429513890d991bc57c98dd83912
  Author: Peter Lieven <address@hidden>
  Date:   2015-03-16 (Mon, 16 Mar 2015)

  Changed paths:
    M block/vpc.c

  Log Message:
  -----------
  block/vpc: make calculate_geometry spec conform

The VHD spec [1] allows for total_sectors of 65535 x 16 x 255 (~127GB)
represented by a CHS geometry. If total_sectors is greater
than 65535 x 16 x 255 this geometry is set as a maximum.

Qemu, Hyper-V and disk2vhd use this special geometry as an indicator
to use the image current size from the footer as disk size.

This patch changes vpc_create to effectively calculate a CxHxS geometry
for the given image size if possible while rounding up if necessary.
If the image size is too big to be represented in CHS we set the maximum
and write the exact requested image size into the footer.

This partly reverts commit 258d2edb, but leaves support for >127G disks
intact.

[1] 
http://download.microsoft.com/download/f/f/e/ffef50a5-07dd-4cf8-aaa3-442c0673a029/Virtual%20Hard%20Disk%20Format%20Spec_10_18_06.doc

Signed-off-by: Peter Lieven <address@hidden>
Message-id: address@hidden
Reviewed-by: Max Reitz <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 03671ded3078ebad1f0a701042622fd5e8918bc9
      
https://github.com/qemu/qemu/commit/03671ded3078ebad1f0a701042622fd5e8918bc9
  Author: Peter Lieven <address@hidden>
  Date:   2015-03-16 (Mon, 16 Mar 2015)

  Changed paths:
    M block/vpc.c

  Log Message:
  -----------
  block/vpc: rename footer->size -> footer->current_size

the field is named current size in the spec. Name it accordingly.

Signed-off-by: Peter Lieven <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 304ee9174f4761d3f4da611352a815ab27baba06
      
https://github.com/qemu/qemu/commit/304ee9174f4761d3f4da611352a815ab27baba06
  Author: Peter Lieven <address@hidden>
  Date:   2015-03-16 (Mon, 16 Mar 2015)

  Changed paths:
    M block/vpc.c

  Log Message:
  -----------
  block/vpc: remove disabled code from get_sector_offset

The code to check the bitmap for the allocation status of each sector
has been "disabled by reason" ever since the vpc driver existed.

The reason might be that we might end up reading sector by sector
in vpc_read if we really used it. This would be a performance desaster.

The current code would furthermore not work if the disabled parts get
reactivated since vpc_read and vpc_write only use get_sector_offset to
check the allocation status of the first sector of a read/write operation.
This might lead to sectors incorrectly treated as zero in vpc_read and
to sectors getting allocated twice in vpc_write.

Signed-off-by: Peter Lieven <address@hidden>
Message-id: address@hidden
Reviewed-by: Max Reitz <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 47aced5078e13d6c1f4fbb59616654e577af8aaa
      
https://github.com/qemu/qemu/commit/47aced5078e13d6c1f4fbb59616654e577af8aaa
  Author: Kevin Wolf <address@hidden>
  Date:   2015-03-16 (Mon, 16 Mar 2015)

  Changed paths:
    M block.c
    M block/qcow2-refcount.c
    M block/vpc.c
    M blockdev.c
    M include/block/block.h
    M migration/block.c
    M monitor.c
    A tests/qemu-iotests/115
    A tests/qemu-iotests/115.out
    A tests/qemu-iotests/121
    A tests/qemu-iotests/121.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  Merge remote-tracking branch 'mreitz/block' into queue-block

* mreitz/block:
  block/vpc: remove disabled code from get_sector_offset
  block/vpc: rename footer->size -> footer->current_size
  block/vpc: make calculate_geometry spec conform
  vpc: Ignore geometry for large images
  block/vpc: optimize vpc_co_get_block_status
  block: Drop bdrv_find
  blockdev: Convert bdrv_find to blk_by_name
  migration: Convert bdrv_find to blk_by_name
  monitor: Convert bdrv_find to blk_by_name
  iotests: Test non-self-referential qcow2 refblocks
  iotests: Add tests for refcount table growth
  qcow2: Respect new_block in alloc_refcount_block()


  Commit: 3521f767067feedd06ef6d2efd24c1190c68638c
      
https://github.com/qemu/qemu/commit/3521f767067feedd06ef6d2efd24c1190c68638c
  Author: Peter Maydell <address@hidden>
  Date:   2015-03-16 (Mon, 16 Mar 2015)

  Changed paths:
    M block.c
    M block/qcow2-refcount.c
    M block/vpc.c
    M block/write-threshold.c
    M blockdev.c
    M include/block/block.h
    M migration/block.c
    M monitor.c
    M qapi/block-core.json
    M qemu-doc.texi
    M qemu-img.c
    M tests/qemu-iotests/049.out
    M tests/qemu-iotests/051.out
    M tests/qemu-iotests/061.out
    M tests/qemu-iotests/087.out
    A tests/qemu-iotests/115
    A tests/qemu-iotests/115.out
    A tests/qemu-iotests/121
    A tests/qemu-iotests/121.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block patches for 2.3-rc0

# gpg: Signature made Mon Mar 16 16:11:55 2015 GMT using RSA key ID C88F2FD6
# gpg: Good signature from "Kevin Wolf <address@hidden>"

* remotes/kevin/tags/for-upstream:
  block/vpc: remove disabled code from get_sector_offset
  block/vpc: rename footer->size -> footer->current_size
  block/vpc: make calculate_geometry spec conform
  vpc: Ignore geometry for large images
  block/vpc: optimize vpc_co_get_block_status
  block: Drop bdrv_find
  blockdev: Convert bdrv_find to blk_by_name
  migration: Convert bdrv_find to blk_by_name
  monitor: Convert bdrv_find to blk_by_name
  iotests: Test non-self-referential qcow2 refblocks
  iotests: Add tests for refcount table growth
  qcow2: Respect new_block in alloc_refcount_block()
  qemu-img: Avoid qerror_report_err() outside QMP handlers, again
  block: Fix block-set-write-threshold not to use funky error class
  block: Deprecate QCOW/QCOW2 encryption
  qemu-img: Fix convert, amend error messages for unknown options
  iotests: Update 051's reference output

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/17b11a1406fd...3521f767067f