[Qemu-commits] [qemu/qemu] d8336c: iotests: Replace time.clock() with Timeout

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: d8336c6b286d9715bfe29a0e0b415dfd7891f671
      
https://github.com/qemu/qemu/commit/d8336c6b286d9715bfe29a0e0b415dfd7891f671
  Author: Kevin Wolf <address@hidden>
  Date:   2018-11-21 (Wed, 21 Nov 2018)

  Changed paths:
    M tests/qemu-iotests/118

  Log Message:
  -----------
  iotests: Replace time.clock() with Timeout

time.clock() is deprecated since Python 3.3. Current Python versions
warn that the function will be removed in Python 3.8, and those warnings
make the test case 118 fail.

Replace it with the Timeout mechanism that is compatible with both
Python 2 and 3, and makes the code even a little nicer.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>


  Commit: fa1cfb40262b8a60e0f93b70491660f242638f81
      
https://github.com/qemu/qemu/commit/fa1cfb40262b8a60e0f93b70491660f242638f81
  Author: Kevin Wolf <address@hidden>
  Date:   2018-11-21 (Wed, 21 Nov 2018)

  Changed paths:
    M tests/qemu-iotests/041
    M tests/qemu-iotests/118
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests: Replace assertEquals() with assertEqual()

TestCase.assertEquals() is deprecated since Python 2.7. Recent Python
versions print a warning when the function is called, which makes test
cases fail.

Replace it with the preferred spelling assertEqual().

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>


  Commit: 155af09d44f584a790118f78448f50f140d0f788
      
https://github.com/qemu/qemu/commit/155af09d44f584a790118f78448f50f140d0f788
  Author: Eric Blake <address@hidden>
  Date:   2018-11-21 (Wed, 21 Nov 2018)

  Changed paths:
    M tests/qemu-iotests/common.tls

  Log Message:
  -----------
  iotests: Skip 233 if certtool not installed

The use of TLS while building qemu is optional. While the
'certtool' binary should be available on every platform that
supports building against TLS, that does not imply that the
developer has installed it.  Make the test gracefully skip
in that case.

Reported-by: Kevin Wolf <address@hidden>
Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Daniel P. Berrangé <address@hidden>
Reviewed-by: Wainer dos Santos Moschetta <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: f0998879e049dad19beed881a1c56643ce536384
      
https://github.com/qemu/qemu/commit/f0998879e049dad19beed881a1c56643ce536384
  Author: Max Reitz <address@hidden>
  Date:   2018-11-21 (Wed, 21 Nov 2018)

  Changed paths:
    M qemu-img.c

  Log Message:
  -----------
  qemu-img: Fix typo

Fixes: d402b6a21a825a5c07aac9251990860723d49f5d
Reported-by: Kevin Wolf <address@hidden>
Cc: address@hidden
Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 3ecd5a4f19fd9a497490a91aaa96e76a5edadd2c
      
https://github.com/qemu/qemu/commit/3ecd5a4f19fd9a497490a91aaa96e76a5edadd2c
  Author: Max Reitz <address@hidden>
  Date:   2018-11-21 (Wed, 21 Nov 2018)

  Changed paths:
    M qemu-img.c

  Log Message:
  -----------
  qemu-img: Fix leak

create_opts was leaked here.  This is not too bad since the process is
about to exit anyway, but relying on that does not make the code nicer
to read.

Fixes: d402b6a21a825a5c07aac9251990860723d49f5d
Reported-by: Kevin Wolf <address@hidden>
Cc: address@hidden
Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 1c7f618f689b0b5b6bbed23a7c159e7dad7b996f
      
https://github.com/qemu/qemu/commit/1c7f618f689b0b5b6bbed23a7c159e7dad7b996f
  Author: Richard W.M. Jones <address@hidden>
  Date:   2018-11-22 (Thu, 22 Nov 2018)

  Changed paths:
    M hw/scsi/scsi-disk.c

  Log Message:
  -----------
  scsi-disk: Fix crash if underlying host file or disk returns error

Commit 40dce4ee6 "scsi-disk: fix rerror/werror=ignore" introduced a
bug which causes qemu to crash with the assertion error below if the
host file or disk returns an error:

  qemu-system-x86_64: hw/scsi/scsi-bus.c:1374: scsi_req_complete:
  Assertion `req->status == -1' failed.

Kevin Wolf suggested this fix:

  < kwolf> Hm, should the final return false; in that patch
     actually be a return true?
  < kwolf> Because I think he didn't intend to change anything
     except BLOCK_ERROR_ACTION_IGNORE

Buglink: https://bugs.launchpad.net/qemu/+bug/1804323
Fixes: 40dce4ee61c68395f6d463fae792f61b7c003bce
Signed-off-by: Richard W.M. Jones <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 2a3d4331fa2d40708188b8000f98ff1f7dcd33bc
      
https://github.com/qemu/qemu/commit/2a3d4331fa2d40708188b8000f98ff1f7dcd33bc
  Author: Alberto Garcia <address@hidden>
  Date:   2018-11-22 (Thu, 22 Nov 2018)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: Fix update of BDRV_O_AUTO_RDONLY in update_flags_from_options()

Commit e35bdc123a4ace9f4d3fcca added the auto-read-only option and the
code to update its corresponding flag in update_flags_from_options(),
but forgot to clear the flag if auto-read-only is false.

Signed-off-by: Alberto Garcia <address@hidden>
Reported-by: Max Reitz <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: e4c8f2925d22584b2008aadea5c70e1e05c2a522
      
https://github.com/qemu/qemu/commit/e4c8f2925d22584b2008aadea5c70e1e05c2a522
  Author: Daniel P. Berrangé <address@hidden>
  Date:   2018-11-22 (Thu, 22 Nov 2018)

  Changed paths:
    M tests/qemu-iotests/233

  Log Message:
  -----------
  iotests: fix nbd test 233 to work correctly with raw images

The first qemu-io command must honour the $IMGFMT that is set rather
than hardcoding qcow2. The qemu-nbd commands should also set $IMGFMT
to avoid the insecure format probe warning.

Signed-off-by: Daniel P. Berrangé <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 6bf7463615752934d7221e5be9820d9da45ab2de
      
https://github.com/qemu/qemu/commit/6bf7463615752934d7221e5be9820d9da45ab2de
  Author: Igor Druzhinin <address@hidden>
  Date:   2018-11-22 (Thu, 22 Nov 2018)

  Changed paths:
    M hw/block/nvme.c

  Log Message:
  -----------
  nvme: call blk_drain in NVMe reset code to avoid lockups

When blk_flush called in NVMe reset path S/C queues are already freed
which means that re-entering AIO handling loop having some IO requests
unfinished will lockup or crash as their SG structures being potentially
reused. Call blk_drain before freeing the queues to avoid this nasty
scenario.

Signed-off-by: Igor Druzhinin <address@hidden>
Acked-by: Paolo Bonzini <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 87ad860c622cc8f8916b5232bd8728c08f938fce
      
https://github.com/qemu/qemu/commit/87ad860c622cc8f8916b5232bd8728c08f938fce
  Author: Paolo Bonzini <address@hidden>
  Date:   2018-11-22 (Thu, 22 Nov 2018)

  Changed paths:
    M hw/block/nvme.c
    M tests/Makefile.include
    M tests/nvme-test.c

  Log Message:
  -----------
  nvme: fix out-of-bounds access to the CMB

Because the CMB BAR has a min_access_size of 2, if you read the last
byte it will try to memcpy *2* bytes from n->cmbuf, causing an off-by-one
error.  This is CVE-2018-16847.

Another way to fix this might be to register the CMB as a RAM memory
region, which would also be more efficient.  However, that might be a
change for big-endian machines; I didn't think this through and I don't
know how real hardware works.  Add a basic testcase for the CMB in case
somebody does this change later on.

Cc: Keith Busch <address@hidden>
Cc: address@hidden
Reported-by: Li Qiang <address@hidden>
Reviewed-by: Li Qiang <address@hidden>
Tested-by: Li Qiang <address@hidden>
Signed-off-by: Paolo Bonzini <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Tested-by: Philippe Mathieu-Daudé <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 2067d39e5e53b053bece6fa15711640123c119ed
      
https://github.com/qemu/qemu/commit/2067d39e5e53b053bece6fa15711640123c119ed
  Author: Kevin Wolf <address@hidden>
  Date:   2018-11-22 (Thu, 22 Nov 2018)

  Changed paths:
    M hw/block/nvme.c

  Log Message:
  -----------
  Revert "nvme: fix oob access issue(CVE-2018-16847)"

This reverts commit 5e3c0220d7e4f0361c4d36c697a8842f2b583402.
We have a better fix commited for this now.

Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 71a86ddece548860f040d565204cf1bf59d74663
      
https://github.com/qemu/qemu/commit/71a86ddece548860f040d565204cf1bf59d74663
  Author: Paolo Bonzini <address@hidden>
  Date:   2018-11-22 (Thu, 22 Nov 2018)

  Changed paths:
    M hw/block/nvme.c

  Log Message:
  -----------
  nvme: fix CMB endianness confusion

The CMB is marked as DEVICE_LITTLE_ENDIAN, so the data must be
read/written as if it was little-endian output (in the case of
big endian, we get two swaps, one in the memory core and one
in nvme.c).

Signed-off-by: Paolo Bonzini <address@hidden>
Tested-by: Peter Maydell <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: ad3a7e4555bc50036a5257a6c1ed652ab0d1b650
      
https://github.com/qemu/qemu/commit/ad3a7e4555bc50036a5257a6c1ed652ab0d1b650
  Author: Logan Gunthorpe <address@hidden>
  Date:   2018-11-22 (Thu, 22 Nov 2018)

  Changed paths:
    M hw/block/nvme.c

  Log Message:
  -----------
  nvme: fix bug with PCI IRQ pins on teardown

When the submission and completion queues are being torn down
the IRQ will be asserted for the completion queue when the
submsission queue is deleted. Then when the completion queue
is deleted it stays asserted. Thus, on systems that do
not use MSI, no further interrupts can be triggered on the host.

Linux sees this as a long delay when unbinding the nvme device.
Eventually the interrupt timeout occurs and it continues.

To fix this we ensure we deassert the IRQ for a CQ when it is
deleted.

Signed-off-by: Logan Gunthorpe <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: a237dea330a2be9a2cbe95056b9a8d67627d95c6
      
https://github.com/qemu/qemu/commit/a237dea330a2be9a2cbe95056b9a8d67627d95c6
  Author: Eric Blake <address@hidden>
  Date:   2018-11-22 (Thu, 22 Nov 2018)

  Changed paths:
    M tests/qemu-iotests/223
    M tests/qemu-iotests/223.out

  Log Message:
  -----------
  iotests: Enhance 223 to cover multiple bitmap granularities

Testing granularity at the same size as the cluster isn't quite
as fun as what happens when it is larger or smaller.  This
enhancement also shows that qemu's nbd server can serve the
same disk over multiple exports simultaneously.

Signed-off-by: Eric Blake <address@hidden>
Tested-by: John Snow <address@hidden>
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 0065c455f9f03ef5e830ede804a7404a8892fbc7
      
https://github.com/qemu/qemu/commit/0065c455f9f03ef5e830ede804a7404a8892fbc7
  Author: Alberto Garcia <address@hidden>
  Date:   2018-11-22 (Thu, 22 Nov 2018)

  Changed paths:
    M block.c
    A tests/qemu-iotests/161
    A tests/qemu-iotests/161.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  block: Update BlockDriverState.inherits_from on bdrv_set_backing_hd()

When a BlockDriverState's child is opened (be it a backing file, the
protocol layer, or any other) inherits_from is set to point to the
parent node. Children opened separately and then attached to a parent
don't have this pointer set.

bdrv_reopen_queue_child() uses this to determine whether a node's
children must also be reopened inheriting the options from the parent
or not. If inherits_from points to the parent then the child is
reopened and its options can be changed, like in this example:

   $ qemu-img create -f qcow2 hd0.qcow2 1M
   $ qemu-img create -f qcow2 hd1.qcow2 1M
   $ $QEMU -drive if=none,node-name=hd0,file=hd0.qcow2,\
            backing.driver=qcow2,backing.file.filename=hd1.qcow2
   (qemu) qemu-io hd0 "reopen -o backing.l2-cache-size=2M"

If the child does not inherit from the parent then it does not get
reopened and its options cannot be changed:

   $ $QEMU -drive if=none,node-name=hd1,file=hd1.qcow2
     -drive if=none,node-name=hd0,file=hd0.qcow2,backing=hd1
   (qemu) qemu-io hd0 "reopen -o backing.l2-cache-size=2M"
   Cannot change the option 'backing.l2-cache-size'

If a disk image has a chain of backing files then all of them are also
connected through their inherits_from pointers (i.e. it's possible to
walk the chain in reverse order from base to top).

However this is broken if the intermediate nodes are removed using
e.g. block-stream because the inherits_from pointer from the base node
becomes NULL:

   $ qemu-img create -f qcow2 hd0.qcow2 1M
   $ qemu-img create -f qcow2 -b hd0.qcow2 hd1.qcow2
   $ qemu-img create -f qcow2 -b hd1.qcow2 hd2.qcow2
   $ $QEMU -drive if=none,file=hd2.qcow2
   (qemu) qemu-io none0 "reopen -o backing.l2-cache-size=2M"
   (qemu) block_stream none0 0 hd0.qcow2
   (qemu) qemu-io none0 "reopen -o backing.l2-cache-size=2M"
   Cannot change the option 'backing.l2-cache-size'

This patch updates the inherits_from pointer if the intermediate nodes
of a backing chain are removed using bdrv_set_backing_hd(), and adds a
test case for this scenario.

Signed-off-by: Alberto Garcia <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 6bd858b3117a5aab066f3cf02ca72000eaa10ddb
      
https://github.com/qemu/qemu/commit/6bd858b3117a5aab066f3cf02ca72000eaa10ddb
  Author: Alberto Garcia <address@hidden>
  Date:   2018-11-22 (Thu, 22 Nov 2018)

  Changed paths:
    M block.c
    M tests/qemu-iotests/161
    M tests/qemu-iotests/161.out

  Log Message:
  -----------
  block: Update BlockDriverState.inherits_from on bdrv_drop_intermediate()

The previous patch fixed the inherits_from pointer after block-stream,
and this one does the same for block-commit.

When block-commit finishes and the 'top' node is not the topmost one
from the backing chain then all nodes above 'base' up to and including
'top' are removed from the chain.

The bdrv_drop_intermediate() call converts a chain like this one:

    base <- intermediate <- top <- active

into this one:

    base <- active

In a simple scenario each backing file from the first chain has the
inherits_from attribute pointing to its parent. This means that
reopening 'active' will recursively reopen all its children, whose
options can be changed in the process.

However after the 'block-commit' call base.inherits_from is NULL and
the chain is broken, so 'base' does not inherit from 'active' and will
not be reopened automatically:

   $ qemu-img create -f qcow2 hd0.qcow2 1M
   $ qemu-img create -f qcow2 -b hd0.qcow2 hd1.qcow2
   $ qemu-img create -f qcow2 -b hd1.qcow2 hd2.qcow2
   $ $QEMU -drive if=none,file=hd2.qcow2

   { 'execute': 'block-commit',
     'arguments': {
       'device': 'none0',
       'top': 'hd1.qcow2' } }

   { 'execute': 'human-monitor-command',
     'arguments': {
  'command-line':
    'qemu-io none0 "reopen -o backing.l2-cache-size=2M"' } }

   { "return": "Cannot change the option 'backing.l2-cache-size'\r\n"}

This patch updates base.inherits_from in this scenario, and adds a
test case.

Signed-off-by: Alberto Garcia <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 5298f4d67a911dd9cefa4c4185eed242074d64c2
      
https://github.com/qemu/qemu/commit/5298f4d67a911dd9cefa4c4185eed242074d64c2
  Author: Peter Maydell <address@hidden>
  Date:   2018-11-23 (Fri, 23 Nov 2018)

  Changed paths:
    M block.c
    M hw/block/nvme.c
    M hw/scsi/scsi-disk.c
    M qemu-img.c
    M tests/Makefile.include
    M tests/nvme-test.c
    M tests/qemu-iotests/041
    M tests/qemu-iotests/118
    A tests/qemu-iotests/161
    A tests/qemu-iotests/161.out
    M tests/qemu-iotests/223
    M tests/qemu-iotests/223.out
    M tests/qemu-iotests/233
    M tests/qemu-iotests/common.tls
    M tests/qemu-iotests/group
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- block: Fix update of BDRV_O_AUTO_RDONLY in update_flags_from_options()
- block: Fix option inheritance after stream/commit job graph changes
- qemu-img: Fix memory leak and typo in error message
- nvme: Fixes for lockups and crashes
- scsi-disk: Fix crash if underlying host file or disk returns error
- Several qemu-iotests fixes and improvements

# gpg: Signature made Thu 22 Nov 2018 18:38:30 GMT
# gpg:                using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <address@hidden>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
  block: Update BlockDriverState.inherits_from on bdrv_drop_intermediate()
  block: Update BlockDriverState.inherits_from on bdrv_set_backing_hd()
  iotests: Enhance 223 to cover multiple bitmap granularities
  nvme: fix bug with PCI IRQ pins on teardown
  nvme: fix CMB endianness confusion
  Revert "nvme: fix oob access issue(CVE-2018-16847)"
  nvme: fix out-of-bounds access to the CMB
  nvme: call blk_drain in NVMe reset code to avoid lockups
  iotests: fix nbd test 233 to work correctly with raw images
  block: Fix update of BDRV_O_AUTO_RDONLY in update_flags_from_options()
  scsi-disk: Fix crash if underlying host file or disk returns error
  qemu-img: Fix leak
  qemu-img: Fix typo
  iotests: Skip 233 if certtool not installed
  iotests: Replace assertEquals() with assertEqual()
  iotests: Replace time.clock() with Timeout

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/ebfd6216ec14...5298f4d67a91
      **NOTE:** This service has been marked for deprecation: 
https://developer.github.com/changes/2018-04-25-github-services-deprecation/

      Functionality will be removed from GitHub.com on January 31st, 2019.