[Qemu-commits] [qemu/qemu] df98d7: docs: clarify multiqueue vs multiple virtqueues

[Peter Maydell]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: df98d7ccc2e9e3e5080cce30a6d9c09dd827dc15
      
https://github.com/qemu/qemu/commit/df98d7ccc2e9e3e5080cce30a6d9c09dd827dc15
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2019-07-25 (Thu, 25 Jul 2019)

  Changed paths:
    M docs/interop/vhost-user.rst

  Log Message:
  -----------
  docs: clarify multiqueue vs multiple virtqueues

The vhost-user specification does not explain when
VHOST_USER_PROTOCOL_F_MQ must be implemented.  This may lead
implementors of vhost-user masters to believe that this protocol feature
is required for any device that has multiple virtqueues.  That would be
a mistake since existing vhost-user slaves offer multiple virtqueues but
do not advertise VHOST_USER_PROTOCOL_F_MQ.

For example, a vhost-net device with one rx/tx queue pair is not
multiqueue.  The slave does not need to advertise
VHOST_USER_PROTOCOL_F_MQ.  Therefore the master must assume it has these
virtqueues and cannot rely on askingt the slave how many virtqueues
exist.

Extend the specification to explain the different between true
multiqueue and regular devices with a fixed virtqueue layout.

Signed-off-by: Stefan Hajnoczi <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Michael S. Tsirkin <address@hidden>
Signed-off-by: Michael S. Tsirkin <address@hidden>
Reviewed-by: Marc-André Lureau <address@hidden>


  Commit: 21e2acd583126db94f6d881005cd58e835160582
      
https://github.com/qemu/qemu/commit/21e2acd583126db94f6d881005cd58e835160582
  Author: Evgeny Yakovlev <address@hidden>
  Date:   2019-07-25 (Thu, 25 Jul 2019)

  Changed paths:
    M hw/i386/acpi-build.c

  Log Message:
  -----------
  i386/acpi: fix gint overflow in crs_range_compare

When very large regions (32GB sized in our case, PCI pass-through of GPUs)
are compared substraction result does not fit into gint.

As a result crs_replace_with_free_ranges does not get sorted ranges and
incorrectly computes PCI64 free space regions. Which then makes linux
guest complain about device and PCI64 hole intersection and device
becomes unusable.

Fix that by returning exactly fitting ranges.

Also fix indentation of an entire crs_replace_with_free_ranges to make
checkpatch happy.

Cc: address@hidden
Signed-off-by: Evgeny Yakovlev <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Evgeny Yakovlev <address@hidden>


  Commit: be1927c97e564346cbd409cb17fe611df74b84e5
      
https://github.com/qemu/qemu/commit/be1927c97e564346cbd409cb17fe611df74b84e5
  Author: Jan Kiszka <address@hidden>
  Date:   2019-07-25 (Thu, 25 Jul 2019)

  Changed paths:
    M hw/intc/ioapic.c

  Log Message:
  -----------
  ioapic: kvm: Skip route updates for masked pins

Masked entries will not generate interrupt messages, thus do no need to
be routed by KVM. This is a cosmetic cleanup, just avoiding warnings of
the kind

qemu-system-x86_64: vtd_irte_get: detected non-present IRTE (index=0, 
high=0xff00, low=0x100)

if the masked entry happens to reference a non-present IRTE.

Cc: address@hidden
Signed-off-by: Jan Kiszka <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Michael S. Tsirkin <address@hidden>
Signed-off-by: Michael S. Tsirkin <address@hidden>
Reviewed-by: Peter Xu <address@hidden>


  Commit: ee4b0c8686f781987879508d7c6dd605b5435bac
      
https://github.com/qemu/qemu/commit/ee4b0c8686f781987879508d7c6dd605b5435bac
  Author: Evgeny Yakovlev <address@hidden>
  Date:   2019-07-25 (Thu, 25 Jul 2019)

  Changed paths:
    M hw/i386/acpi-build.c

  Log Message:
  -----------
  i386/acpi: show PCI Express bus on pxb-pcie expanders

Show PCIe host bridge PNP id with PCI host bridge as a compatible id
when expanding a pcie bus.

Cc: address@hidden
Signed-off-by: Evgeny Yakovlev <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Michael S. Tsirkin <address@hidden>
Signed-off-by: Michael S. Tsirkin <address@hidden>


  Commit: ffa207d08253ffffb3993a1dbe09e40af4fc91f1
      
https://github.com/qemu/qemu/commit/ffa207d08253ffffb3993a1dbe09e40af4fc91f1
  Author: David Hildenbrand <address@hidden>
  Date:   2019-07-25 (Thu, 25 Jul 2019)

  Changed paths:
    M hw/virtio/virtio-balloon.c

  Log Message:
  -----------
  virtio-balloon: Fix wrong sign extension of PFNs

If we directly cast from int to uint64_t, we will first sign-extend to
an int64_t, which is wrong. We actually want to treat the PFNs like
unsigned values.

As far as I can see, this dates back to the initial virtio-balloon
commit, but wasn't triggered as fairly big guests would be required.

Cc: address@hidden
Reported-by: Michael S. Tsirkin <address@hidden>
Signed-off-by: David Hildenbrand <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Michael S. Tsirkin <address@hidden>
Signed-off-by: Michael S. Tsirkin <address@hidden>
Reviewed-by: David Gibson <address@hidden>


  Commit: 483f13524bb2a08b7ff6a7560b846564ed3b0c33
      
https://github.com/qemu/qemu/commit/483f13524bb2a08b7ff6a7560b846564ed3b0c33
  Author: David Hildenbrand <address@hidden>
  Date:   2019-07-25 (Thu, 25 Jul 2019)

  Changed paths:
    M hw/virtio/virtio-balloon.c

  Log Message:
  -----------
  virtio-balloon: Fix QEMU crashes on pagesize > BALLOON_PAGE_SIZE

We are using the wrong functions to set/clear bits, effectively touching
multiple bits, writing out of range of the bitmap, resulting in memory
corruptions. We have to use set_bit()/clear_bit() instead.

Can easily be reproduced by starting a qemu guest on hugetlbfs memory,
inflating the balloon. QEMU crashes. This never could have worked
properly - especially, also pages would have been discarded when the
first sub-page would be inflated (the whole bitmap would be set).

While testing I realized, that on hugetlbfs it is pretty much impossible
to discard a page - the guest just frees the 4k sub-pages in random order
most of the time. I was only able to discard a hugepage a handful of
times - so I hope that now works correctly.

Fixes: ed48c59875b6 ("virtio-balloon: Safely handle BALLOON_PAGE_SIZE < host 
page size")
Fixes: b27b32391404 ("virtio-balloon: Fix possible guest memory corruption with 
inflates & deflates")
Cc: address@hidden #v4.0.0
Acked-by: David Gibson <address@hidden>
Signed-off-by: David Hildenbrand <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Michael S. Tsirkin <address@hidden>
Signed-off-by: Michael S. Tsirkin <address@hidden>


  Commit: 2ffc49eea1bbd454913a88a0ad872c2649b36950
      
https://github.com/qemu/qemu/commit/2ffc49eea1bbd454913a88a0ad872c2649b36950
  Author: David Hildenbrand <address@hidden>
  Date:   2019-07-25 (Thu, 25 Jul 2019)

  Changed paths:
    M hw/virtio/virtio-balloon.c

  Log Message:
  -----------
  virtio-balloon: Simplify deflate with pbp

Let's simplify this - the case we are optimizing for is very hard to
trigger and not worth the effort. If we're switching from inflation to
deflation, let's reset the pbp.

Acked-by: David Gibson <address@hidden>
Signed-off-by: David Hildenbrand <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Michael S. Tsirkin <address@hidden>
Signed-off-by: Michael S. Tsirkin <address@hidden>


  Commit: e6129b271b9dccca22c84870e313c315f2c70063
      
https://github.com/qemu/qemu/commit/e6129b271b9dccca22c84870e313c315f2c70063
  Author: David Hildenbrand <address@hidden>
  Date:   2019-07-25 (Thu, 25 Jul 2019)

  Changed paths:
    M hw/virtio/virtio-balloon.c

  Log Message:
  -----------
  virtio-balloon: Better names for offset variables in inflate/deflate code

"host_page_base" is really confusing, let's make this clearer, also
rename the other offsets to indicate to which base they apply.

offset -> mr_offset
ram_offset -> rb_offset
host_page_base -> rb_aligned_offset

While at it, use QEMU_ALIGN_DOWN() instead of a handcrafted computation
and move the computation to the place where it is needed.

Acked-by: David Gibson <address@hidden>
Signed-off-by: David Hildenbrand <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Michael S. Tsirkin <address@hidden>
Signed-off-by: Michael S. Tsirkin <address@hidden>


  Commit: 1c5cfc2b7153dd72bf4b8ddc456408eb2b9b66d8
      
https://github.com/qemu/qemu/commit/1c5cfc2b7153dd72bf4b8ddc456408eb2b9b66d8
  Author: David Hildenbrand <address@hidden>
  Date:   2019-07-25 (Thu, 25 Jul 2019)

  Changed paths:
    M hw/virtio/virtio-balloon.c

  Log Message:
  -----------
  virtio-balloon: Rework pbp tracking data

Using the address of a RAMBlock to test for a matching pbp is not really
safe. Instead, let's use the guest physical address of the base page
along with the page size (via the number of subpages).

Also, let's allocate the bitmap separately. This makes the code
easier to read and maintain - we can reuse bitmap_new().

Prepare the code to move the PBP out of the device.

Fixes: ed48c59875b6 ("virtio-balloon: Safely handle BALLOON_PAGE_SIZE < host 
page size")
Fixes: b27b32391404 ("virtio-balloon: Fix possible guest memory corruption with 
inflates & deflates")
Cc: address@hidden #v4.0.0
Signed-off-by: David Hildenbrand <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Michael S. Tsirkin <address@hidden>
Signed-off-by: Michael S. Tsirkin <address@hidden>


  Commit: a8cd64d488325f3be5c4ddec4bf07efb3b8c7330
      
https://github.com/qemu/qemu/commit/a8cd64d488325f3be5c4ddec4bf07efb3b8c7330
  Author: David Hildenbrand <address@hidden>
  Date:   2019-07-25 (Thu, 25 Jul 2019)

  Changed paths:
    M hw/virtio/virtio-balloon.c
    M include/hw/virtio/virtio-balloon.h

  Log Message:
  -----------
  virtio-balloon: Use temporary PBP only

We still have multiple issues in the current code
- The PBP is not freed during unrealize()
- The PBP is not reset on device resets: After a reset, the PBP is stale.
- We are not indicating VIRTIO_BALLOON_F_MUST_TELL_HOST, therefore
  guests (esp. legacy guests) will reuse pages without deflating,
  turning the PBP stale. Adding that would require compat handling.

Instead, let's use the PBP only temporarily, when processing one bulk of
inflation requests. This will keep guest_page_size > 4k working (with
Linux guests). There is nothing to do for deflation requests anymore.
The pbp is only used for a limited amount of time.

Fixes: ed48c59875b6 ("virtio-balloon: Safely handle BALLOON_PAGE_SIZE < host 
page size")
Cc: address@hidden #v4.0.0
Suggested-by: Michael S. Tsirkin <address@hidden>
Signed-off-by: David Hildenbrand <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Michael S. Tsirkin <address@hidden>
Signed-off-by: Michael S. Tsirkin <address@hidden>
Acked-by: David Gibson <address@hidden>


  Commit: 9a7ca8a7c920360db9dcaf616ca6f1440c025043
      
https://github.com/qemu/qemu/commit/9a7ca8a7c920360db9dcaf616ca6f1440c025043
  Author: David Hildenbrand <address@hidden>
  Date:   2019-07-25 (Thu, 25 Jul 2019)

  Changed paths:
    M hw/virtio/virtio-balloon.c

  Log Message:
  -----------
  virtio-balloon: don't track subpages for the PBP

As ramblocks cannot get removed/readded while we are processing a bulk
of inflation requests, there is no more need to track the page size
in form of the number of subpages.

Suggested-by: David Gibson <address@hidden>
Signed-off-by: David Hildenbrand <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Michael S. Tsirkin <address@hidden>
Signed-off-by: Michael S. Tsirkin <address@hidden>


  Commit: 1b47b37c33ec01ae1efc527f4c97f97f93723bc4
      
https://github.com/qemu/qemu/commit/1b47b37c33ec01ae1efc527f4c97f97f93723bc4
  Author: Michael S. Tsirkin <address@hidden>
  Date:   2019-07-25 (Thu, 25 Jul 2019)

  Changed paths:
    M hw/virtio/virtio-balloon.c

  Log Message:
  -----------
  virtio-balloon: free pbp more aggressively

Previous patches switched to a temporary pbp but that does not go far
enough: after device uses a buffer, guest is free to reuse it, so
tracking the page and freeing it later is wrong.

Free and reset the pbp after we push each element.

Fixes: ed48c59875b6 ("virtio-balloon: Safely handle BALLOON_PAGE_SIZE < host 
page size")
Cc: address@hidden #v4.0.0
Cc: David Hildenbrand <address@hidden>
Signed-off-by: Michael S. Tsirkin <address@hidden>


  Commit: 61c1e6a97deac9d5f9748ac1fa4e31e84e19f55c
      
https://github.com/qemu/qemu/commit/61c1e6a97deac9d5f9748ac1fa4e31e84e19f55c
  Author: Peter Maydell <address@hidden>
  Date:   2019-07-25 (Thu, 25 Jul 2019)

  Changed paths:
    M docs/interop/vhost-user.rst
    M hw/i386/acpi-build.c
    M hw/intc/ioapic.c
    M hw/virtio/virtio-balloon.c
    M include/hw/virtio/virtio-balloon.h

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging

virtio, pc: fixes, cleanups

A bunch of fixes all over the place.

Signed-off-by: Michael S. Tsirkin <address@hidden>

# gpg: Signature made Thu 25 Jul 2019 16:19:33 BST
# gpg:                using RSA key 281F0DB8D28D5469
# gpg: Good signature from "Michael S. Tsirkin <address@hidden>" [full]
# gpg:                 aka "Michael S. Tsirkin <address@hidden>" [full]
# Primary key fingerprint: 0270 606B 6F3C DF3D 0B17  0970 C350 3912 AFBE 8E67
#      Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA  8A0D 281F 0DB8 D28D 5469

* remotes/mst/tags/for_upstream:
  virtio-balloon: free pbp more aggressively
  virtio-balloon: don't track subpages for the PBP
  virtio-balloon: Use temporary PBP only
  virtio-balloon: Rework pbp tracking data
  virtio-balloon: Better names for offset variables in inflate/deflate code
  virtio-balloon: Simplify deflate with pbp
  virtio-balloon: Fix QEMU crashes on pagesize > BALLOON_PAGE_SIZE
  virtio-balloon: Fix wrong sign extension of PFNs
  i386/acpi: show PCI Express bus on pxb-pcie expanders
  ioapic: kvm: Skip route updates for masked pins
  i386/acpi: fix gint overflow in crs_range_compare
  docs: clarify multiqueue vs multiple virtqueues

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/b43bea01b853...61c1e6a97dea