[Qemu-commits] [qemu/qemu] 452589: vl.c/exit: pause cpus before closing block devices

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 452589b6b47e8dc6353df257fc803dfc1383bed8
      
https://github.com/qemu/qemu/commit/452589b6b47e8dc6353df257fc803dfc1383bed8
  Author: Dr. David Alan Gilbert <address@hidden>
  Date:   2017-08-01 (Tue, 01 Aug 2017)

  Changed paths:
    M vl.c

  Log Message:
  -----------
  vl.c/exit: pause cpus before closing block devices

There's a rare exit seg if the guest is accessing
IO during exit.
It's always hitting the atomic_inc(&bs->in_flight) with a NULL
bs. This was added recently in 99723548  but I don't see it
as the cause.

Flip vl.c around so we pause the cpus before closing the block devices,
that way we shouldn't have anything trying to access them when
they're gone.

This was originally Red Hat bz 
https://bugzilla.redhat.com/show_bug.cgi?id=1451015

Signed-off-by: Dr. David Alan Gilbert <address@hidden>
Reported-by: Cong Li <address@hidden>

--
This is a very rare race, I'll leave it running in a loop to see if
we hit anything else and to check this really fixes it.

I do worry if there are other cases that can trigger this - e.g.
hot-unplug or ejecting a CD.

Message-Id: <address@hidden>
Signed-off-by: Paolo Bonzini <address@hidden>


  Commit: f70d3451fe468eacddb15ccf5fd170754510b0a0
      
https://github.com/qemu/qemu/commit/f70d3451fe468eacddb15ccf5fd170754510b0a0
  Author: Dr. David Alan Gilbert <address@hidden>
  Date:   2017-08-01 (Tue, 01 Aug 2017)

  Changed paths:
    M include/exec/ram_addr.h

  Log Message:
  -----------
  cpu_physical_memory_sync_dirty_bitmap: Fix alignment check

This code has an optimised, word aligned version, and a boring
unaligned version.  Recently 084140bd498909 fixed a missing offset
addition from the core of both versions.  However, the offset isn't
necessarily aligned and thus the choice between the two versions
needs fixing up to also include the offset.

Symptom:
  A few stuck unsent pages during migration; not normally noticed
unless under very low bandwidth in which case the migration may get
stuck never ending and never performing a 2nd sync; noticed by
a hanging postcopy-test on a very heavily loaded system.

Fixes: 084140bd498909

Signed-off-by: Dr. David Alan Gilbert <address@hidden>
Reported-by: Alex Benneé <address@hidden>
Tested-by: Alex Benneé <address@hidden>

--
v2
  Move 'page' inside the if (Comment from Paolo)
Message-Id: <address@hidden>
Signed-off-by: Paolo Bonzini <address@hidden>


  Commit: d73f0247228830a7730bb60af467e8d47aee78cf
      
https://github.com/qemu/qemu/commit/d73f0247228830a7730bb60af467e8d47aee78cf
  Author: Laurent Vivier <address@hidden>
  Date:   2017-08-01 (Tue, 01 Aug 2017)

  Changed paths:
    M accel/accel.c

  Log Message:
  -----------
  accel: cleanup error output

Only emit "XXX accelerator not found", if there are not
further accelerators listed. eg

   accel=kvm:tcg

doesn't print a "KVM accelerator not found" warning
when it falls back to tcg, but a

   accel=kvm

prints a warning, since no fallback is given.

Suggested-by: Daniel P. Berrange <address@hidden>
Suggested-by: Paolo Bonzini <address@hidden>
Signed-off-by: Laurent Vivier <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Paolo Bonzini <address@hidden>


  Commit: 4db0db1fa6b653970148f6ff7a24bede1d52ef9d
      
https://github.com/qemu/qemu/commit/4db0db1fa6b653970148f6ff7a24bede1d52ef9d
  Author: Marc-André Lureau <address@hidden>
  Date:   2017-08-01 (Tue, 01 Aug 2017)

  Changed paths:
    M chardev/char-fd.c
    M include/chardev/char-fd.h

  Log Message:
  -----------
  char-fd: remove useless chr pointer

Apparently unused since it was introduced in commit
a29753f8aa79a34a324afebe340182a51a5aef11. Now, it can be trivially
accessed by CHARDEV() of self.

Signed-off-by: Marc-André Lureau <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Paolo Bonzini <address@hidden>


  Commit: 0ec846bface0f9733ca61ba18e3d4b72bfd9f8ca
      
https://github.com/qemu/qemu/commit/0ec846bface0f9733ca61ba18e3d4b72bfd9f8ca
  Author: Anton Nefedov <address@hidden>
  Date:   2017-08-01 (Tue, 01 Aug 2017)

  Changed paths:
    M chardev/char.c
    M include/chardev/char.h
    M vl.c

  Log Message:
  -----------
  char: don't exit on hmp 'chardev-add help'

qemu_chr_new_from_opts() is used from both vl.c and hmp,
and it is quite confusing to see qemu suddenly exit after receiving a help
option in hmp.

Do exit(0) from vl.c instead.

Signed-off-by: Anton Nefedov <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Paolo Bonzini <address@hidden>


  Commit: eb22aeca65f3769af33ba559757b42f24f743c18
      
https://github.com/qemu/qemu/commit/eb22aeca65f3769af33ba559757b42f24f743c18
  Author: Daniel P. Berrange <address@hidden>
  Date:   2017-08-01 (Tue, 01 Aug 2017)

  Changed paths:
    M qemu-doc.texi

  Log Message:
  -----------
  docs: document deprecation policy & deprecated features in appendix

The deprecation of features in QEMU is totally adhoc currently,
with no way for the user to get a list of what is deprecated
in each release. This adds an appendix to the doc that records
when each deprecation was made and provides text explaining
what to use instead, if anything.

Since there has been no formal policy around removal of deprecated
features in the past, any deprecations prior to 2.10.0 are to be
treated as if they had been made at the 2.10.0 release. Thus the
earliest that existing deprecations will be deleted is the start
of the 2.12.0 cycle.

Signed-off-by: Daniel P. Berrange <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Thomas Huth <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Paolo Bonzini <address@hidden>


  Commit: 4fadfa00301695a4985e2a229cab857b2ce5c775
      
https://github.com/qemu/qemu/commit/4fadfa00301695a4985e2a229cab857b2ce5c775
  Author: Peng Hao <address@hidden>
  Date:   2017-08-01 (Tue, 01 Aug 2017)

  Changed paths:
    M target/i386/kvm.c

  Log Message:
  -----------
  target-i386: kvm_get/put_vcpu_events don't handle sipi_vector

qemu call kvm_get_vcpu_events, and kernel return sipi_vector always
0, never valid when reporting to user space. But when qemu calls
kvm_put_vcpu_events will make sipi_vector in kernel be 0. This will
accidently modify sipi_vector when sipi_vector in kernel is not 0.

Signed-off-by: Peng Hao <address@hidden>
Reviewed-by: Liu Yi <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Paolo Bonzini <address@hidden>


  Commit: f5aa69bdc3418773f26747ca282c291519626ece
      
https://github.com/qemu/qemu/commit/f5aa69bdc3418773f26747ca282c291519626ece
  Author: Anthony PERARD <address@hidden>
  Date:   2017-08-01 (Tue, 01 Aug 2017)

  Changed paths:
    M exec.c

  Log Message:
  -----------
  exec: Add lock parameter to qemu_ram_ptr_length

Commit 04bf2526ce87f21b32c9acba1c5518708c243ad0 (exec: use
qemu_ram_ptr_length to access guest ram) start using qemu_ram_ptr_length
instead of qemu_map_ram_ptr, but when used with Xen, the behavior of
both function is different. They both call xen_map_cache, but one with
"lock", meaning the mapping of guest memory is never released
implicitly, and the second one without, which means, mapping can be
release later, when needed.

In the context of address_space_{read,write}_continue, the ptr to those
mapping should not be locked because it is used immediatly and never
used again.

The lock parameter make it explicit in which context qemu_ram_ptr_length
is called.

Signed-off-by: Anthony PERARD <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Stefano Stabellini <address@hidden>
Signed-off-by: Paolo Bonzini <address@hidden>


  Commit: 393c13b940be8f2e5b126cd9f442c12e7ecb4cac
      
https://github.com/qemu/qemu/commit/393c13b940be8f2e5b126cd9f442c12e7ecb4cac
  Author: Paolo Bonzini <address@hidden>
  Date:   2017-08-01 (Tue, 01 Aug 2017)

  Changed paths:
    M hw/bt/sdp.c

  Log Message:
  -----------
  bt: stop the sdp memory allocation craziness

Clang static analyzer reports a memory leak.  Actually, the allocated
memory escapes here:
   record->attribute_list[record->attributes].pair = data;

but clang is correct that the memory might leak if len is zero.  We
know it isn't; assert that it is the case.

The craziness doesn't end there.  The memory is freed by
bt_l2cap_sdp_close_ch:
  g_free(sdp->service_list[i].attribute_list->pair);

which actually should have been written like this:
  g_free(sdp->service_list[i].attribute_list[0].pair);

The attribute_list is sorted with qsort; but indeed the first
entry of attribute_list should point to "data" even after the qsort,
because the first record has id SDP_ATTR_RECORD_HANDLE, whose
numeric value is zero.

But hang on.  The qsort function is

    static int sdp_attributeid_compare(
          const struct sdp_service_attribute_s *a,
          const struct sdp_service_attribute_s *b)
    {
  return (int) b->attribute_id - a->attribute_id;
    }

but no one ever writes attribute_id.  So it only works if qsort is
stable, and who knows what else is broken, but we can fix it by
setting attribute_id in the while loop.

Signed-off-by: Paolo Bonzini <address@hidden>


  Commit: 8bfce83a3b50d45b7107a50a6cd61d5304a925a2
      
https://github.com/qemu/qemu/commit/8bfce83a3b50d45b7107a50a6cd61d5304a925a2
  Author: Daniel P. Berrange <address@hidden>
  Date:   2017-08-01 (Tue, 01 Aug 2017)

  Changed paths:
    M qemu-options.hx

  Log Message:
  -----------
  qemu-options: document existance of versioned machine types

The -machine docs did not explain what the versioned machine
types are for, nor that they'll be maintained across
releases.

Signed-off-by: Daniel P. Berrange <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Paolo Bonzini <address@hidden>


  Commit: 1931076077254a2886daa7c830c7838ebd1f81ef
      
https://github.com/qemu/qemu/commit/1931076077254a2886daa7c830c7838ebd1f81ef
  Author: Jay Zhou <address@hidden>
  Date:   2017-08-01 (Tue, 01 Aug 2017)

  Changed paths:
    M memory.c

  Log Message:
  -----------
  migration: optimize the downtime

Qemu_savevm_state_cleanup takes about 300ms in my ram migration tests
with a 8U24G vm(20G is really occupied), the main cost comes from
KVM_SET_USER_MEMORY_REGION ioctl when mem.memory_size = 0 in
kvm_set_user_memory_region. In kmod, the main cost is
kvm_zap_obsolete_pages, which traverses the active_mmu_pages list to
zap the unsync sptes.

It can be optimized by delaying memory_global_dirty_log_stop to the next
vm_start.

Changes v2->v3:
 - NULL VMChangeStateHandler if it is deleted and protect the scenario
   of nested invocations of memory_global_dirty_log_start/stop [Paolo]

Changes v1->v2:
 - create a VMChangeStateHandler in memory.c to reduce the coupling [Paolo]

Signed-off-by: Jay Zhou <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Paolo Bonzini <address@hidden>


  Commit: dfaea0c1a8482c5410917f91a9e2b88d4954b69e
      
https://github.com/qemu/qemu/commit/dfaea0c1a8482c5410917f91a9e2b88d4954b69e
  Author: Mao Zhongyi <address@hidden>
  Date:   2017-08-01 (Tue, 01 Aug 2017)

  Changed paths:
    M hw/scsi/vmw_pvscsi.c

  Log Message:
  -----------
  hw/scsi/vmw_pvscsi: Remove the dead error handling

qemu_bh_new() is a wrapper around aio_bh_new(), which returns
null only when g_new() does. It doesn't. So remove the dead
error handling.

Reviewed-by: Dmitry Fleytman <address@hidden>
Cc: Paolo Bonzini <address@hidden>
Cc: Markus Armbruster <address@hidden>
Signed-off-by: Mao Zhongyi <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Paolo Bonzini <address@hidden>


  Commit: fafeb41cd0abac412d7c77da1b680d5df805b454
      
https://github.com/qemu/qemu/commit/fafeb41cd0abac412d7c77da1b680d5df805b454
  Author: Mao Zhongyi <address@hidden>
  Date:   2017-08-01 (Tue, 01 Aug 2017)

  Changed paths:
    M hw/scsi/vmw_pvscsi.c

  Log Message:
  -----------
  hw/scsi/vmw_pvscsi: Convert to realize

Convert a device model where initialization obviously
can't fail, make it implement realize() rather than init().

Reviewed-by: Dmitry Fleytman <address@hidden>
Cc: Paolo Bonzini <address@hidden>
Cc: Markus Armbruster <address@hidden>
Signed-off-by: Mao Zhongyi <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Paolo Bonzini <address@hidden>


  Commit: bc706fa9039d875bb33ad7e9b27423a42455e17b
      
https://github.com/qemu/qemu/commit/bc706fa9039d875bb33ad7e9b27423a42455e17b
  Author: Paolo Bonzini <address@hidden>
  Date:   2017-08-01 (Tue, 01 Aug 2017)

  Changed paths:
    M tests/rtc-test.c

  Log Message:
  -----------
  rtc-test: cleanup register_b_set_flag test

Introduce set_datetime_bcd/assert_datetime_bcd, and handle
UIP correctly.

Signed-off-by: Paolo Bonzini <address@hidden>


  Commit: da3a392f0562c1deffd2bc1258d2893bb397009c
      
https://github.com/qemu/qemu/commit/da3a392f0562c1deffd2bc1258d2893bb397009c
  Author: Paolo Bonzini <address@hidden>
  Date:   2017-08-01 (Tue, 01 Aug 2017)

  Changed paths:
    M tests/rtc-test.c

  Log Message:
  -----------
  rtc-test: introduce more update tests

Test divider reset and UIP behavior.

Signed-off-by: Paolo Bonzini <address@hidden>


  Commit: 6a51d83a17e8213db353dd6756685fd9e3513e13
      
https://github.com/qemu/qemu/commit/6a51d83a17e8213db353dd6756685fd9e3513e13
  Author: Paolo Bonzini <address@hidden>
  Date:   2017-08-01 (Tue, 01 Aug 2017)

  Changed paths:
    M hw/timer/mc146818rtc.c

  Log Message:
  -----------
  mc146818rtc: simplify check_update_timer

Move all the optimized cases together, since they all have UF=1 in
common.

Signed-off-by: Paolo Bonzini <address@hidden>


  Commit: 33f21e4f044ac1c37f60edc1f1aee628be8f463b
      
https://github.com/qemu/qemu/commit/33f21e4f044ac1c37f60edc1f1aee628be8f463b
  Author: Paolo Bonzini <address@hidden>
  Date:   2017-08-01 (Tue, 01 Aug 2017)

  Changed paths:
    M hw/timer/mc146818rtc.c

  Log Message:
  -----------
  mc146818rtc: implement UIP latching as intended

In some cases, the guest can observe the wrong ordering of UIP and
interrupts.  This can happen if the VCPU exit is timed like this:
      iothread                 VCPU
                            ... wait for interrupt ...
t-100ns                           read register A
t          wake up, take BQL
t+100ns                             update_in_progress
                                return false
                              return UIP=0
     trigger interrupt

The interrupt is late; the VCPU expected the falling edge of UIP to
happen after the interrupt.  update_in_progress is already trying to
cover this case by latching UIP if the timer is going to fire soon,
and the fix is documented in the commit message for commit 56038ef623
("RTC: Update the RTC clock only when reading it", 2012-09-10).  It
cannot be tested with qtest, because its timing of interrupts vs. reads
is exact.

However, the implementation was incorrect because UIP cmos_ioport_read
cleared register A instead of leaving that to rtc_update_timer.  Fixing
the implementation of cmos_ioport_read to match the commit message,
however, breaks the "uip-stuck" test case from the previous patch.
To fix it, skip update timer optimizations if UIP has been latched.

Signed-off-by: Paolo Bonzini <address@hidden>


  Commit: 82d3d409b8b650164817ead0cb48298c6973d731
      
https://github.com/qemu/qemu/commit/82d3d409b8b650164817ead0cb48298c6973d731
  Author: Peter Maydell <address@hidden>
  Date:   2017-08-01 (Tue, 01 Aug 2017)

  Changed paths:
    M accel/accel.c
    M chardev/char-fd.c
    M chardev/char.c
    M exec.c
    M hw/bt/sdp.c
    M hw/scsi/vmw_pvscsi.c
    M hw/timer/mc146818rtc.c
    M include/chardev/char-fd.h
    M include/chardev/char.h
    M include/exec/ram_addr.h
    M memory.c
    M qemu-doc.texi
    M qemu-options.hx
    M target/i386/kvm.c
    M tests/rtc-test.c
    M vl.c

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* Xen fix (Anthony)
* chardev fixes (Anton, Marc-André)
* small dead code removal (Zhongyi)
* documentation (Dan)
* bugfixes (David)
* decrease migration downtime (Jay)
* improved error output (Laurent)
* RTC tests and bugfix (me)
* Bluetooth clang analyzer fix (me)
* KVM CPU hotplug race (Peng Hao)
* Two other patches from Philippe's clang analyzer series

# gpg: Signature made Tue 01 Aug 2017 16:56:21 BST
# gpg:                using RSA key 0xBFFBD25F78C7AE83
# gpg: Good signature from "Paolo Bonzini <address@hidden>"
# gpg:                 aka "Paolo Bonzini <address@hidden>"
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream:
  mc146818rtc: implement UIP latching as intended
  mc146818rtc: simplify check_update_timer
  rtc-test: introduce more update tests
  rtc-test: cleanup register_b_set_flag test
  hw/scsi/vmw_pvscsi: Convert to realize
  hw/scsi/vmw_pvscsi: Remove the dead error handling
  migration: optimize the downtime
  qemu-options: document existance of versioned machine types
  bt: stop the sdp memory allocation craziness
  exec: Add lock parameter to qemu_ram_ptr_length
  target-i386: kvm_get/put_vcpu_events don't handle sipi_vector
  docs: document deprecation policy & deprecated features in appendix
  char: don't exit on hmp 'chardev-add help'
  char-fd: remove useless chr pointer
  accel: cleanup error output
  cpu_physical_memory_sync_dirty_bitmap: Fix alignment check
  vl.c/exit: pause cpus before closing block devices

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/3b64f272d36b...82d3d409b8b6