[Qemu-commits] [qemu/qemu] d069bc: multifd: Initialize local variable

[Peter Maydell]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: d069bcca6c8cfcc1f2b056cb794ad726ab7c5f16
      
https://github.com/qemu/qemu/commit/d069bcca6c8cfcc1f2b056cb794ad726ab7c5f16
  Author: Juan Quintela <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  multifd: Initialize local variable

Fill everything with zero, so the padding fields are also initialized.

Signed-off-by: Juan Quintela <address@hidden>
Reviewed-by: Daniel P. Berrangé <address@hidden>


  Commit: b99784ef6c3beff2c43ea8d3bc50cdcd7903658f
      
https://github.com/qemu/qemu/commit/b99784ef6c3beff2c43ea8d3bc50cdcd7903658f
  Author: Juan Quintela <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M tests/qtest/migration-test.c

  Log Message:
  -----------
  migration-test: Add migration multifd test

We set multifd-channels.

Signed-off-by: Juan Quintela <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Reviewed-by: Thomas Huth <address@hidden>
Tested-by: Wei Yang <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 4d65a6216bfc44891ac298b74a6921d479805131
      
https://github.com/qemu/qemu/commit/4d65a6216bfc44891ac298b74a6921d479805131
  Author: Juan Quintela <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration: Make sure that we don't call write() in case of error

If we are exiting due to an error/finish/.... Just don't try to even
touch the channel with one IO operation.

Signed-off-by: Juan Quintela <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 6a22c5444c7cf79aaa47d1bc9485cf5dfe90429b
      
https://github.com/qemu/qemu/commit/6a22c5444c7cf79aaa47d1bc9485cf5dfe90429b
  Author: Juan Quintela <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M tests/qtest/migration-test.c

  Log Message:
  -----------
  migration-test: introduce functions to handle string parameters

Signed-off-by: Juan Quintela <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: ddad81bd28de665475a87693a93e6cf5d6fd8bab
      
https://github.com/qemu/qemu/commit/ddad81bd28de665475a87693a93e6cf5d6fd8bab
  Author: Laurent Vivier <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M vl.c

  Log Message:
  -----------
  runstate: ignore finishmigrate -> prelaunch transition

Commit 1bd71dce4bf2 tries to prevent a finishmigrate -> prelaunch
transition by exiting at the beginning of the main_loop_should_exit()
function if the state is already finishmigrate.

As the finishmigrate state is set in the migration thread it can
happen concurrently to the function. The migration thread and the
function are normally protected by the iothread mutex and thus the
state should no evolve between the start of the function and its end.

Unfortunately during the function life the lock is released by
pause_all_vcpus() just before the point we need to be sure we are
not in finishmigrate state and if the migration thread is waiting
for the lock it will take the opportunity to change the state
to finishmigrate.

The only way to be sure we are not in the finishmigrate state when
we need is to check the state after the pause_all_vcpus() function.

Fixes: 1bd71dce4bf2 ("runstate: ignore exit request in finish migrate state")
Signed-off-by: Laurent Vivier <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 03acb4e94deada2dd32412725420db82682f2e2e
      
https://github.com/qemu/qemu/commit/03acb4e94deada2dd32412725420db82682f2e2e
  Author: Daniel Henrique Barboza <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  ram.c: remove unneeded labels

ram_save_queue_pages() has an 'err' label that can be replaced by
'return -1' instead.

Same thing with ram_discard_range(), and in this case we can also
get rid of the 'ret' variable and return either '-1' on error
or the result of ram_block_discard_range().

CC: Juan Quintela <address@hidden>
CC: Dr. David Alan Gilbert <address@hidden>
Signed-off-by: Daniel Henrique Barboza <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 97e1e06780e70f6e98a0d2df881e0c0927d3aeb6
      
https://github.com/qemu/qemu/commit/97e1e06780e70f6e98a0d2df881e0c0927d3aeb6
  Author: Dr. David Alan Gilbert <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M migration/migration.c
    M migration/migration.h
    M migration/ram.c
    M migration/trace-events

  Log Message:
  -----------
  migration: Rate limit inside host pages

When using hugepages, rate limiting is necessary within each huge
page, since a 1G huge page can take a significant time to send, so
you end up with bursty behaviour.

Fixes: 4c011c37ecb3 ("postcopy: Send whole huge pages")
Reported-by: Lin Ma <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Reviewed-by: Peter Xu <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 2667c98722ddec12c9f43902c6c5668023e1dce9
      
https://github.com/qemu/qemu/commit/2667c98722ddec12c9f43902c6c5668023e1dce9
  Author: Fangrui Song <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M migration/migration.c

  Log Message:
  -----------
  migration: Fix incorrect integer->float conversion caught by clang

Clang does not like qmp_migrate_set_downtime()'s code to clamp double
@value to 0..INT64_MAX:

    qemu/migration/migration.c:2038:24: error: implicit conversion from 'long' 
to 'double' changes value from 9223372036854775807 to 9223372036854775808 
[-Werror,-Wimplicit-int-float-conversion]

The warning will be enabled by default in clang 10. It is not
available for clang <= 9.

The clamp is actually useless; @value is checked to be within
0..MAX_MIGRATE_DOWNTIME_SECONDS immediately before.  Delete it.

While there, make the conversion from double to int64_t explicit.

Signed-off-by: Fangrui Song <address@hidden>
Reviewed-by: Markus Armbruster <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Reviewed-by: Richard Henderson <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
[Patch split, commit message improved]
Signed-off-by: Markus Armbruster <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 603d5a42d3cca58d69dbe9728a47767896486cae
      
https://github.com/qemu/qemu/commit/603d5a42d3cca58d69dbe9728a47767896486cae
  Author: Yury Kotov <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M migration/migration.c

  Log Message:
  -----------
  migration: Fix the re-run check of the migrate-incoming command

The current check sets an error but doesn't fail the command.
This may cause a problem if new connection attempt by the same URI
affects the first connection.

Signed-off-by: Yury Kotov <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 619bd31d1f7cc5878e75e989a78d6f1c79356d4a
      
https://github.com/qemu/qemu/commit/619bd31d1f7cc5878e75e989a78d6f1c79356d4a
  Author: Marc-André Lureau <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M exec.c

  Log Message:
  -----------
  misc: use QEMU_IS_ALIGNED

Signed-off-by: Marc-André Lureau <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Reviewed-by: Stefan Berger <address@hidden>
Reviewed-by: Paolo Bonzini <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: bd5de61e7b926b5bb9dae254630bf3c8be575bc0
      
https://github.com/qemu/qemu/commit/bd5de61e7b926b5bb9dae254630bf3c8be575bc0
  Author: Scott Cheloha <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M migration/savevm.c

  Log Message:
  -----------
  migration: add savevm_state_handler_remove()

Create a function to abstract common logic needed when removing a
SaveStateEntry element from the savevm_state.handlers queue.

For now we just remove the element.  Soon it will involve additional
cleanup.

Signed-off-by: Scott Cheloha <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 174723ffe586e453f8ed4010ea07bbf79805b63f
      
https://github.com/qemu/qemu/commit/174723ffe586e453f8ed4010ea07bbf79805b63f
  Author: Scott Cheloha <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M migration/savevm.c

  Log Message:
  -----------
  migration: savevm_state_handler_insert: constant-time element insertion

savevm_state's SaveStateEntry TAILQ is a priority queue.  Priority
sorting is maintained by searching from head to tail for a suitable
insertion spot.  Insertion is thus an O(n) operation.

If we instead keep track of the head of each priority's subqueue
within that larger queue we can reduce this operation to O(1) time.

savevm_state_handler_remove() becomes slightly more complex to
accomodate these gains: we need to replace the head of a priority's
subqueue when removing it.

With O(1) insertion, booting VMs with many SaveStateEntry objects is
more plausible.  For example, a ppc64 VM with maxmem=8T has 40000 such
objects to insert.

Signed-off-by: Scott Cheloha <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: e65cec5e5d97927d22b39167d3e8edeffc771788
      
https://github.com/qemu/qemu/commit/e65cec5e5d97927d22b39167d3e8edeffc771788
  Author: Yury Kotov <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration/ram: Yield periodically to the main loop

Usually, incoming migration coroutine yields to the main loop
while its IO-channel is waiting for data to receive. But there is a case
when RAM migration and data receive have the same speed: VM with huge
zeroed RAM. In this case, IO-channel won't read and thus the main loop
is stuck and for instance, it doesn't respond to QMP commands.

For this case, yield periodically, but not too often, so as not to
affect the speed of migration.

Signed-off-by: Yury Kotov <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 2e36bc1b884a6fc6bc626b13db4960773ad976f7
      
https://github.com/qemu/qemu/commit/2e36bc1b884a6fc6bc626b13db4960773ad976f7
  Author: Wei Yang <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration/postcopy: reduce memset when it is zero page and 
matches_target_page_size

In this case, page_buffer content would not be used.

Skip this to save some time.

Signed-off-by: Wei Yang <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: ca1a6b708ba2d94f2dd64bb8ec04223a708c9b40
      
https://github.com/qemu/qemu/commit/ca1a6b708ba2d94f2dd64bb8ec04223a708c9b40
  Author: Wei Yang <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration/postcopy: wait for decompress thread in precopy

Compress is not supported with postcopy, it is safe to wait for
decompress thread just in precopy.

This is a preparation for later patch.

Signed-off-by: Wei Yang <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 4cbb3c63c165ae71b71f764030c7b16d98679391
      
https://github.com/qemu/qemu/commit/4cbb3c63c165ae71b71f764030c7b16d98679391
  Author: Wei Yang <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration/postcopy: count target page number to decide the place_needed

In postcopy, it requires to place whole host page instead of target
page.

Currently, it relies on the page offset to decide whether this is the
last target page. We also can count the target page number during the
iteration. When the number of target page equals
(host page size / target page size), this means it is the last target
page in the host page.

This is a preparation for non-ordered target page transmission.

Signed-off-by: Wei Yang <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: e5e73b0f90d0ac956782378b92897455e8104a7e
      
https://github.com/qemu/qemu/commit/e5e73b0f90d0ac956782378b92897455e8104a7e
  Author: Wei Yang <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration/postcopy: set all_zero to true on the first target page

For the first target page, all_zero is set to true for this round check.

After target_pages introduced, we could leverage this variable instead
of checking the address offset.

Signed-off-by: Wei Yang <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 91ba442f5c26eaf45e2f78fc7e40fe6a7fdf7b9d
      
https://github.com/qemu/qemu/commit/91ba442f5c26eaf45e2f78fc7e40fe6a7fdf7b9d
  Author: Wei Yang <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration/postcopy: enable random order target page arrival

After using number of target page received to track one host page, we
could have the capability to handle random order target page arrival in
one host page.

This is a preparation for enabling compress during postcopy.

Signed-off-by: Wei Yang <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 644acf99b8cb8437f65600cf00c2e090bf3e3bc2
      
https://github.com/qemu/qemu/commit/644acf99b8cb8437f65600cf00c2e090bf3e3bc2
  Author: Wei Yang <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M migration/migration.c
    M migration/ram.c

  Log Message:
  -----------
  migration/postcopy: enable compress during postcopy

postcopy requires to place a whole host page, while migration thread
migrate memory in target page size. This makes postcopy need to collect
all target pages in one host page before placing via userfaultfd.

To enable compress during postcopy, there are two problems to solve:

    1. Random order for target page arrival
    2. Target pages in one host page arrives without interrupt by target
       page from other host page

The first one is handled by previous cleanup patch.

This patch handles the second one by:

    1. Flush compress thread for each host page
    2. Wait for decompress thread for before placing host page

Signed-off-by: Wei Yang <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: eab54aa78ffd9fb7895b20fc2761ee998479489b
      
https://github.com/qemu/qemu/commit/eab54aa78ffd9fb7895b20fc2761ee998479489b
  Author: Wei Yang <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration/multifd: clean pages after filling packet

This is a preparation for the next patch:

    not use multifd during postcopy.

Without enabling postcopy, everything looks good. While after enabling
postcopy, migration may fail even not use multifd during postcopy. The
reason is the pages is not properly cleared and *old* target page will
continue to be transferred.

After clean pages, migration succeeds.

Signed-off-by: Wei Yang <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: c6b3a2e0c4342d171b35970921bff709c007dd62
      
https://github.com/qemu/qemu/commit/c6b3a2e0c4342d171b35970921bff709c007dd62
  Author: Wei Yang <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration/multifd: not use multifd during postcopy

We don't support multifd during postcopy, but user still could enable
both multifd and postcopy. This leads to migration failure.

Skip multifd during postcopy.

Signed-off-by: Wei Yang <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: f76e32eb05041ab001184ab16afb56524adccd0c
      
https://github.com/qemu/qemu/commit/f76e32eb05041ab001184ab16afb56524adccd0c
  Author: Jiahui Cen <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration/multifd: fix nullptr access in terminating multifd threads

One multifd channel will shutdown all the other multifd's IOChannel when it
fails to receive an IOChannel. In this senario, if some multifds had not
received its IOChannel yet, it would try to shutdown its IOChannel which could
cause nullptr access at qio_channel_shutdown.

Here is the coredump stack:
    #0  object_get_class (obj=obj@entry=0x0) at qom/object.c:908
    #1  0x00005563fdbb8f4a in qio_channel_shutdown (ioc=0x0, 
how=QIO_CHANNEL_SHUTDOWN_BOTH, errp=0x0) at io/channel.c:355
    #2  0x00005563fd7b4c5f in multifd_recv_terminate_threads (err=<optimized 
out>) at migration/ram.c:1280
    #3  0x00005563fd7bc019 in multifd_recv_new_channel 
(ioc=ioc@entry=0x556400255610, errp=errp@entry=0x7ffec07dce00) at 
migration/ram.c:1478
    #4  0x00005563fda82177 in migration_ioc_process_incoming 
(ioc=ioc@entry=0x556400255610, errp=errp@entry=0x7ffec07dce30) at 
migration/migration.c:605
    #5  0x00005563fda8567d in migration_channel_process_incoming 
(ioc=0x556400255610) at migration/channel.c:44
    #6  0x00005563fda83ee0 in socket_accept_incoming_migration 
(listener=0x5563fff6b920, cioc=0x556400255610, opaque=<optimized out>) at 
migration/socket.c:166
    #7  0x00005563fdbc25cd in qio_net_listener_channel_func (ioc=<optimized 
out>, condition=<optimized out>, opaque=<optimized out>) at io/net-listener.c:54
    #8  0x00007f895b6fe9a9 in g_main_context_dispatch () from 
/usr/lib64/libglib-2.0.so.0
    #9  0x00005563fdc18136 in glib_pollfds_poll () at util/main-loop.c:218
    #10 0x00005563fdc181b5 in os_host_main_loop_wait (timeout=1000000000) at 
util/main-loop.c:241
    #11 0x00005563fdc183a2 in main_loop_wait (nonblocking=nonblocking@entry=0) 
at util/main-loop.c:517
    #12 0x00005563fd8edb37 in main_loop () at vl.c:1791
    #13 0x00005563fd74fd45 in main (argc=<optimized out>, argv=<optimized out>, 
envp=<optimized out>) at vl.c:4473

To fix it up, let's check p->c before calling qio_channel_shutdown.

Signed-off-by: Jiahui Cen <address@hidden>
Signed-off-by: Ying Fang <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 9560a48ecc0c20d87bc458a6db77fba651605819
      
https://github.com/qemu/qemu/commit/9560a48ecc0c20d87bc458a6db77fba651605819
  Author: Jiahui Cen <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration/multifd: fix destroyed mutex access in terminating multifd threads

One multifd will lock all the other multifds' IOChannel mutex to inform them
to quit by setting p->quit or shutting down p->c. In this senario, if some
multifds had already been terminated and 
multifd_load_cleanup/multifd_save_cleanup
had destroyed their mutex, it could cause destroyed mutex access when trying
lock their mutex.

Here is the coredump stack:
    #0  0x00007f81a2794437 in raise () from /usr/lib64/libc.so.6
    #1  0x00007f81a2795b28 in abort () from /usr/lib64/libc.so.6
    #2  0x00007f81a278d1b6 in __assert_fail_base () from /usr/lib64/libc.so.6
    #3  0x00007f81a278d262 in __assert_fail () from /usr/lib64/libc.so.6
    #4  0x000055eb1bfadbd3 in qemu_mutex_lock_impl (mutex=0x55eb1e2d1988, 
file=<optimized out>, line=<optimized out>) at util/qemu-thread-posix.c:64
    #5  0x000055eb1bb4564a in multifd_send_terminate_threads (err=<optimized 
out>) at migration/ram.c:1015
    #6  0x000055eb1bb4bb7f in multifd_send_thread (opaque=0x55eb1e2d19f8) at 
migration/ram.c:1171
    #7  0x000055eb1bfad628 in qemu_thread_start (args=0x55eb1e170450) at 
util/qemu-thread-posix.c:502
    #8  0x00007f81a2b36df5 in start_thread () from /usr/lib64/libpthread.so.0
    #9  0x00007f81a286048d in clone () from /usr/lib64/libc.so.6

To fix it up, let's destroy the mutex after all the other multifd threads had
been terminated.

Signed-off-by: Jiahui Cen <address@hidden>
Signed-off-by: Ying Fang <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 8bba004ccabd0d71e8b5ca056e576d2737c17305
      
https://github.com/qemu/qemu/commit/8bba004ccabd0d71e8b5ca056e576d2737c17305
  Author: Alexey Romko <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  Bug #1829242 correction.

Added type conversions to ram_addr_t before all left shifts of page
indexes to TARGET_PAGE_BITS, to correct overflows when the page
address was 4Gb and more.

Signed-off-by: Alexey Romko <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 1df2c9a26fcb2fa32d099f8e9adcdae4207872e3
      
https://github.com/qemu/qemu/commit/1df2c9a26fcb2fa32d099f8e9adcdae4207872e3
  Author: Peter Xu <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M backends/dbus-vmstate.c
    M hw/arm/stellaris.c
    M hw/core/qdev.c
    M hw/display/ads7846.c
    M hw/i2c/core.c
    M hw/input/stellaris_input.c
    M hw/intc/apic_common.c
    M hw/misc/max111x.c
    M hw/net/eepro100.c
    M hw/pci/pci.c
    M hw/ppc/spapr.c
    M hw/timer/arm_timer.c
    M hw/tpm/tpm_emulator.c
    M include/migration/vmstate.h
    M migration/savevm.c

  Log Message:
  -----------
  migration: Define VMSTATE_INSTANCE_ID_ANY

Define the new macro VMSTATE_INSTANCE_ID_ANY for callers who wants to
auto-generate the vmstate instance ID.  Previously it was hard coded
as -1 instead of this macro.  It helps to change this default value in
the follow up patches.  No functional change.

Signed-off-by: Peter Xu <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 93062e23619e057743757ee53bf7f8e07f7a3710
      
https://github.com/qemu/qemu/commit/93062e23619e057743757ee53bf7f8e07f7a3710
  Author: Peter Xu <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M hw/intc/apic_common.c
    M include/migration/register.h
    M include/migration/vmstate.h
    M migration/savevm.c
    M stubs/vmstate.c

  Log Message:
  -----------
  migration: Change SaveStateEntry.instance_id into uint32_t

It was always used as 32bit, so define it as used to be clear.
Instead of using -1 as the auto-gen magic value, we switch to
UINT32_MAX.  We also make sure that we don't auto-gen this value to
avoid overflowed instance IDs without being noticed.

Suggested-by: Juan Quintela <address@hidden>
Signed-off-by: Peter Xu <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 0ab994867c365db21e15f9503922c79234d8e40e
      
https://github.com/qemu/qemu/commit/0ab994867c365db21e15f9503922c79234d8e40e
  Author: Peter Xu <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M hw/intc/apic_common.c

  Log Message:
  -----------
  apic: Use 32bit APIC ID for migration instance ID

Migration is silently broken now with x2apic config like this:

     -smp 200,maxcpus=288,sockets=2,cores=72,threads=2 \
     -device intel-iommu,intremap=on,eim=on

After migration, the guest kernel could hang at anything, due to
x2apic bit not migrated correctly in IA32_APIC_BASE on some vcpus, so
any operations related to x2apic could be broken then (e.g., RDMSR on
x2apic MSRs could fail because KVM would think that the vcpu hasn't
enabled x2apic at all).

The issue is that the x2apic bit was never applied correctly for vcpus
whose ID > 255 when migrate completes, and that's because when we
migrate APIC we use the APICCommonState.id as instance ID of the
migration stream, while that's too short for x2apic.

Let's use the newly introduced initial_apic_id for that.

Signed-off-by: Peter Xu <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Reviewed-by: Eduardo Habkost <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 4746dbf8a98d560e20dbd22d0e8405b38478b409
      
https://github.com/qemu/qemu/commit/4746dbf8a98d560e20dbd22d0e8405b38478b409
  Author: Eric Auger <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M include/migration/vmstate.h
    M include/qemu/queue.h
    M migration/trace-events
    M migration/vmstate-types.c
    M tests/test-vmstate.c

  Log Message:
  -----------
  migration: Support QLIST migration

Support QLIST migration using the same principle as QTAILQ:
94869d5c52 ("migration: migrate QTAILQ").

The VMSTATE_QLIST_V macro has the same proto as VMSTATE_QTAILQ_V.
The change mainly resides in QLIST RAW macros: QLIST_RAW_INSERT_HEAD
and QLIST_RAW_REVERSE.

Tests also are provided.

Signed-off-by: Eric Auger <address@hidden>
Reviewed-by: Peter Xu <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: ddac5cb2d95774cd019bfaf93c54ffd921095fea
      
https://github.com/qemu/qemu/commit/ddac5cb2d95774cd019bfaf93c54ffd921095fea
  Author: Juan Quintela <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  multifd: Be consistent about using uint64_t

We transmit ram_addr_t always as uint64_t.  Be consistent in its
use (on 64bit system, it is always uint64_t problem is 32bits).

Signed-off-by: Juan Quintela <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 26deea00260139fc5f323c3bf9db82a5d470538a
      
https://github.com/qemu/qemu/commit/26deea00260139fc5f323c3bf9db82a5d470538a
  Author: Peter Maydell <address@hidden>
  Date:   2020-01-20 (Mon, 20 Jan 2020)

  Changed paths:
    M backends/dbus-vmstate.c
    M exec.c
    M hw/arm/stellaris.c
    M hw/core/qdev.c
    M hw/display/ads7846.c
    M hw/i2c/core.c
    M hw/input/stellaris_input.c
    M hw/intc/apic_common.c
    M hw/misc/max111x.c
    M hw/net/eepro100.c
    M hw/pci/pci.c
    M hw/ppc/spapr.c
    M hw/timer/arm_timer.c
    M hw/tpm/tpm_emulator.c
    M include/migration/register.h
    M include/migration/vmstate.h
    M include/qemu/queue.h
    M migration/migration.c
    M migration/migration.h
    M migration/ram.c
    M migration/savevm.c
    M migration/trace-events
    M migration/vmstate-types.c
    M stubs/vmstate.c
    M tests/qtest/migration-test.c
    M tests/test-vmstate.c
    M vl.c

  Log Message:
  -----------
  Merge remote-tracking branch 
'remotes/juanquintela/tags/migration-pull-pull-request' into staging

Migration pull request

# gpg: Signature made Mon 20 Jan 2020 10:29:53 GMT
# gpg:                using RSA key 1899FF8EDEBF58CCEE034B82F487EF185872D723
# gpg: Good signature from "Juan Quintela <address@hidden>" [full]
# gpg:                 aka "Juan Quintela <address@hidden>" [full]
# Primary key fingerprint: 1899 FF8E DEBF 58CC EE03  4B82 F487 EF18 5872 D723

* remotes/juanquintela/tags/migration-pull-pull-request: (29 commits)
  multifd: Be consistent about using uint64_t
  migration: Support QLIST migration
  apic: Use 32bit APIC ID for migration instance ID
  migration: Change SaveStateEntry.instance_id into uint32_t
  migration: Define VMSTATE_INSTANCE_ID_ANY
  Bug #1829242 correction.
  migration/multifd: fix destroyed mutex access in terminating multifd threads
  migration/multifd: fix nullptr access in terminating multifd threads
  migration/multifd: not use multifd during postcopy
  migration/multifd: clean pages after filling packet
  migration/postcopy: enable compress during postcopy
  migration/postcopy: enable random order target page arrival
  migration/postcopy: set all_zero to true on the first target page
  migration/postcopy: count target page number to decide the place_needed
  migration/postcopy: wait for decompress thread in precopy
  migration/postcopy: reduce memset when it is zero page and 
matches_target_page_size
  migration/ram: Yield periodically to the main loop
  migration: savevm_state_handler_insert: constant-time element insertion
  migration: add savevm_state_handler_remove()
  misc: use QEMU_IS_ALIGNED
  ...

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/7fb38daf256b...26deea002601