[Qemu-commits] [qemu/qemu] 449f91: migration/rdma: Fix uninitialised rdma_return_path

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 449f91b2c8d2ddf53629d908593f9acf52301b2d
      
https://github.com/qemu/qemu/commit/449f91b2c8d2ddf53629d908593f9acf52301b2d
  Author: Dr. David Alan Gilbert <address@hidden>
  Date:   2018-09-26 (Wed, 26 Sep 2018)

  Changed paths:
    M migration/rdma.c

  Log Message:
  -----------
  migration/rdma: Fix uninitialised rdma_return_path

Clang correctly errors out moaning that rdma_return_path
is used uninitialised in the earlier error paths.
Make it NULL so that the error path ignores it.

Fixes: 55cc1b5937a8e709e4c102e74b206281073aab82
Signed-off-by: Dr. David Alan Gilbert <address@hidden>
Reported-by: Cornelia Huck <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: be8b02edae9aa3a64c1e76cb4067b4bbbb170448
      
https://github.com/qemu/qemu/commit/be8b02edae9aa3a64c1e76cb4067b4bbbb170448
  Author: Xiao Guangrong <address@hidden>
  Date:   2018-09-26 (Wed, 26 Sep 2018)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration: fix calculating xbzrle_counters.cache_miss_rate

As Peter pointed out:
| - xbzrle_counters.cache_miss is done in save_xbzrle_page(), so it's
|   per-guest-page granularity
|
| - RAMState.iterations is done for each ram_find_and_save_block(), so
|   it's per-host-page granularity
|
| An example is that when we migrate a 2M huge page in the guest, we
| will only increase the RAMState.iterations by 1 (since
| ram_find_and_save_block() will be called once), but we might increase
| xbzrle_counters.cache_miss for 2M/4K=512 times (we'll call
| save_xbzrle_page() that many times) if all the pages got cache miss.
| Then IMHO the cache miss rate will be 512/1=51200% (while it should
| actually be just 100% cache miss).

And he also suggested as xbzrle_counters.cache_miss_rate is the only
user of rs->iterations we can adapt it to count target guest page
numbers

After that, rename 'iterations' to 'target_page_count' to better reflect
its meaning

Suggested-by: Peter Xu <address@hidden>
Reviewed-by: Peter Xu <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Xiao Guangrong <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: e8f3735fa3aa457303534121794b0452d22d440f
      
https://github.com/qemu/qemu/commit/e8f3735fa3aa457303534121794b0452d22d440f
  Author: Xiao Guangrong <address@hidden>
  Date:   2018-09-26 (Wed, 26 Sep 2018)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration: handle the error condition properly

ram_find_and_save_block() can return negative if any error hanppens,
however, it is completely ignored in current code

Signed-off-by: Xiao Guangrong <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 827beacb474c60b1801399c40ec717c65f9c4e34
      
https://github.com/qemu/qemu/commit/827beacb474c60b1801399c40ec717c65f9c4e34
  Author: Jose Ricardo Ziviani <address@hidden>
  Date:   2018-09-26 (Wed, 26 Sep 2018)

  Changed paths:
    M migration/savevm.c
    M vl.c

  Log Message:
  -----------
  Add a hint message to loadvm and exits on failure

This patch adds a small hint for the failure case of the load snapshot
process. It may be useful for users to remember that the VM
configuration has changed between the save and load processes.

(qemu) loadvm vm-20180903083641
Unknown savevm section or instance 'cpu_common' 4.
Make sure that your current VM setup matches your saved VM setup, including any 
hotplugged devices
Error -22 while loading VM state
(qemu) device_add host-spapr-cpu-core,core-id=4
(qemu) loadvm vm-20180903083641
(qemu) c
(qemu) info status
VM status: running

It also exits Qemu if the snapshot cannot be loaded before reaching the
main loop (-loadvm in the command line).

$ qemu-system-ppc64 ... -loadvm vm-20180903083641
qemu-system-ppc64: Unknown savevm section or instance 'cpu_common' 4.
Make sure that your current VM setup matches your saved VM setup, including any 
hotplugged devices
qemu-system-ppc64: Error -22 while loading VM state
$

Signed-off-by: Jose Ricardo Ziviani <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 48df9d8002914137b0a1e5c4ca4d402224f205f9
      
https://github.com/qemu/qemu/commit/48df9d8002914137b0a1e5c4ca4d402224f205f9
  Author: Xiao Guangrong <address@hidden>
  Date:   2018-09-26 (Wed, 26 Sep 2018)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration: do not flush_compressed_data at the end of iteration

flush_compressed_data() needs to wait all compression threads to
finish their work, after that all threads are free until the
migration feeds new request to them, reducing its call can improve
the throughput and use CPU resource more effectively

We do not need to flush all threads at the end of iteration, the
data can be kept locally until the memory block is changed or
memory migration starts over in that case we will meet a dirtied
page which may still exists in compression threads's ring

Signed-off-by: Xiao Guangrong <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 76e030004f7fbabc2f978e8cd44052e17abd55b2
      
https://github.com/qemu/qemu/commit/76e030004f7fbabc2f978e8cd44052e17abd55b2
  Author: Xiao Guangrong <address@hidden>
  Date:   2018-09-26 (Wed, 26 Sep 2018)

  Changed paths:
    M hmp.c
    M migration/migration.c
    M migration/ram.c
    M migration/ram.h
    M qapi/migration.json

  Log Message:
  -----------
  migration: show the statistics of compression

Currently, it includes:
pages: amount of pages compressed and transferred to the target VM
busy: amount of count that no free thread to compress data
busy-rate: rate of thread busy
compressed-size: amount of bytes after compression
compression-rate: rate of compressed size

Reviewed-by: Juan Quintela <address@hidden>
Reviewed-by: Peter Xu <address@hidden>
Signed-off-by: Xiao Guangrong <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 32b054954fb4a14b201f88d0b7fc94c60b8a740e
      
https://github.com/qemu/qemu/commit/32b054954fb4a14b201f88d0b7fc94c60b8a740e
  Author: Xiao Guangrong <address@hidden>
  Date:   2018-09-26 (Wed, 26 Sep 2018)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration: use save_page_use_compression in flush_compressed_data

It avoids to touch compression locks if xbzrle and compression
are both enabled

Signed-off-by: Xiao Guangrong <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: fe7307740124347157286dc42fe60182d53efd2d
      
https://github.com/qemu/qemu/commit/fe7307740124347157286dc42fe60182d53efd2d
  Author: Wei Huang <address@hidden>
  Date:   2018-09-26 (Wed, 26 Sep 2018)

  Changed paths:
    A tests/migration/Makefile
    R tests/migration/rebuild-x86-bootblock.sh
    A tests/migration/x86-a-b-bootblock.S
    R tests/migration/x86-a-b-bootblock.s

  Log Message:
  -----------
  tests/migration: Convert x86 boot block compilation script into Makefile

The x86 boot block header currently is generated with a shell script.
To better support other CPUs (e.g. aarch64), we convert the script
into Makefile. This allows us to 1) support cross-compilation easily,
and 2) avoid creating a script file for every architecture.

Note that, in the new design, the cross compiler prefix can be specified by
setting the CROSS_PREFIX in "make" command. Also to allow gcc pre-processor
to include the C-style file correctly, it also renames the
x86-a-b-bootblock.s file extension from .s to .S.

Reviewed-by: Juan Quintela <address@hidden>
Reviewed-by: Andrew Jones <address@hidden>
Signed-off-by: Wei Huang <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: d54927efdc93d0405c12ec94a9f466e41d8c3b52
      
https://github.com/qemu/qemu/commit/d54927efdc93d0405c12ec94a9f466e41d8c3b52
  Author: Wei Huang <address@hidden>
  Date:   2018-09-26 (Wed, 26 Sep 2018)

  Changed paths:
    M tests/migration-test.c
    M tests/migration/Makefile
    A tests/migration/i386/Makefile
    A tests/migration/i386/a-b-bootblock.S
    A tests/migration/i386/a-b-bootblock.h
    R tests/migration/x86-a-b-bootblock.S
    R tests/migration/x86-a-b-bootblock.h

  Log Message:
  -----------
  tests/migration: Support cross compilation in generating boot header file

Recently a new configure option, CROSS_CC_GUEST, was added to
$(TARGET)-softmmu/config-target.mak to support TCG-related tests. This
patch tries to leverage this option to support cross compilation when the
migration boot block file is being re-generated:

 * The x86 related files are moved to a new sub-dir (named ./i386).
 * A new top-layer Makefile is created in tests/migration/ directory.
   This Makefile searches and parses CROSS_CC_GUEST to generate CROSS_PREFIX.
   The CROSS_PREFIX, if available, is then passed to migration/$ARCH/Makefile.

Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Wei Huang <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: e51e711b1bef44861cf6a9d0f426d0a6bd119227
      
https://github.com/qemu/qemu/commit/e51e711b1bef44861cf6a9d0f426d0a6bd119227
  Author: Wei Huang <address@hidden>
  Date:   2018-09-26 (Wed, 26 Sep 2018)

  Changed paths:
    M tests/migration-test.c
    A tests/migration/migration-test.h

  Log Message:
  -----------
  tests/migration: Add migration-test header file

This patch moves the settings related migration-test from the
migration-test.c file to a new header file.

Reviewed-by: Juan Quintela <address@hidden>
Reviewed-by: Andrew Jones <address@hidden>
Signed-off-by: Wei Huang <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 9cf4bb8730c669c40550e635a9e2b8ee4f1664ca
      
https://github.com/qemu/qemu/commit/9cf4bb8730c669c40550e635a9e2b8ee4f1664ca
  Author: Dr. David Alan Gilbert <address@hidden>
  Date:   2018-09-26 (Wed, 26 Sep 2018)

  Changed paths:
    M migration/savevm.c

  Log Message:
  -----------
  migration/postcopy: Clear have_listen_thread

Clear have_listen_thread when we exit the thread.
The fallout from this was that various things thought there was
an ongoing postcopy after the postcopy had finished.

The case that failed was postcopy->savevm->loadvm.

This corresponds to RH bug https://bugzilla.redhat.com/show_bug.cgi?id=1608765

Signed-off-by: Dr. David Alan Gilbert <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Peter Xu <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 096c83b7219c5a2145435afc8be750281e9cb447
      
https://github.com/qemu/qemu/commit/096c83b7219c5a2145435afc8be750281e9cb447
  Author: Dr. David Alan Gilbert <address@hidden>
  Date:   2018-09-26 (Wed, 26 Sep 2018)

  Changed paths:
    M migration/savevm.c

  Log Message:
  -----------
  migration: cleanup in error paths in loadvm

There's a couple of error paths in qemu_loadvm_state
which happen early on but after we've initialised the
load state; that needs to be cleaned up otherwise
we can hit asserts if the state gets reinitialised later.

Signed-off-by: Dr. David Alan Gilbert <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Peter Xu <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: fc71e3e562b796c54a78d13485d93259a7473241
      
https://github.com/qemu/qemu/commit/fc71e3e562b796c54a78d13485d93259a7473241
  Author: Thomas Huth <address@hidden>
  Date:   2018-09-26 (Wed, 26 Sep 2018)

  Changed paths:
    M tests/migration-test.c

  Log Message:
  -----------
  tests/migration: Speed up the test on ppc64

The SLOF boot process is always quite slow ... but we can speed it up
a little bit by specifying "-nodefaults" and by using the "nvramrc"
variable instead of "boot-command" (since "nvramrc" is evaluated earlier
in the SLOF boot process than "boot-command").

Signed-off-by: Thomas Huth <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Reviewed-by: Laurent Vivier <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 0284a2a81c60209a6f5edfceafc20c20dfc42b04
      
https://github.com/qemu/qemu/commit/0284a2a81c60209a6f5edfceafc20c20dfc42b04
  Author: Marc-André Lureau <address@hidden>
  Date:   2018-09-26 (Wed, 26 Sep 2018)

  Changed paths:
    M migration/migration.c

  Log Message:
  -----------
  migration: fix QEMUFile leak

Spotted by ASAN while running:

$ tests/migration-test -p /x86_64/migration/postcopy/recovery

=================================================================
==18034==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 33864 byte(s) in 1 object(s) allocated from:
    #0 0x7f3da7f31e50 in calloc (/lib64/libasan.so.5+0xeee50)
    #1 0x7f3da644441d in g_malloc0 (/lib64/libglib-2.0.so.0+0x5241d)
    #2 0x55af9db15440 in qemu_fopen_channel_input 
/home/elmarco/src/qemu/migration/qemu-file-channel.c:183
    #3 0x55af9db15413 in channel_get_output_return_path 
/home/elmarco/src/qemu/migration/qemu-file-channel.c:159
    #4 0x55af9db0d4ac in qemu_file_get_return_path 
/home/elmarco/src/qemu/migration/qemu-file.c:78
    #5 0x55af9dad5e4f in open_return_path_on_source 
/home/elmarco/src/qemu/migration/migration.c:2295
    #6 0x55af9dadb3bf in migrate_fd_connect 
/home/elmarco/src/qemu/migration/migration.c:3111
    #7 0x55af9dae1bf3 in migration_channel_connect 
/home/elmarco/src/qemu/migration/channel.c:91
    #8 0x55af9daddeca in socket_outgoing_migration 
/home/elmarco/src/qemu/migration/socket.c:108
    #9 0x55af9e13d3db in qio_task_complete /home/elmarco/src/qemu/io/task.c:158
    #10 0x55af9e13ca03 in qio_task_thread_result 
/home/elmarco/src/qemu/io/task.c:89
    #11 0x7f3da643b1ca in g_idle_dispatch gmain.c:5535

Signed-off-by: Marc-André Lureau <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Peter Xu <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 05306935b1ae49107c2dc2f301574dd6c29b6838
      
https://github.com/qemu/qemu/commit/05306935b1ae49107c2dc2f301574dd6c29b6838
  Author: Fei Li <address@hidden>
  Date:   2018-09-26 (Wed, 26 Sep 2018)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration: fix the compression code

Add judgement in compress_threads_save_cleanup() to check whether the
static CompressParam *comp_param has been allocated. If not, just
return; or else segmentation fault will occur when using the NULL
comp_param's parameters.  One test case can reproduce this is: set
the compression on and migrate to a wrong nonexistent host IP address.

Our current code does not judge before handling comp_param[idx]'s quit
and cond that whether they have been initialized. If not initialized,
"qemu_mutex_lock_impl: Assertion `mutex->initialized' failed." will
occur. Fix this by squashing the terminate_compression_threads() into
compress_threads_save_cleanup() and employing the existing judgement
condition.  One test case can reproduce this error is: set the
compression on and fail to fully setup the default eight compression
thread in compress_threads_save_setup().

Signed-off-by: Fei Li <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Peter Xu <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 341ba0df4c69269cac839ddbacb2a0ca641a856d
      
https://github.com/qemu/qemu/commit/341ba0df4c69269cac839ddbacb2a0ca641a856d
  Author: Peter Maydell <address@hidden>
  Date:   2018-09-26 (Wed, 26 Sep 2018)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration/ram.c: Avoid taking address of fields in packed MultiFDInit_t struct

Taking the address of a field in a packed struct is a bad idea, because
it might not be actually aligned enough for that pointer type (and
thus cause a crash on dereference on some host architectures). Newer
versions of clang warn about this:

migration/ram.c:651:19: warning: taking address of packed member 'magic' of 
class or structure 'MultiFDInit_t' may result in an unaligned pointer value 
[-Waddress-of-packed-member]
migration/ram.c:652:19: warning: taking address of packed member 'version' of 
class or structure 'MultiFDInit_t' may result in an unaligned pointer value 
[-Waddress-of-packed-member]
migration/ram.c:737:19: warning: taking address of packed member 'magic' of 
class or structure 'MultiFDPacket_t' may result in an unaligned pointer value 
[-Waddress-of-packed-member]
migration/ram.c:745:19: warning: taking address of packed member 'version' of 
class or structure 'MultiFDPacket_t' may result in an unaligned pointer value 
[-Waddress-of-packed-member]
migration/ram.c:755:19: warning: taking address of packed member 'size' of 
class or structure 'MultiFDPacket_t' may result in an unaligned pointer value 
[-Waddress-of-packed-member]

Avoid the bug by not using the "modify in place" byteswapping
functions.

Signed-off-by: Peter Maydell <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Marc-André Lureau <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 042938f46e1c477419d1931381fdadffaa49d45e
      
https://github.com/qemu/qemu/commit/042938f46e1c477419d1931381fdadffaa49d45e
  Author: Peter Maydell <address@hidden>
  Date:   2018-09-28 (Fri, 28 Sep 2018)

  Changed paths:
    M hmp.c
    M migration/migration.c
    M migration/ram.c
    M migration/ram.h
    M migration/rdma.c
    M migration/savevm.c
    M qapi/migration.json
    M tests/migration-test.c
    A tests/migration/Makefile
    A tests/migration/i386/Makefile
    A tests/migration/i386/a-b-bootblock.S
    A tests/migration/i386/a-b-bootblock.h
    A tests/migration/migration-test.h
    R tests/migration/rebuild-x86-bootblock.sh
    R tests/migration/x86-a-b-bootblock.h
    R tests/migration/x86-a-b-bootblock.s
    M vl.c

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20180926a' 
into staging

Migration pull 2018-09-26

This supercedes Juan's pull from the 13th

# gpg: Signature made Wed 26 Sep 2018 18:07:30 BST
# gpg:                using RSA key 0516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <address@hidden>"
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A  9FA9 0516 331E BC5B FDE7

* remotes/dgilbert/tags/pull-migration-20180926a:
  migration/ram.c: Avoid taking address of fields in packed MultiFDInit_t struct
  migration: fix the compression code
  migration: fix QEMUFile leak
  tests/migration: Speed up the test on ppc64
  migration: cleanup in error paths in loadvm
  migration/postcopy: Clear have_listen_thread
  tests/migration: Add migration-test header file
  tests/migration: Support cross compilation in generating boot header file
  tests/migration: Convert x86 boot block compilation script into Makefile
  migration: use save_page_use_compression in flush_compressed_data
  migration: show the statistics of compression
  migration: do not flush_compressed_data at the end of iteration
  Add a hint message to loadvm and exits on failure
  migration: handle the error condition properly
  migration: fix calculating xbzrle_counters.cache_miss_rate
  migration/rdma: Fix uninitialised rdma_return_path

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/567ea80834fc...042938f46e1c
      **NOTE:** This service has been marked for deprecation: 
https://developer.github.com/changes/2018-04-25-github-services-deprecation/

      Functionality will be removed from GitHub.com on January 31st, 2019.