[Qemu-commits] [qemu/qemu] f22f92: migration: introduce postcopy-blocktime capability

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: f22f928ec929c425c5aa402b5cc4d6d1fa4be238
      
https://github.com/qemu/qemu/commit/f22f928ec929c425c5aa402b5cc4d6d1fa4be238
  Author: Alexey Perevalov <address@hidden>
  Date:   2018-04-25 (Wed, 25 Apr 2018)

  Changed paths:
    M migration/migration.c
    M migration/migration.h
    M qapi/migration.json

  Log Message:
  -----------
  migration: introduce postcopy-blocktime capability

Right now it could be used on destination side to
enable vCPU blocktime calculation for postcopy live migration.
vCPU blocktime - it's time since vCPU thread was put into
interruptible sleep, till memory page was copied and thread awake.

Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Signed-off-by: Alexey Perevalov <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 2a4c42f18c987496c2c48764d4785a9d6448874a
      
https://github.com/qemu/qemu/commit/2a4c42f18c987496c2c48764d4785a9d6448874a
  Author: Alexey Perevalov <address@hidden>
  Date:   2018-04-25 (Wed, 25 Apr 2018)

  Changed paths:
    M migration/migration.h
    M migration/postcopy-ram.c

  Log Message:
  -----------
  migration: add postcopy blocktime ctx into MigrationIncomingState

This patch adds request to kernel space for UFFD_FEATURE_THREAD_ID, in
case this feature is provided by kernel.

PostcopyBlocktimeContext is encapsulated inside postcopy-ram.c,
due to it being a postcopy-only feature.
Also it defines PostcopyBlocktimeContext's instance live time.
Information from PostcopyBlocktimeContext instance will be provided
much after postcopy migration end, instance of PostcopyBlocktimeContext
will live till QEMU exit, but part of it (vcpu_addr,
page_fault_vcpu_time) used only during calculation, will be released
when postcopy ended or failed.

To enable postcopy blocktime calculation on destination, need to
request proper compatibility (Patch for documentation will be at the
tail of the patch set).

As an example following command enable that capability, assume QEMU was
started with
-chardev socket,id=charmonitor,path=/var/lib/migrate-vm-monitor.sock
option to control it

address@hidden "{\"execute\" : \"qmp_capabilities\"}\r\n \
{\"execute\": \"migrate-set-capabilities\" , \"arguments\":   {
\"capabilities\": [ { \"capability\": \"postcopy-blocktime\", \"state\":
true } ] } }" | nc -U /var/lib/migrate-vm-monitor.sock

Or just with HMP
(qemu) migrate_set_capability postcopy-blocktime on

Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Signed-off-by: Alexey Perevalov <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 575b0b332ea25986fd348c45a91e642bca912137
      
https://github.com/qemu/qemu/commit/575b0b332ea25986fd348c45a91e642bca912137
  Author: Alexey Perevalov <address@hidden>
  Date:   2018-04-25 (Wed, 25 Apr 2018)

  Changed paths:
    M migration/postcopy-ram.c
    M migration/trace-events

  Log Message:
  -----------
  migration: calculate vCPU blocktime on dst side

This patch provides blocktime calculation per vCPU,
as a summary and as a overlapped value for all vCPUs.

This approach was suggested by Peter Xu, as an improvements of
previous approch where QEMU kept tree with faulted page address and cpus bitmask
in it. Now QEMU is keeping array with faulted page address as value and vCPU
as index. It helps to find proper vCPU at UFFD_COPY time. Also it keeps
list for blocktime per vCPU (could be traced with page_fault_addr)

Blocktime will not calculated if postcopy_blocktime field of
MigrationIncomingState wasn't initialized.

Signed-off-by: Alexey Perevalov <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 9ed01779e8984b71cf62e4732de8d05ff091df36
      
https://github.com/qemu/qemu/commit/9ed01779e8984b71cf62e4732de8d05ff091df36
  Author: Alexey Perevalov <address@hidden>
  Date:   2018-04-25 (Wed, 25 Apr 2018)

  Changed paths:
    M docs/devel/migration.rst

  Log Message:
  -----------
  migration: postcopy_blocktime documentation

Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Signed-off-by: Alexey Perevalov <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 346f3dab04ee2391e521fd276883e63959eb9ea9
      
https://github.com/qemu/qemu/commit/346f3dab04ee2391e521fd276883e63959eb9ea9
  Author: Alexey Perevalov <address@hidden>
  Date:   2018-04-25 (Wed, 25 Apr 2018)

  Changed paths:
    M tests/migration-test.c

  Log Message:
  -----------
  migration: add blocktime calculation into migration-test

This patch just requests blocktime calculation,
and check it in case when UFFD_FEATURE_THREAD_ID feature is set
on the host.

Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Signed-off-by: Alexey Perevalov <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 65ace060455122a461cdc9302238b914084bcd42
      
https://github.com/qemu/qemu/commit/65ace060455122a461cdc9302238b914084bcd42
  Author: Alexey Perevalov <address@hidden>
  Date:   2018-04-25 (Wed, 25 Apr 2018)

  Changed paths:
    M hmp.c
    M migration/migration.c
    M migration/migration.h
    M migration/postcopy-ram.c
    M migration/trace-events
    M qapi/migration.json

  Log Message:
  -----------
  migration: add postcopy total blocktime into query-migrate

Postcopy total blocktime is available on destination side only.
But query-migrate was possible only for source. This patch
adds ability to call query-migrate on destination.
To be able to see postcopy blocktime, need to request postcopy-blocktime
capability.

The query-migrate command will show following sample result:
{"return":
    "postcopy-vcpu-blocktime": [115, 100],
    "status": "completed",
    "postcopy-blocktime": 100
}}

postcopy_vcpu_blocktime contains list, where the first item is the first
vCPU in QEMU.

This patch has a drawback, it combines states of incoming and
outgoing migration. Ongoing migration state will overwrite incoming
state. Looks like better to separate query-migrate for incoming and
outgoing migration or add parameter to indicate type of migration.

Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Signed-off-by: Alexey Perevalov <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 263a289ae61c8344a417a95b0142650fdff3af56
      
https://github.com/qemu/qemu/commit/263a289ae61c8344a417a95b0142650fdff3af56
  Author: Xiao Guangrong <address@hidden>
  Date:   2018-04-25 (Wed, 25 Apr 2018)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration: stop compressing page in migration thread

As compression is a heavy work, do not do it in migration thread,
instead, we post it out as a normal page

Reviewed-by: Wei Wang <address@hidden>
Reviewed-by: Peter Xu <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Signed-off-by: Xiao Guangrong <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: dcaf446ebda5d87e05eb41cdbafb7ae4a7cc4a62
      
https://github.com/qemu/qemu/commit/dcaf446ebda5d87e05eb41cdbafb7ae4a7cc4a62
  Author: Xiao Guangrong <address@hidden>
  Date:   2018-04-25 (Wed, 25 Apr 2018)

  Changed paths:
    M migration/qemu-file.c
    M migration/qemu-file.h
    M migration/ram.c

  Log Message:
  -----------
  migration: stop compression to allocate and free memory frequently

Current code uses compress2() to compress memory which manages memory
internally, that causes huge memory is allocated and freed very
frequently

More worse, frequently returning memory to kernel will flush TLBs
and trigger invalidation callbacks on mmu-notification which
interacts with KVM MMU, that dramatically reduce the performance
of VM

So, we maintain the memory by ourselves and reuse it for each
compression

Reviewed-by: Peter Xu <address@hidden>
Reviewed-by: Jiang Biao <address@hidden>
Signed-off-by: Xiao Guangrong <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 797ca154b4c68dbd8e93382f714388ab311f672d
      
https://github.com/qemu/qemu/commit/797ca154b4c68dbd8e93382f714388ab311f672d
  Author: Xiao Guangrong <address@hidden>
  Date:   2018-04-25 (Wed, 25 Apr 2018)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration: stop decompression to allocate and free memory frequently

Current code uses uncompress() to decompress memory which manages
memory internally, that causes huge memory is allocated and freed
very frequently, more worse, frequently returning memory to kernel
will flush TLBs

So, we maintain the memory by ourselves and reuse it for each
decompression

Reviewed-by: Peter Xu <address@hidden>
Reviewed-by: Jiang Biao <address@hidden>
Signed-off-by: Xiao Guangrong <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 34ab9e9743aeaf265929d930747f101fa5c76fea
      
https://github.com/qemu/qemu/commit/34ab9e9743aeaf265929d930747f101fa5c76fea
  Author: Xiao Guangrong <address@hidden>
  Date:   2018-04-25 (Wed, 25 Apr 2018)

  Changed paths:
    M migration/qemu-file.c
    M migration/ram.c

  Log Message:
  -----------
  migration: detect compression and decompression errors

Currently the page being compressed is allowed to be updated by
the VM on the source QEMU, correspondingly the destination QEMU
just ignores the decompression error. However, we completely miss
the chance to catch real errors, then the VM is corrupted silently

To make the migration more robuster, we copy the page to a buffer
first to avoid it being written by VM, then detect and handle the
errors of both compression and decompression errors properly

Reviewed-by: Peter Xu <address@hidden>
Signed-off-by: Xiao Guangrong <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 059ff0fb29dd3a56ac2843676915efc279938c6b
      
https://github.com/qemu/qemu/commit/059ff0fb29dd3a56ac2843676915efc279938c6b
  Author: Xiao Guangrong <address@hidden>
  Date:   2018-04-25 (Wed, 25 Apr 2018)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration: introduce control_save_page()

Abstract the common function control_save_page() to cleanup the code,
no logic is changed

Reviewed-by: Peter Xu <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Signed-off-by: Xiao Guangrong <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 1faa5665c0f1df2eff291454a3a85625a3bc93dd
      
https://github.com/qemu/qemu/commit/1faa5665c0f1df2eff291454a3a85625a3bc93dd
  Author: Xiao Guangrong <address@hidden>
  Date:   2018-04-25 (Wed, 25 Apr 2018)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration: move some code to ram_save_host_page

Move some code from ram_save_target_page() to ram_save_host_page()
to make it be more readable for latter patches that dramatically
clean ram_save_target_page() up

Reviewed-by: Peter Xu <address@hidden>
Signed-off-by: Xiao Guangrong <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: a8ec91f941c5f83123796331c09333d3557eb5fc
      
https://github.com/qemu/qemu/commit/a8ec91f941c5f83123796331c09333d3557eb5fc
  Author: Xiao Guangrong <address@hidden>
  Date:   2018-04-25 (Wed, 25 Apr 2018)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration: move calling control_save_page to the common place

The function is called by both ram_save_page and ram_save_target_page,
so move it to the common caller to cleanup the code

Reviewed-by: Peter Xu <address@hidden>
Signed-off-by: Xiao Guangrong <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: d7400a3409982a52ac451cd3ca9caee9db670ca7
      
https://github.com/qemu/qemu/commit/d7400a3409982a52ac451cd3ca9caee9db670ca7
  Author: Xiao Guangrong <address@hidden>
  Date:   2018-04-25 (Wed, 25 Apr 2018)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration: move calling save_zero_page to the common place

save_zero_page() is always our first approach to try, move it to
the common place before calling ram_save_compressed_page
and ram_save_page

Reviewed-by: Peter Xu <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Signed-off-by: Xiao Guangrong <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 65dacaa04fa7e6104cbcee9251c7845355769a10
      
https://github.com/qemu/qemu/commit/65dacaa04fa7e6104cbcee9251c7845355769a10
  Author: Xiao Guangrong <address@hidden>
  Date:   2018-04-25 (Wed, 25 Apr 2018)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration: introduce save_normal_page()

It directly sends the page to the stream neither checking zero nor
using xbzrle or compression

Reviewed-by: Peter Xu <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Signed-off-by: Xiao Guangrong <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: da3f56cb2e767016d3f204837a77caf35b463f90
      
https://github.com/qemu/qemu/commit/da3f56cb2e767016d3f204837a77caf35b463f90
  Author: Xiao Guangrong <address@hidden>
  Date:   2018-04-25 (Wed, 25 Apr 2018)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration: remove ram_save_compressed_page()

Now, we can reuse the path in ram_save_page() to post the page out
as normal, then the only thing remained in ram_save_compressed_page()
is compression that we can move it out to the caller

Reviewed-by: Peter Xu <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Signed-off-by: Xiao Guangrong <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Dr. David Alan Gilbert <address@hidden>


  Commit: 8e383d19b4486355630c78944e4751fb953bef1c
      
https://github.com/qemu/qemu/commit/8e383d19b4486355630c78944e4751fb953bef1c
  Author: Peter Maydell <address@hidden>
  Date:   2018-04-26 (Thu, 26 Apr 2018)

  Changed paths:
    M docs/devel/migration.rst
    M hmp.c
    M migration/migration.c
    M migration/migration.h
    M migration/postcopy-ram.c
    M migration/qemu-file.c
    M migration/qemu-file.h
    M migration/ram.c
    M migration/trace-events
    M qapi/migration.json
    M tests/migration-test.c

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20180425a' 
into staging

Migration pull for 2.13

Alexey Perevalov postcopy blocktime statistics
Xiao Guangrong's compression performance improvements

# gpg: Signature made Wed 25 Apr 2018 20:21:13 BST
# gpg:                using RSA key 0516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <address@hidden>"
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A  9FA9 0516 331E BC5B FDE7

* remotes/dgilbert/tags/pull-migration-20180425a:
  migration: remove ram_save_compressed_page()
  migration: introduce save_normal_page()
  migration: move calling save_zero_page to the common place
  migration: move calling control_save_page to the common place
  migration: move some code to ram_save_host_page
  migration: introduce control_save_page()
  migration: detect compression and decompression errors
  migration: stop decompression to allocate and free memory frequently
  migration: stop compression to allocate and free memory frequently
  migration: stop compressing page in migration thread
  migration: add postcopy total blocktime into query-migrate
  migration: add blocktime calculation into migration-test
  migration: postcopy_blocktime documentation
  migration: calculate vCPU blocktime on dst side
  migration: add postcopy blocktime ctx into MigrationIncomingState
  migration: introduce postcopy-blocktime capability

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/4743c23509a5...8e383d19b448