[Qemu-devel] [PATCH 0/18 RFC v3] Localhost migration

[Lei Li]

Hi,

This patch series tries to add a new approach to localhost migration 
support in QEMU. 

When doing localhost migration, the host memory will balloon up 
during the period. This procedure might need to consume twice the 
memory needed to run the guest for some time. So we want to add 
a new live migration mechanism for localhost migration. 

Following I copied from last version that Anthony added for the 
benefit of the other reviewers:

The goal here is to allow "live upgrade" of a running QEMU instance.  
The work flow would look like this:

1) Guests are running QEMU release 1.6.1
2) Admin installs QEMU release 1.6.2 via RPM or deb
3) Admin does localhost migration with page flipping to use new version
   of QEMU.

Page flipping is used in order to avoid requiring that there is enough
free memory to fit an additional copy of the largest guest which is the
requirement today with localhost migration.

You can also read from the link below:
http://lists.gnu.org/archive/html/qemu-devel/2013-06/msg02577.html

The plan is:

1) Add new command to do localhost migration.

   The qmp interface introduced like:

   { 'command': 'localhost-migrate', 'data': {'uri': 'str'} }

2) Use different mechanism than current live migration.

   The very basic work flow like:

       qemu on the source (the source and destination are both on localhost)
              |
              V
           Stop VM
              |
              V   
          Create threads
              |
              V
       Page flipping through vmsplice
              |
              V
       MADV_DONTNEED the ram pages which are already flipped
              |
              V
       Migration completes

   As stopping VM first, we expect/resume the page flipping through vmsplice
   is fast enough to meet *live migration (low downtime).

Notes:
Currently the work flow is not exactly the same as description
above. For the first step, the work flow we implemented is:
stop VM and copy ram pages via unix domain socket, MADV_DONTNEED 
ram pages that already copied. After that, will replace to vmsplice
mechanism instead of copying pages.

Known issues:
Now it has not been completely tested, and there is one problem
on exporting MemoryRegion and RAMBlock into migration-local.c, as 
the code of overriding of rdma hooks like save_page was moved there. 
So still mark it as RFC.

To avoid wasting time and make sure it's the way should be headed, 
send it out earlier to have your suggestions.

Your suggestions and comments would be very appreciated!


Changes since v1&v2:
  - Interface changes including:
      Convert a new qmp command to a capability suggested by Anthony
      and Paolo.
      Create a new URI prefix 'local' instead of a new command-line 
      switch suggested by Michael R. Hines.

  - Integrate to migration_thread from Michael R. Hines.
  - Override rdma hooks suggested by Paolo and Michael R.  

TODO:
- Introduce a mechanism to exchange a PIPE via SCM_RIGHTS.
- benchmark/evaluation.

Lei Li (18):
  migration: export MIG_STATE_xxx flags 
  savevm: export qemu_save_device_state()
  rename is_active to is_block_active
  savevm: set right return value for qemu_file_rate_limit
  savevm: add comments for qemu_file_get_error()
  rdma: bugfix for ram_control_load_hook
  arch_init: export RAM_SAVE_xxx flags
  migration-local: introduce qemu_fopen_local()
  exec: export qemu_get_ram_block()
  migration-local: implementation of outgoing part
  migration: introduce capability localhost
  arch_init: factor out ram_save_blocks()
  arch_init: adjust ram_save_setup() for migrate_is_localhost
  arch_init: skip migration_bitmap_sync for local migration
  migration: adjust migration_thread() for local migration
  migration-local: implementation of incoming part
  migration: add prefix for local migration to incoming migration
  hmp: better fomat for info migrate_capabilities

 Makefile.objs                 |    1 +
 arch_init.c                   |   62 ++++---
 block-migration.c             |    2 +-
 exec.c                        |    7 +-
 hmp.c                         |    5 +-
 include/exec/cpu-all.h        |    1 +
 include/migration/migration.h |   37 ++++
 include/migration/qemu-file.h |   10 +-
 include/migration/vmstate.h   |    2 +-
 include/sysemu/sysemu.h       |    1 +
 migration-local.c             |  397 +++++++++++++++++++++++++++++++++++++++++
 migration.c                   |  107 ++++++----
 qapi-schema.json              |    8 +-
 savevm.c                      |   47 ++++--
 14 files changed, 688 insertions(+), 89 deletions(-)
 create mode 100644 migration-local.c