[Qemu-commits] [qemu/qemu] 263170: docs: Add a doc about multiple thread compression

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 263170e679dfb456f8812a0100073990586cecb5
      
https://github.com/qemu/qemu/commit/263170e679dfb456f8812a0100073990586cecb5
  Author: Liang Li <address@hidden>
  Date:   2015-05-06 (Wed, 06 May 2015)

  Changed paths:
    A docs/multi-thread-compression.txt

  Log Message:
  -----------
  docs: Add a doc about multiple thread compression

Give some details about the multiple thread (de)compression and
how to use it in live migration.

Signed-off-by: Liang Li <address@hidden>
Signed-off-by: Yang Zhang <address@hidden>
Reviewed-by: Dr.David Alan Gilbert <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 8706d2d566cbf4bad2c5597bb57358e3d5f5caf0
      
https://github.com/qemu/qemu/commit/8706d2d566cbf4bad2c5597bb57358e3d5f5caf0
  Author: Liang Li <address@hidden>
  Date:   2015-05-06 (Wed, 06 May 2015)

  Changed paths:
    M arch_init.c
    M include/migration/migration.h
    M migration/migration.c

  Log Message:
  -----------
  migration: Add the framework of multi-thread compression

Add the code to create and destroy the multiple threads those will
be used to do data compression. Left some functions empty to keep
clearness, and the code will be added later.

Signed-off-by: Liang Li <address@hidden>
Signed-off-by: Yang Zhang <address@hidden>
Reviewed-by: Dr.David Alan Gilbert <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 3fcb38c223510cf88c6101f5d218ce0840d1354c
      
https://github.com/qemu/qemu/commit/3fcb38c223510cf88c6101f5d218ce0840d1354c
  Author: Liang Li <address@hidden>
  Date:   2015-05-06 (Wed, 06 May 2015)

  Changed paths:
    M arch_init.c
    M include/migration/migration.h
    M migration/migration.c

  Log Message:
  -----------
  migration: Add the framework of multi-thread decompression

Add the code to create and destroy the multiple threads those will be
used to do data decompression. Left some functions empty just to keep
clearness, and the code will be added later.

Signed-off-by: Liang Li <address@hidden>
Signed-off-by: Yang Zhang <address@hidden>
Reviewed-by: Dr.David Alan Gilbert <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 44f0eadc338f55d3bffe4fccefb1d4241defa418
      
https://github.com/qemu/qemu/commit/44f0eadc338f55d3bffe4fccefb1d4241defa418
  Author: Liang Li <address@hidden>
  Date:   2015-05-06 (Wed, 06 May 2015)

  Changed paths:
    M include/migration/qemu-file.h
    M migration/qemu-file.c

  Log Message:
  -----------
  qemu-file: Add compression functions to QEMUFile

qemu_put_compression_data() compress the data and put it to QEMUFile.
qemu_put_qemu_file() put the data in the buffer of source QEMUFile to
destination QEMUFile.

Signed-off-by: Liang Li <address@hidden>
Signed-off-by: Yang Zhang <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 474ddaf6e3aebc470f4665ef4f7ce6578448c6d1
      
https://github.com/qemu/qemu/commit/474ddaf6e3aebc470f4665ef4f7ce6578448c6d1
  Author: Liang Li <address@hidden>
  Date:   2015-05-06 (Wed, 06 May 2015)

  Changed paths:
    M arch_init.c

  Log Message:
  -----------
  arch_init: Alloc and free data struct for compression

Define the data structure and variables used to do multiple thread
compression, and add the code to initialize and free them.

Signed-off-by: Liang Li <address@hidden>
Signed-off-by: Yang Zhang <address@hidden>
Reviewed-by: Dr.David Alan Gilbert <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 3caf633dbde8a347cff49e960691c7fa6a82afa1
      
https://github.com/qemu/qemu/commit/3caf633dbde8a347cff49e960691c7fa6a82afa1
  Author: Liang Li <address@hidden>
  Date:   2015-05-06 (Wed, 06 May 2015)

  Changed paths:
    M arch_init.c

  Log Message:
  -----------
  arch_init: Add and free data struct for decompression

Define the data structure and variables used to do multiple thread
decompression, and add the code to initialize and free them.

Signed-off-by: Liang Li <address@hidden>
Signed-off-by: Yang Zhang <address@hidden>
Reviewed-by: Dr.David Alan Gilbert <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: e2102428c09901788a5e585f32f9e805137f5967
      
https://github.com/qemu/qemu/commit/e2102428c09901788a5e585f32f9e805137f5967
  Author: Liang Li <address@hidden>
  Date:   2015-05-06 (Wed, 06 May 2015)

  Changed paths:
    M arch_init.c

  Log Message:
  -----------
  migration: Split save_zero_page from ram_save_page

Split the function save_zero_page from ram_save_page so that we can
reuse it later.

Signed-off-by: Liang Li <address@hidden>
Signed-off-by: Yang Zhang <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 20eb617eacf7a0ce76d9dd10ff246d6ae7f0b4e1
      
https://github.com/qemu/qemu/commit/20eb617eacf7a0ce76d9dd10ff246d6ae7f0b4e1
  Author: Liang Li <address@hidden>
  Date:   2015-05-07 (Thu, 07 May 2015)

  Changed paths:
    M arch_init.c

  Log Message:
  -----------
  migration: Add the core code of multi-thread compression

Implement the core logic of the multiple thread compression. At this
point, multiple thread compression can't co-work with xbzrle yet.

Signed-off-by: Liang Li <address@hidden>
Signed-off-by: Yang Zhang <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 98f1138902195bd9ab8a753d0ee2cf2a0a88b6e8
      
https://github.com/qemu/qemu/commit/98f1138902195bd9ab8a753d0ee2cf2a0a88b6e8
  Author: Liang Li <address@hidden>
  Date:   2015-05-07 (Thu, 07 May 2015)

  Changed paths:
    M arch_init.c

  Log Message:
  -----------
  migration: Make compression co-work with xbzrle

Now, multiple thread compression can co-work with xbzrle. when
xbzrle is on, multiple thread compression will only work at the
first round of RAM data sync.

Signed-off-by: Liang Li <address@hidden>
Signed-off-by: Yang Zhang <address@hidden>
Reviewed-by: Dr.David Alan Gilbert <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 68ae113646dc84637359472e89669e5547dc5ee3
      
https://github.com/qemu/qemu/commit/68ae113646dc84637359472e89669e5547dc5ee3
  Author: Liang Li <address@hidden>
  Date:   2015-05-07 (Thu, 07 May 2015)

  Changed paths:
    M arch_init.c

  Log Message:
  -----------
  migration: Add the core code for decompression

Implement the core logic of multiple thread decompression,
the decompression can work now.

Signed-off-by: Liang Li <address@hidden>
Signed-off-by: Yang Zhang <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: dde4e694ae576462990b2ce711e62565e085c261
      
https://github.com/qemu/qemu/commit/dde4e694ae576462990b2ce711e62565e085c261
  Author: Liang Li <address@hidden>
  Date:   2015-05-07 (Thu, 07 May 2015)

  Changed paths:
    M migration/migration.c
    M qapi-schema.json

  Log Message:
  -----------
  migration: Add interface to control compression

The multiple compression threads can be turned on/off through
qmp and hmp interface before doing live migration.

Signed-off-by: Liang Li <address@hidden>
Signed-off-by: Yang Zhang <address@hidden>
Reviewed-by: Dr.David Alan Gilbert <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 43c60a81ba15ea040709be5809a279a4ca59b26b
      
https://github.com/qemu/qemu/commit/43c60a81ba15ea040709be5809a279a4ca59b26b
  Author: Liang Li <address@hidden>
  Date:   2015-05-07 (Thu, 07 May 2015)

  Changed paths:
    M include/migration/migration.h
    M migration/migration.c
    M qapi-schema.json

  Log Message:
  -----------
  migration: Use an array instead of 3 parameters

Put the three parameters related to multiple thread (de)compression
into an int array, and use an enum type to index the parameter.

Signed-off-by: Liang Li <address@hidden>
Signed-off-by: Yang Zhang <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 85de83231ecde075c6b25897f2e74cd1767880e3
      
https://github.com/qemu/qemu/commit/85de83231ecde075c6b25897f2e74cd1767880e3
  Author: Liang Li <address@hidden>
  Date:   2015-05-07 (Thu, 07 May 2015)

  Changed paths:
    M migration/migration.c
    M qapi-schema.json
    M qmp-commands.hx

  Log Message:
  -----------
  migration: Add qmp commands to set and query parameters

Add the qmp commands to tune and query the parameters used in live
migration.

Signed-off-by: Liang Li <address@hidden>
Signed-off-by: Yang Zhang <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 50e9a629c6c92e73260cd3d7c2e3f5bfd84e47e2
      
https://github.com/qemu/qemu/commit/50e9a629c6c92e73260cd3d7c2e3f5bfd84e47e2
  Author: Liang Li <address@hidden>
  Date:   2015-05-07 (Thu, 07 May 2015)

  Changed paths:
    M hmp-commands.hx
    M hmp.c
    M hmp.h
    M monitor.c

  Log Message:
  -----------
  migration: Add hmp interface to set and query parameters

Add the hmp interface to tune and query the parameters used in
live migration.

Signed-off-by: Liang Li <address@hidden>
Signed-off-by: Yang Zhang <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 27ff42e29a1d12e9d9dbc473bbca36a50baf278b
      
https://github.com/qemu/qemu/commit/27ff42e29a1d12e9d9dbc473bbca36a50baf278b
  Author: Michael Chapman <address@hidden>
  Date:   2015-05-07 (Thu, 07 May 2015)

  Changed paths:
    M arch_init.c

  Log Message:
  -----------
  migration: avoid divide by zero in xbzrle cache miss rate

This bug manifested itself as a VM that could not be resumed by libvirt
following a migration:

  # virsh resume example
  error: Failed to resume domain example
  error: internal error: cannot parse json {"return":
    {"xbzrle-cache":
      {..., "cache-miss-rate": -nan, ...},
      ...
    }
  }: lexical error: malformed number, a digit is required after the minus sign.

This patch also ensures xbzrle_cache_miss_prev and iterations_prev are
cleared at the start of the migration.

Signed-off-by: Michael Chapman <address@hidden>
Reviewed-by: Amit Shah <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 362ba4e3ee801e8f5e28d72d0009547384222927
      
https://github.com/qemu/qemu/commit/362ba4e3ee801e8f5e28d72d0009547384222927
  Author: Liang Li <address@hidden>
  Date:   2015-05-07 (Thu, 07 May 2015)

  Changed paths:
    M arch_init.c

  Log Message:
  -----------
  migration: Fix migration state update issue

If live migration is very fast and can be completed in 1 second,
the dirty_sync_count of MigrationState will not be updated.
Then you will see "dirty sync count: 0" in qemu monitor even if
the actual dirty sync count is not 0.

Signed-off-by: Liang Li <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Reviewed-by: Dr.David Alan Gilbert <address@hidden>
Signed-off-by: Juan Quintela <address@hidden>


  Commit: 838686357b1a175e9a32569700a153b207a9e10f
      
https://github.com/qemu/qemu/commit/838686357b1a175e9a32569700a153b207a9e10f
  Author: Peter Maydell <address@hidden>
  Date:   2015-05-07 (Thu, 07 May 2015)

  Changed paths:
    M arch_init.c
    A docs/multi-thread-compression.txt
    M hmp-commands.hx
    M hmp.c
    M hmp.h
    M include/migration/migration.h
    M include/migration/qemu-file.h
    M migration/migration.c
    M migration/qemu-file.c
    M monitor.c
    M qapi-schema.json
    M qmp-commands.hx

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/juanquintela/tags/migration/20150507-1' 
into staging

migration/next for 20150507

# gpg: Signature made Thu May  7 17:42:19 2015 BST using RSA key ID 5872D723
# gpg: Good signature from "Juan Quintela <address@hidden>"
# gpg:                 aka "Juan Quintela <address@hidden>"

* remotes/juanquintela/tags/migration/20150507-1:
  migration: Fix migration state update issue
  migration: avoid divide by zero in xbzrle cache miss rate
  migration: Add hmp interface to set and query parameters
  migration: Add qmp commands to set and query parameters
  migration: Use an array instead of 3 parameters
  migration: Add interface to control compression
  migration: Add the core code for decompression
  migration: Make compression co-work with xbzrle
  migration: Add the core code of multi-thread compression
  migration: Split save_zero_page from ram_save_page
  arch_init: Add and free data struct for decompression
  arch_init: Alloc and free data struct for compression
  qemu-file: Add compression functions to QEMUFile
  migration: Add the framework of multi-thread decompression
  migration: Add the framework of multi-thread compression
  docs: Add a doc about multiple thread compression

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/38003aee196a...838686357b1a