[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-commits] [qemu/qemu] 870352: migrate/ram: remove "ram_bulk_stage"
|
From: |
Peter Maydell |
|
Subject: |
[Qemu-commits] [qemu/qemu] 870352: migrate/ram: remove "ram_bulk_stage" and "fpo_enab... |
|
Date: |
Wed, 12 May 2021 11:53:58 -0700 |
Branch: refs/heads/staging
Home: https://github.com/qemu/qemu
Commit: 870352753fe5d1d74e0fd4e8fb0b0880559942c2
https://github.com/qemu/qemu/commit/870352753fe5d1d74e0fd4e8fb0b0880559942c2
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-11 (Tue, 11 May 2021)
Changed paths:
M hw/virtio/virtio-balloon.c
M hw/virtio/virtio-mem.c
M include/migration/misc.h
M migration/ram.c
Log Message:
-----------
migrate/ram: remove "ram_bulk_stage" and "fpo_enabled"
The bulk stage is kind of weird: migration_bitmap_find_dirty() will
indicate a dirty page, however, ram_save_host_page() will never save it, as
migration_bitmap_clear_dirty() detects that it is not dirty.
We already fill the bitmap in ram_list_init_bitmaps() with ones, marking
everything dirty - it didn't used to be that way, which is why we needed
an explicit first bulk stage.
Let's simplify: make the bitmap the single source of thuth. Explicitly
handle the "xbzrle_enabled after first round" case.
Regarding XBZRLE (implicitly handled via "ram_bulk_stage = false" right
now), there is now a slight change in behavior:
- Colo: When starting, it will be disabled (was implicitly enabled)
until the first round actually finishes.
- Free page hinting: When starting, XBZRLE will be disabled (was implicitly
enabled) until the first round actually finished.
- Snapshots: When starting, XBZRLE will be disabled. We essentially only
do a single run, so I guess it will never actually get disabled.
Postcopy seems to indirectly disable it in ram_save_page(), so there
shouldn't be really any change.
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Juan Quintela <quintela@redhat.com>
Cc: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Cc: Andrey Gruzdev <andrey.gruzdev@virtuozzo.com>
Cc: Peter Xu <peterx@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210216105039.40680-1-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: f647b1d9bc15563d115a5a9a8d1fd82573e075b8
https://github.com/qemu/qemu/commit/f647b1d9bc15563d115a5a9a8d1fd82573e075b8
Author: Kunkun Jiang <jiangkunkun@huawei.com>
Date: 2021-05-11 (Tue, 11 May 2021)
Changed paths:
M migration/ram.c
Log Message:
-----------
migration/ram: Reduce unnecessary rate limiting
When the host page is a huge page and something is sent in the
current iteration, migration_rate_limit() should be executed.
If not, it can be omitted.
Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
Reviewed-by: David Edmondson <david.edmondson@oracle.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20210316125716.1243-2-jiangkunkun@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: 6db0279588df653db5a88a51f8be154046df57c1
https://github.com/qemu/qemu/commit/6db0279588df653db5a88a51f8be154046df57c1
Author: Kunkun Jiang <jiangkunkun@huawei.com>
Date: 2021-05-11 (Tue, 11 May 2021)
Changed paths:
M migration/ram.c
Log Message:
-----------
migration/ram: Optimize ram_save_host_page()
Starting from pss->page, ram_save_host_page() will check every page
and send the dirty pages up to the end of the current host page or
the boundary of used_length of the block. If the host page size is
a huge page, the step "check" will take a lot of time.
It will improve performance to use migration_bitmap_find_dirty().
Tested on Kunpeng 920; VM parameters: 1U 4G (page size 1G)
The time of ram_save_host_page() in the last round of ram saving:
before optimize: 9250us after optimize: 34us
Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20210316125716.1243-3-jiangkunkun@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: 98bb98839d5390be423f55daf9934d7b2eb3d4d2
https://github.com/qemu/qemu/commit/98bb98839d5390be423f55daf9934d7b2eb3d4d2
Author: Markus Armbruster <armbru@redhat.com>
Date: 2021-05-11 (Tue, 11 May 2021)
Changed paths:
M migration/migration.c
M monitor/hmp-cmds.c
M qapi/migration.json
Log Message:
-----------
migration: Drop redundant query-migrate result @blocked
Result @blocked is redundant. Unfortunately, we realized this too
close to the release to risk dropping it, so we deprecated it
instead, in commit e11ce6c06.
Since it was deprecated from the start, we can delete it without
the customary grace period. Do so.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210429140424.2802929-1-armbru@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: b8f886aeb6bec567767ad01b371eb4761be757cf
https://github.com/qemu/qemu/commit/b8f886aeb6bec567767ad01b371eb4761be757cf
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-11 (Tue, 11 May 2021)
Changed paths:
M hw/core/numa.c
M include/exec/cpu-common.h
M softmmu/physmem.c
M util/vfio-helpers.c
Log Message:
-----------
util: vfio-helpers: Factor out and fix processing of existing ram blocks
Factor it out into common code when a new notifier is registered, just
as done with the memory region notifier. This keeps logic about how to
process existing ram blocks at a central place.
Just like when adding a new ram block, we have to register the max_length.
Ram blocks are only "fake resized". All memory (max_length) is mapped.
Print the warning from inside qemu_vfio_ram_block_added().
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210429112708.12291-2-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: a7a9324889da2ac2ebeef61208ebd02166d36f66
https://github.com/qemu/qemu/commit/a7a9324889da2ac2ebeef61208ebd02166d36f66
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-11 (Tue, 11 May 2021)
Changed paths:
M hw/core/numa.c
M hw/i386/xen/xen-mapcache.c
M include/exec/ramlist.h
M softmmu/physmem.c
M target/i386/hax/hax-mem.c
M target/i386/sev.c
M util/vfio-helpers.c
Log Message:
-----------
numa: Teach ram block notifiers about resizeable ram blocks
Ram block notifiers are currently not aware of resizes. To properly
handle resizes during migration, we want to teach ram block notifiers about
resizeable ram.
Introduce the basic infrastructure but keep using max_size in the
existing notifiers. Supply the max_size when adding and removing ram
blocks. Also, notify on resizes.
Acked-by: Paul Durrant <paul@xen.org>
Reviewed-by: Peter Xu <peterx@redhat.com>
Cc: xen-devel@lists.xenproject.org
Cc: haxm-team@intel.com
Cc: Paul Durrant <paul@xen.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Anthony Perard <anthony.perard@citrix.com>
Cc: Wenchao Wang <wenchao.wang@intel.com>
Cc: Colin Xu <colin.xu@intel.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210429112708.12291-3-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: e3b6c38a4d04acc74dded59f8dd69c60460db7fc
https://github.com/qemu/qemu/commit/e3b6c38a4d04acc74dded59f8dd69c60460db7fc
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-11 (Tue, 11 May 2021)
Changed paths:
M hw/core/numa.c
Log Message:
-----------
numa: Make all callbacks of ram block notifiers optional
Let's make add/remove optional. We want to introduce a RAM block
notifier for RAM migration that is only interested in resize events.
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210429112708.12291-4-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: 3e53e5cf0ae32c45442d1568a5fc49e926841c44
https://github.com/qemu/qemu/commit/3e53e5cf0ae32c45442d1568a5fc49e926841c44
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-11 (Tue, 11 May 2021)
Changed paths:
M include/exec/memory.h
M migration/migration.c
M migration/migration.h
M migration/ram.c
M softmmu/physmem.c
Log Message:
-----------
migration/ram: Handle RAM block resizes during precopy
Resizing while migrating is dangerous and does not work as expected.
The whole migration code works on the usable_length of ram blocks and does
not expect this to change at random points in time.
In the case of precopy, the ram block size must not change on the source,
after syncing the RAM block list in ram_save_setup(), so as long as the
guest is still running on the source.
Resizing can be trigger *after* (but not during) a reset in
ACPI code by the guest
- hw/arm/virt-acpi-build.c:acpi_ram_update()
- hw/i386/acpi-build.c:acpi_ram_update()
Use the ram block notifier to get notified about resizes. Let's simply
cancel migration and indicate the reason. We'll continue running on the
source. No harm done.
Update the documentation. Postcopy will be handled separately.
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210429112708.12291-5-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Manual merge
Commit: d634436e38b4f5b4c72131746b6bc8f051fd626c
https://github.com/qemu/qemu/commit/d634436e38b4f5b4c72131746b6bc8f051fd626c
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-11 (Tue, 11 May 2021)
Changed paths:
M softmmu/physmem.c
Log Message:
-----------
exec: Relax range check in ram_block_discard_range()
We want to make use of ram_block_discard_range() in the RAM block resize
callback when growing a RAM block, *before* used_length is changed.
Let's relax the check. As RAM blocks always mmap the whole max_length area,
we cannot corrupt unrelated data.
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210429112708.12291-6-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: eb1dafa01c5c59f23c02cae384e6e3bc4e7ff189
https://github.com/qemu/qemu/commit/eb1dafa01c5c59f23c02cae384e6e3bc4e7ff189
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-11 (Tue, 11 May 2021)
Changed paths:
M migration/ram.c
Log Message:
-----------
migration/ram: Discard RAM when growing RAM blocks after
ram_postcopy_incoming_init()
In case we grow our RAM after ram_postcopy_incoming_init() (e.g., when
synchronizing the RAM block state with the migration source), the resized
part would not get discarded. Let's perform that when being notified
about a resize while postcopy has been advised, but is not listening
yet. With precopy, the process is as following:
1. VM created
- RAM blocks are created
2. Incomming migration started
- Postcopy is advised
- All pages in RAM blocks are discarded
3. Precopy starts
- RAM blocks are resized to match the size on the migration source.
- RAM pages from precopy stream are loaded
- Uffd handler is registered, postcopy starts listening
4. Guest started, postcopy running
- Pagefaults get resolved, pages get placed
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210429112708.12291-7-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: a988faaf83b303da299bf9fe06d62c5031d9b1d1
https://github.com/qemu/qemu/commit/a988faaf83b303da299bf9fe06d62c5031d9b1d1
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-11 (Tue, 11 May 2021)
Changed paths:
M migration/ram.c
Log Message:
-----------
migration/ram: Simplify host page handling in ram_load_postcopy()
Add two new helper functions. This will come in come handy once we want to
handle ram block resizes while postcopy is active.
Note that ram_block_from_stream() will already print proper errors.
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210429112708.12291-8-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: c73712e64638c1221af749d5e580110f09761757
https://github.com/qemu/qemu/commit/c73712e64638c1221af749d5e580110f09761757
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-11 (Tue, 11 May 2021)
Changed paths:
M include/exec/ramblock.h
M migration/postcopy-ram.c
M migration/ram.c
Log Message:
-----------
migration/ram: Handle RAM block resizes during postcopy
Resizing while migrating is dangerous and does not work as expected.
The whole migration code works with the usable_length of a ram block and
does not expect this value to change at random points in time.
In the case of postcopy, relying on used_length is racy as soon as the
guest is running. Also, when used_length changes we might leave the
uffd handler registered for some memory regions, reject valid pages
when migrating and fail when sending the recv bitmap to the source.
Resizing can be trigger *after* (but not during) a reset in
ACPI code by the guest
- hw/arm/virt-acpi-build.c:acpi_ram_update()
- hw/i386/acpi-build.c:acpi_ram_update()
Let's remember the original used_length in a separate variable and
use it in relevant postcopy code. Make sure to update it when we resize
during precopy, when synchronizing the RAM block sizes with the source.
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210429112708.12291-9-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: 70565bf9551f9a6b23f9948f0504e051946f9700
https://github.com/qemu/qemu/commit/70565bf9551f9a6b23f9948f0504e051946f9700
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-11 (Tue, 11 May 2021)
Changed paths:
M migration/multifd.c
Log Message:
-----------
migration/multifd: Print used_length of memory block
We actually want to print the used_length, against which we check.
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210429112708.12291-10-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: df19fe4b57ba046716b637de34805893f9eb4e81
https://github.com/qemu/qemu/commit/df19fe4b57ba046716b637de34805893f9eb4e81
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-11 (Tue, 11 May 2021)
Changed paths:
M migration/ram.c
Log Message:
-----------
migration/ram: Use offset_in_ramblock() in range checks
We never read or write beyond the used_length of memory blocks when
migrating. Make this clearer by using offset_in_ramblock() consistently.
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210429112708.12291-11-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: 6a9011cdcf95dfbf82ff1ec2e62934b66af5eab2
https://github.com/qemu/qemu/commit/6a9011cdcf95dfbf82ff1ec2e62934b66af5eab2
Author: Dr. David Alan Gilbert <dgilbert@redhat.com>
Date: 2021-05-11 (Tue, 11 May 2021)
Changed paths:
M tests/qtest/migration-test.c
Log Message:
-----------
tests/migration-test: Fix "true" vs true
Accidental use of "true" as a boolean; spotted by coverity
and Peter.
Fixes: b99784ef6c3
Fixes: d795f47466e
Reported-by: Peter Maydell <peter.maydell@linaro.org>
Reported-by: Coverity (CID 1432373, 1432292, 1432288)
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20210504100545.112213-1-dgilbert@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: 457ad7d1daa0193ba8761a805d1bdd33897bae18
https://github.com/qemu/qemu/commit/457ad7d1daa0193ba8761a805d1bdd33897bae18
Author: Peter Maydell <peter.maydell@linaro.org>
Date: 2021-05-11 (Tue, 11 May 2021)
Changed paths:
M tests/qtest/migration-test.c
Log Message:
-----------
tests/qtest/migration-test: Use g_autofree to avoid leaks on error paths
Coverity notices that several places in the migration-test code fail
to free memory in error-exit paths. This is pretty unimportant in
test case code, but we can avoid having to manually free the memory
entirely by using g_autofree.
The places where Coverity spotted a leak were relating to early exits
not freeing 'uri' in test_precopy_unix(), do_test_validate_uuid(),
migrate_postcopy_prepare() and test_migrate_auto_converge(). This
patch converts all the string-allocation in the test code to
g_autofree for consistency.
Fixes: Coverity CID 1432313, 1432315, 1432352, 1432364
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20210506185819.9010-1-peter.maydell@linaro.org>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: 872df23afc70985af5a458e0c4bd2b984559015c
https://github.com/qemu/qemu/commit/872df23afc70985af5a458e0c4bd2b984559015c
Author: Hyman <huangy81@chinatelecom.cn>
Date: 2021-05-11 (Tue, 11 May 2021)
Changed paths:
M tests/migration/guestperf/comparison.py
M tests/migration/guestperf/engine.py
M tests/migration/guestperf/scenario.py
M tests/migration/guestperf/shell.py
Log Message:
-----------
tests/migration: introduce multifd into guestperf
Guestperf tool does not cover the multifd-enabled migration
currently, it is worth supporting so that developers can
analysis the migration performance with all kinds of
migration.
To request that multifd is enabled, with 4 channels:
$ ./tests/migration/guestperf.py \
--multifd --multifd-channels 4 --output output.json
To run the entire standardized set of multifd-enabled
comparisons, with unix migration:
$ ./tests/migration/guestperf-batch.py \
--dst-host localhost --transport unix \
--filter compr-multifd* --output outputdir
Signed-off-by: Hyman Huang(黄勇) <huangy81@chinatelecom.cn>
Message-Id:
<cfeeb04d17ad932c42a9871294058b77429ad1b7.1616171924.git.huangy81@chinatelecom.cn>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: 07b0a39c09092a6f7e72b1a0381b65418755c5d8
https://github.com/qemu/qemu/commit/07b0a39c09092a6f7e72b1a0381b65418755c5d8
Author: Peter Maydell <peter.maydell@linaro.org>
Date: 2021-05-12 (Wed, 12 May 2021)
Changed paths:
M hw/core/numa.c
M hw/i386/xen/xen-mapcache.c
M hw/virtio/virtio-balloon.c
M hw/virtio/virtio-mem.c
M include/exec/cpu-common.h
M include/exec/memory.h
M include/exec/ramblock.h
M include/exec/ramlist.h
M include/migration/misc.h
M migration/migration.c
M migration/migration.h
M migration/multifd.c
M migration/postcopy-ram.c
M migration/ram.c
M monitor/hmp-cmds.c
M qapi/migration.json
M softmmu/physmem.c
M target/i386/hax/hax-mem.c
M target/i386/sev.c
M tests/migration/guestperf/comparison.py
M tests/migration/guestperf/engine.py
M tests/migration/guestperf/scenario.py
M tests/migration/guestperf/shell.py
M tests/qtest/migration-test.c
M util/vfio-helpers.c
Log Message:
-----------
Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20210511a'
into staging
Migration pull 2021-05-11
The largest change in this set is David's changes for ram block size
changing; then there's a pile of other cleanups and fixes.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
# gpg: Signature made Tue 11 May 2021 16:06:42 BST
# gpg: using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>"
[full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A 9FA9 0516 331E BC5B FDE7
* remotes/dgilbert/tags/pull-migration-20210511a:
tests/migration: introduce multifd into guestperf
tests/qtest/migration-test: Use g_autofree to avoid leaks on error paths
tests/migration-test: Fix "true" vs true
migration/ram: Use offset_in_ramblock() in range checks
migration/multifd: Print used_length of memory block
migration/ram: Handle RAM block resizes during postcopy
migration/ram: Simplify host page handling in ram_load_postcopy()
migration/ram: Discard RAM when growing RAM blocks after
ram_postcopy_incoming_init()
exec: Relax range check in ram_block_discard_range()
migration/ram: Handle RAM block resizes during precopy
numa: Make all callbacks of ram block notifiers optional
numa: Teach ram block notifiers about resizeable ram blocks
util: vfio-helpers: Factor out and fix processing of existing ram blocks
migration: Drop redundant query-migrate result @blocked
migration/ram: Optimize ram_save_host_page()
migration/ram: Reduce unnecessary rate limiting
migrate/ram: remove "ram_bulk_stage" and "fpo_enabled"
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Compare: https://github.com/qemu/qemu/compare/3e9f48bcdabe...07b0a39c0909
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- [Qemu-commits] [qemu/qemu] 870352: migrate/ram: remove "ram_bulk_stage" and "fpo_enab...,
Peter Maydell <=