[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-commits] [qemu/qemu] 1a3735: migrate/ram: remove "ram_bulk_stage"
|
From: |
Peter Maydell |
|
Subject: |
[Qemu-commits] [qemu/qemu] 1a3735: migrate/ram: remove "ram_bulk_stage" and "fpo_enab... |
|
Date: |
Fri, 14 May 2021 04:04:19 -0700 |
Branch: refs/heads/staging
Home: https://github.com/qemu/qemu
Commit: 1a37352277763220739290c689867540ec193d06
https://github.com/qemu/qemu/commit/1a37352277763220739290c689867540ec193d06
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-13 (Thu, 13 May 2021)
Changed paths:
M hw/virtio/virtio-balloon.c
M hw/virtio/virtio-mem.c
M include/migration/misc.h
M migration/ram.c
Log Message:
-----------
migrate/ram: remove "ram_bulk_stage" and "fpo_enabled"
The bulk stage is kind of weird: migration_bitmap_find_dirty() will
indicate a dirty page, however, ram_save_host_page() will never save it, as
migration_bitmap_clear_dirty() detects that it is not dirty.
We already fill the bitmap in ram_list_init_bitmaps() with ones, marking
everything dirty - it didn't used to be that way, which is why we needed
an explicit first bulk stage.
Let's simplify: make the bitmap the single source of thuth. Explicitly
handle the "xbzrle_enabled after first round" case.
Regarding XBZRLE (implicitly handled via "ram_bulk_stage = false" right
now), there is now a slight change in behavior:
- Colo: When starting, it will be disabled (was implicitly enabled)
until the first round actually finishes.
- Free page hinting: When starting, XBZRLE will be disabled (was implicitly
enabled) until the first round actually finished.
- Snapshots: When starting, XBZRLE will be disabled. We essentially only
do a single run, so I guess it will never actually get disabled.
Postcopy seems to indirectly disable it in ram_save_page(), so there
shouldn't be really any change.
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Juan Quintela <quintela@redhat.com>
Cc: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Cc: Andrey Gruzdev <andrey.gruzdev@virtuozzo.com>
Cc: Peter Xu <peterx@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210216105039.40680-1-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: 23feba906e42747463aa233fb54c58d7f02430c9
https://github.com/qemu/qemu/commit/23feba906e42747463aa233fb54c58d7f02430c9
Author: Kunkun Jiang <jiangkunkun@huawei.com>
Date: 2021-05-13 (Thu, 13 May 2021)
Changed paths:
M migration/ram.c
Log Message:
-----------
migration/ram: Reduce unnecessary rate limiting
When the host page is a huge page and something is sent in the
current iteration, migration_rate_limit() should be executed.
If not, it can be omitted.
Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
Reviewed-by: David Edmondson <david.edmondson@oracle.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20210316125716.1243-2-jiangkunkun@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: ba1b7c812c1efa49f9c297b6a22c598ff0eb5a4b
https://github.com/qemu/qemu/commit/ba1b7c812c1efa49f9c297b6a22c598ff0eb5a4b
Author: Kunkun Jiang <jiangkunkun@huawei.com>
Date: 2021-05-13 (Thu, 13 May 2021)
Changed paths:
M migration/ram.c
Log Message:
-----------
migration/ram: Optimize ram_save_host_page()
Starting from pss->page, ram_save_host_page() will check every page
and send the dirty pages up to the end of the current host page or
the boundary of used_length of the block. If the host page size is
a huge page, the step "check" will take a lot of time.
It will improve performance to use migration_bitmap_find_dirty().
Tested on Kunpeng 920; VM parameters: 1U 4G (page size 1G)
The time of ram_save_host_page() in the last round of ram saving:
before optimize: 9250us after optimize: 34us
Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20210316125716.1243-3-jiangkunkun@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: 372043f389126bf6bb4ba88b970f3dfcaf86b722
https://github.com/qemu/qemu/commit/372043f389126bf6bb4ba88b970f3dfcaf86b722
Author: Markus Armbruster <armbru@redhat.com>
Date: 2021-05-13 (Thu, 13 May 2021)
Changed paths:
M migration/migration.c
M monitor/hmp-cmds.c
M qapi/migration.json
Log Message:
-----------
migration: Drop redundant query-migrate result @blocked
Result @blocked is redundant. Unfortunately, we realized this too
close to the release to risk dropping it, so we deprecated it
instead, in commit e11ce6c06.
Since it was deprecated from the start, we can delete it without
the customary grace period. Do so.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210429140424.2802929-1-armbru@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: 082851a3af1450fa714e9a0a3ca7cb4b9dbd8855
https://github.com/qemu/qemu/commit/082851a3af1450fa714e9a0a3ca7cb4b9dbd8855
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-13 (Thu, 13 May 2021)
Changed paths:
M hw/core/numa.c
M include/exec/cpu-common.h
M softmmu/physmem.c
M util/vfio-helpers.c
Log Message:
-----------
util: vfio-helpers: Factor out and fix processing of existing ram blocks
Factor it out into common code when a new notifier is registered, just
as done with the memory region notifier. This keeps logic about how to
process existing ram blocks at a central place.
Just like when adding a new ram block, we have to register the max_length.
Ram blocks are only "fake resized". All memory (max_length) is mapped.
Print the warning from inside qemu_vfio_ram_block_added().
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210429112708.12291-2-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: 8f44304c76036ac8544468c9306c5b30d1fdd201
https://github.com/qemu/qemu/commit/8f44304c76036ac8544468c9306c5b30d1fdd201
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-13 (Thu, 13 May 2021)
Changed paths:
M hw/core/numa.c
M hw/i386/xen/xen-mapcache.c
M include/exec/ramlist.h
M softmmu/physmem.c
M target/i386/hax/hax-mem.c
M target/i386/sev.c
M util/vfio-helpers.c
Log Message:
-----------
numa: Teach ram block notifiers about resizeable ram blocks
Ram block notifiers are currently not aware of resizes. To properly
handle resizes during migration, we want to teach ram block notifiers about
resizeable ram.
Introduce the basic infrastructure but keep using max_size in the
existing notifiers. Supply the max_size when adding and removing ram
blocks. Also, notify on resizes.
Acked-by: Paul Durrant <paul@xen.org>
Reviewed-by: Peter Xu <peterx@redhat.com>
Cc: xen-devel@lists.xenproject.org
Cc: haxm-team@intel.com
Cc: Paul Durrant <paul@xen.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Anthony Perard <anthony.perard@citrix.com>
Cc: Wenchao Wang <wenchao.wang@intel.com>
Cc: Colin Xu <colin.xu@intel.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210429112708.12291-3-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: e15c7d1e8c34bbffec2aa88f8fe7cd9812b1c71f
https://github.com/qemu/qemu/commit/e15c7d1e8c34bbffec2aa88f8fe7cd9812b1c71f
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-13 (Thu, 13 May 2021)
Changed paths:
M hw/core/numa.c
Log Message:
-----------
numa: Make all callbacks of ram block notifiers optional
Let's make add/remove optional. We want to introduce a RAM block
notifier for RAM migration that is only interested in resize events.
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210429112708.12291-4-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: c7c0e72408df5e7821c0e995122fb2fe0ac001f1
https://github.com/qemu/qemu/commit/c7c0e72408df5e7821c0e995122fb2fe0ac001f1
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-13 (Thu, 13 May 2021)
Changed paths:
M include/exec/memory.h
M migration/migration.c
M migration/migration.h
M migration/ram.c
M softmmu/physmem.c
Log Message:
-----------
migration/ram: Handle RAM block resizes during precopy
Resizing while migrating is dangerous and does not work as expected.
The whole migration code works on the usable_length of ram blocks and does
not expect this to change at random points in time.
In the case of precopy, the ram block size must not change on the source,
after syncing the RAM block list in ram_save_setup(), so as long as the
guest is still running on the source.
Resizing can be trigger *after* (but not during) a reset in
ACPI code by the guest
- hw/arm/virt-acpi-build.c:acpi_ram_update()
- hw/i386/acpi-build.c:acpi_ram_update()
Use the ram block notifier to get notified about resizes. Let's simply
cancel migration and indicate the reason. We'll continue running on the
source. No harm done.
Update the documentation. Postcopy will be handled separately.
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210429112708.12291-5-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Manual merge
Commit: dcdc460767ed0a650e06ff256fa2a52ff1b57047
https://github.com/qemu/qemu/commit/dcdc460767ed0a650e06ff256fa2a52ff1b57047
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-13 (Thu, 13 May 2021)
Changed paths:
M softmmu/physmem.c
Log Message:
-----------
exec: Relax range check in ram_block_discard_range()
We want to make use of ram_block_discard_range() in the RAM block resize
callback when growing a RAM block, *before* used_length is changed.
Let's relax the check. As RAM blocks always mmap the whole max_length area,
we cannot corrupt unrelated data.
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210429112708.12291-6-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: cc61c703b6a8b9dfdfb92d5f3fa1961f6d232926
https://github.com/qemu/qemu/commit/cc61c703b6a8b9dfdfb92d5f3fa1961f6d232926
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-13 (Thu, 13 May 2021)
Changed paths:
M migration/ram.c
Log Message:
-----------
migration/ram: Discard RAM when growing RAM blocks after
ram_postcopy_incoming_init()
In case we grow our RAM after ram_postcopy_incoming_init() (e.g., when
synchronizing the RAM block state with the migration source), the resized
part would not get discarded. Let's perform that when being notified
about a resize while postcopy has been advised, but is not listening
yet. With precopy, the process is as following:
1. VM created
- RAM blocks are created
2. Incomming migration started
- Postcopy is advised
- All pages in RAM blocks are discarded
3. Precopy starts
- RAM blocks are resized to match the size on the migration source.
- RAM pages from precopy stream are loaded
- Uffd handler is registered, postcopy starts listening
4. Guest started, postcopy running
- Pagefaults get resolved, pages get placed
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210429112708.12291-7-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: 6a23f6399a1f8807447f8680f80e4d536522683e
https://github.com/qemu/qemu/commit/6a23f6399a1f8807447f8680f80e4d536522683e
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-13 (Thu, 13 May 2021)
Changed paths:
M migration/ram.c
Log Message:
-----------
migration/ram: Simplify host page handling in ram_load_postcopy()
Add two new helper functions. This will come in come handy once we want to
handle ram block resizes while postcopy is active.
Note that ram_block_from_stream() will already print proper errors.
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210429112708.12291-8-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
dgilbert: Added brackets in host_page_from_ram_block_offset
to cause uintptr_t to cast the sum, to fix armhf-cross build
Commit: 898ba906ccb85ba17d65e64a31cc13128803ef86
https://github.com/qemu/qemu/commit/898ba906ccb85ba17d65e64a31cc13128803ef86
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-13 (Thu, 13 May 2021)
Changed paths:
M include/exec/ramblock.h
M migration/postcopy-ram.c
M migration/ram.c
Log Message:
-----------
migration/ram: Handle RAM block resizes during postcopy
Resizing while migrating is dangerous and does not work as expected.
The whole migration code works with the usable_length of a ram block and
does not expect this value to change at random points in time.
In the case of postcopy, relying on used_length is racy as soon as the
guest is running. Also, when used_length changes we might leave the
uffd handler registered for some memory regions, reject valid pages
when migrating and fail when sending the recv bitmap to the source.
Resizing can be trigger *after* (but not during) a reset in
ACPI code by the guest
- hw/arm/virt-acpi-build.c:acpi_ram_update()
- hw/i386/acpi-build.c:acpi_ram_update()
Let's remember the original used_length in a separate variable and
use it in relevant postcopy code. Make sure to update it when we resize
during precopy, when synchronizing the RAM block sizes with the source.
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210429112708.12291-9-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: c1668bde5cce6c1b4dbe4b2981266cedbd111504
https://github.com/qemu/qemu/commit/c1668bde5cce6c1b4dbe4b2981266cedbd111504
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-13 (Thu, 13 May 2021)
Changed paths:
M migration/multifd.c
Log Message:
-----------
migration/multifd: Print used_length of memory block
We actually want to print the used_length, against which we check.
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210429112708.12291-10-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: 542147f4e5701fbb5b41c13d64ce90200f4ca766
https://github.com/qemu/qemu/commit/542147f4e5701fbb5b41c13d64ce90200f4ca766
Author: David Hildenbrand <david@redhat.com>
Date: 2021-05-13 (Thu, 13 May 2021)
Changed paths:
M migration/ram.c
Log Message:
-----------
migration/ram: Use offset_in_ramblock() in range checks
We never read or write beyond the used_length of memory blocks when
migrating. Make this clearer by using offset_in_ramblock() consistently.
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210429112708.12291-11-david@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: a1209bb7107c850be2909393a8304ea7ae747137
https://github.com/qemu/qemu/commit/a1209bb7107c850be2909393a8304ea7ae747137
Author: Dr. David Alan Gilbert <dgilbert@redhat.com>
Date: 2021-05-13 (Thu, 13 May 2021)
Changed paths:
M tests/qtest/migration-test.c
Log Message:
-----------
tests/migration-test: Fix "true" vs true
Accidental use of "true" as a boolean; spotted by coverity
and Peter.
Fixes: b99784ef6c3
Fixes: d795f47466e
Reported-by: Peter Maydell <peter.maydell@linaro.org>
Reported-by: Coverity (CID 1432373, 1432292, 1432288)
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20210504100545.112213-1-dgilbert@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: ff7b9b56cd7053eef0aaf40c32c2be475fd37df8
https://github.com/qemu/qemu/commit/ff7b9b56cd7053eef0aaf40c32c2be475fd37df8
Author: Peter Maydell <peter.maydell@linaro.org>
Date: 2021-05-13 (Thu, 13 May 2021)
Changed paths:
M tests/qtest/migration-test.c
Log Message:
-----------
tests/qtest/migration-test: Use g_autofree to avoid leaks on error paths
Coverity notices that several places in the migration-test code fail
to free memory in error-exit paths. This is pretty unimportant in
test case code, but we can avoid having to manually free the memory
entirely by using g_autofree.
The places where Coverity spotted a leak were relating to early exits
not freeing 'uri' in test_precopy_unix(), do_test_validate_uuid(),
migrate_postcopy_prepare() and test_migrate_auto_converge(). This
patch converts all the string-allocation in the test code to
g_autofree for consistency.
Fixes: Coverity CID 1432313, 1432315, 1432352, 1432364
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20210506185819.9010-1-peter.maydell@linaro.org>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: 1c3baa1ac4dee2b52837fda89d1d9deeb5da512e
https://github.com/qemu/qemu/commit/1c3baa1ac4dee2b52837fda89d1d9deeb5da512e
Author: Hyman <huangy81@chinatelecom.cn>
Date: 2021-05-13 (Thu, 13 May 2021)
Changed paths:
M tests/migration/guestperf/comparison.py
M tests/migration/guestperf/engine.py
M tests/migration/guestperf/scenario.py
M tests/migration/guestperf/shell.py
Log Message:
-----------
tests/migration: introduce multifd into guestperf
Guestperf tool does not cover the multifd-enabled migration
currently, it is worth supporting so that developers can
analysis the migration performance with all kinds of
migration.
To request that multifd is enabled, with 4 channels:
$ ./tests/migration/guestperf.py \
--multifd --multifd-channels 4 --output output.json
To run the entire standardized set of multifd-enabled
comparisons, with unix migration:
$ ./tests/migration/guestperf-batch.py \
--dst-host localhost --transport unix \
--filter compr-multifd* --output outputdir
Signed-off-by: Hyman Huang(黄勇) <huangy81@chinatelecom.cn>
Message-Id:
<cfeeb04d17ad932c42a9871294058b77429ad1b7.1616171924.git.huangy81@chinatelecom.cn>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit: 96662996eda78c48aadddd4e76d8615c7eb72d80
https://github.com/qemu/qemu/commit/96662996eda78c48aadddd4e76d8615c7eb72d80
Author: Peter Maydell <peter.maydell@linaro.org>
Date: 2021-05-14 (Fri, 14 May 2021)
Changed paths:
M hw/core/numa.c
M hw/i386/xen/xen-mapcache.c
M hw/virtio/virtio-balloon.c
M hw/virtio/virtio-mem.c
M include/exec/cpu-common.h
M include/exec/memory.h
M include/exec/ramblock.h
M include/exec/ramlist.h
M include/migration/misc.h
M migration/migration.c
M migration/migration.h
M migration/multifd.c
M migration/postcopy-ram.c
M migration/ram.c
M monitor/hmp-cmds.c
M qapi/migration.json
M softmmu/physmem.c
M target/i386/hax/hax-mem.c
M target/i386/sev.c
M tests/migration/guestperf/comparison.py
M tests/migration/guestperf/engine.py
M tests/migration/guestperf/scenario.py
M tests/migration/guestperf/shell.py
M tests/qtest/migration-test.c
M util/vfio-helpers.c
Log Message:
-----------
Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20210513a'
into staging
Migration pull 2021-05-13
Fix of the 2021-05-11 version, with a fix to build on the armhf
cross.
The largest change in this set is David's changes for ram block size
changing; then there's a pile of other cleanups and fixes.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
# gpg: Signature made Thu 13 May 2021 18:36:06 BST
# gpg: using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>"
[full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A 9FA9 0516 331E BC5B FDE7
* remotes/dgilbert/tags/pull-migration-20210513a:
tests/migration: introduce multifd into guestperf
tests/qtest/migration-test: Use g_autofree to avoid leaks on error paths
tests/migration-test: Fix "true" vs true
migration/ram: Use offset_in_ramblock() in range checks
migration/multifd: Print used_length of memory block
migration/ram: Handle RAM block resizes during postcopy
migration/ram: Simplify host page handling in ram_load_postcopy()
migration/ram: Discard RAM when growing RAM blocks after
ram_postcopy_incoming_init()
exec: Relax range check in ram_block_discard_range()
migration/ram: Handle RAM block resizes during precopy
numa: Make all callbacks of ram block notifiers optional
numa: Teach ram block notifiers about resizeable ram blocks
util: vfio-helpers: Factor out and fix processing of existing ram blocks
migration: Drop redundant query-migrate result @blocked
migration/ram: Optimize ram_save_host_page()
migration/ram: Reduce unnecessary rate limiting
migrate/ram: remove "ram_bulk_stage" and "fpo_enabled"
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Compare: https://github.com/qemu/qemu/compare/2d3fc4e2b069...96662996eda7
- [Qemu-commits] [qemu/qemu] 1a3735: migrate/ram: remove "ram_bulk_stage" and "fpo_enab...,
Peter Maydell <=