[Qemu-devel] [PATCH v3 for-qmp-1.6 0/4] dump-guest-memory: correct the vmcores

[Laszlo Ersek]

v2->v3 [Paolo]:
- The memory API guarantees an increasing traversal. Drop the hash
  tables and build the final list directly in the region_add callback,
  merging any adjacent region only with the last element on the list.
  (Patch #3.)

  When dumping a 3700 MB guest, the following messages were logged by
  the debug fprintf() (which is disabled by default):

  guest_phys_blocks_region_add: target_start=0000000000000000 
target_end=00000000000a0000: added (count: 1)
  guest_phys_blocks_region_add: target_start=00000000000c0000 
target_end=00000000000ca000: added (count: 2)
  guest_phys_blocks_region_add: target_start=00000000000ca000 
target_end=00000000000cd000: joined (count: 2)
  guest_phys_blocks_region_add: target_start=00000000000cd000 
target_end=00000000000d0000: joined (count: 2)
  guest_phys_blocks_region_add: target_start=00000000000d0000 
target_end=00000000000f0000: joined (count: 2)
  guest_phys_blocks_region_add: target_start=00000000000f0000 
target_end=0000000000100000: joined (count: 2)
  guest_phys_blocks_region_add: target_start=0000000000100000 
target_end=00000000e0000000: joined (count: 2)
  guest_phys_blocks_region_add: target_start=00000000fc000000 
target_end=00000000fc800000: added (count: 3)
  guest_phys_blocks_region_add: target_start=00000000fffe0000 
target_end=0000000100000000: added (count: 4)
  guest_phys_blocks_region_add: target_start=0000000100000000 
target_end=0000000107400000: added (count: 5)

  The vmcore (verified with "crash") had the following PT_LOAD entries:

  Offset     VirtAddr           PhysAddr           FileSiz    MemSiz
  ---------- ------------------ ------------------ ---------- ----------
    0x0004b8 0x0000000000000000 0x0000000000000000   0x0a0000   0x0a0000
    0x0a04b8 0x0000000000000000 0x00000000000c0000 0xdff40000 0xdff40000
  0xdffe04b8 0x0000000000000000 0x00000000fc000000   0x800000   0x800000
  0xe07e04b8 0x0000000000000000 0x00000000fffe0000   0x020000   0x020000
  0xe08004b8 0x0000000000000000 0x0000000100000000  0x7400000  0x7400000

v1->v2 [Luiz]:
- fix up the cpu_get_dump_info() prototype in "target-s390x/arch_dump.c"
  that has been introduced between my posting of v1 and Luiz's applying
  it to qmp-1.6. (Patch #4.)

v1 blurb:

Conceptually, the dump-guest-memory command works as follows:
(a) pause the guest,
(b) get a snapshot of the guest's physical memory map, as provided by
    qemu,
(c) retrieve the guest's virtual mappings, as seen by the guest (this is
    where paging=true vs. paging=false makes a difference),
(d) filter (c) as requested by the QMP caller,
(e) write ELF headers, keying off (b) -- the guest's physmap -- and (d)
    -- the filtered guest mappings.
(f) dump RAM contents, keying off the same (b) and (d),
(g) unpause the guest (if necessary).

Patch #1 affects step (e); specifically, how (d) is matched against (b),
when "paging" is "true", and the guest kernel maps more guest-physical
RAM than it actually has.

This can be done by non-malicious, clean-state guests (eg. a pristine
RHEL-6.4 guest), and may cause libbfd errors due to PT_LOAD entries
(coming directly from the guest page tables) exceeding the vmcore file's
size.

Patches #2 to #4 are independent of the "paging" option (or, more
precisely, affect them equally); they affect (b). Currently input
parameter (b), that is, the guest's physical memory map as provided by
qemu, is implicitly represented by "ram_list.blocks". As a result, steps
and outputs dependent on (b) will refer to qemu-internal offsets.

Unfortunately, this breaks when the guest-visible physical addresses
diverge from the qemu-internal, RAMBlock based representation. This can
happen eg. for guests > 3.5 GB, due to the 32-bit PCI hole; see patch #4
for a diagram.

Patch #2 introduces input parameter (b) explicitly, as a reasonably
minimal map of guest-physical address ranges. (Minimality is not a hard
requirement here, it just decreases the number of PT_LOAD entries
written to the vmcore header.) Patch #3 populates this map. Patch #4
rebases the dump-guest-memory command to it, so that steps (e) and (f)
work with guest-phys addresses.

As a result, the "crash" utility can parse vmcores dumped for big x86_64
guests (paging=false).

Please refer to Red Hat Bugzilla 981582
<https://bugzilla.redhat.com/show_bug.cgi?id=981582>.

Laszlo Ersek (4):
  dump: clamp guest-provided mapping lengths to ramblock sizes
  dump: introduce GuestPhysBlockList
  dump: populate guest_phys_blocks
  dump: rebase from host-private RAMBlock offsets to guest-physical
    addresses

 include/sysemu/dump.h           |    4 +-
 include/sysemu/memory_mapping.h |   30 +++++++-
 dump.c                          |  171 ++++++++++++++++++++++-----------------
 memory_mapping.c                |  123 ++++++++++++++++++++++++++--
 stubs/dump.c                    |    3 +-
 target-i386/arch_dump.c         |   10 ++-
 target-s390x/arch_dump.c        |    3 +-
 7 files changed, 251 insertions(+), 93 deletions(-)