How to modify QEMU guest ram from host side and ensure changes are visible to a KVM guest instantly

[Alan R Ford]

Hi qemu-discuss,

What are the proper functions to call, or means in QEMU source code to change 
values in guest ram memory and then have a KVM style guest see the changes 
straight away? 

Details : I am trying to modify an area of guest memory in a new ARM-32 machine 
I'm working on. Based on inspecting existing machine source code it seems the 
correct way to copy from a "source" pointer on the host to the guest memory 
address space at "guest_address" is as follows...

     address_space_write(&address_space_memory, guest_address, 
MEMTXATTRS_UNSPECIFIED, source, length);

A number of machines also use a more direct method similar to the following...

     memory_region_host_blk = memory_region_get_ram_ptr(&memory_region); /* Get 
pointer to guest ram */
     memcpy (memory_region_host_blk, source, length);   /* Copy to guest ram */

When I use "address_space_write()" a tcg style guest sees the change instantly, 
however a KVM style guest doesn't seem to see the changes straight away. It 
does see them eventually but too late for my purposes.

Is there another method that should be used to modify guest memory and have the 
changes synced straight away to a KVM guest's point of view? Is there an extra 
function that can be called after the memory is altered to accomplish this?

I can actually get things to work through a very convoluted method. It involves 
disabling then re-enabling the memory_region associated with the memory as 
follows....

      address_space_write(.......);   /* Make the change to guest memory */

      memory_region_transaction_begin();
      memory_region_set_enabled(memory_region, 0);    /* Disable region */
      memory_region_transaction_commit();
      memory_region_transaction_begin();
      memory_region_set_enabled(memory_region, 1);     /* Enable region */
      memory_region_transaction_commit();

After this weird procedure the guest sees the change with no apparent delay. 
I'm assuming this causes some kind of cache to be reset on the guest side or 
some kind of "dirty" mark is placed against the region from the guest's 
perspective. I can't imagine this procedure is very efficient. 

Background :

I thought I'd try this question on the general discussion board before 
presenting to qemu-devel because I'm admittedly not a QEMU and KVM expert and 
the answer to this might be simple despite my sincerest efforts to find an 
answer. The qemu-devel discussion board seems pretty hard core.

I am working on a new machine for ARM32. The NAND controller on the machine has 
a function whereby if you write certain values to particular I/O registers then 
an area of memory called the NAND Buffer needs to change to reflect different 
values, such as the NAND ID or the NAND status. Normally this area just 
contains data transferred from to or from the NAND. When the system first boots 
up this area is filled with the first 1MiB of NAND data (the bootloader) and it 
is therefore expected that this area of memory is executable.

The obvious answer to all of this is to use an IO style memory region for this 
special area. This way when the KVM machine queries these memory locations, 
there will be a KVM_EXIT_MMIO and the QEMU host can supply the correct values. 
The problem with this is that this area of memory must be executable. MMIO 
segments of memory are apparently not executable in QEMU.

I see that there are a lot of mechanisms to log "dirty" memory using the KVM 
api however my understanding is that this is for the benefit of the host. That 
is, the host is able to see when the guest has "dirtied" a slot of memory and 
take action. I can't really see a reverse mechanism where if the host "dirties" 
a segment of memory then the guest can be made aware and take action.

If anyone knows of other examples in QEMU source code where this kind of thing 
happens, and if you point me to them, then I'd be very appreciative.

Thanks in advance for your guidance and help. 

Berto.