Re: [PATCH v3 2/2] hw/arm/virt: Support for virtio-mem-pci

[David Hildenbrand]

On 10.01.22 11:50, Peter Maydell wrote:
> On Sat, 8 Jan 2022 at 07:22, Gavin Shan <gshan@redhat.com> wrote:
>>
>> Hi Peter,
>>
>> On 1/8/22 12:40 AM, Peter Maydell wrote:
>>> On Fri, 3 Dec 2021 at 23:34, Gavin Shan <gshan@redhat.com> wrote:
>>>> diff --git a/hw/virtio/virtio-mem.c b/hw/virtio/virtio-mem.c
>>>> index b20595a496..21e4d572ab 100644
>>>> --- a/hw/virtio/virtio-mem.c
>>>> +++ b/hw/virtio/virtio-mem.c
>>>> @@ -125,7 +125,7 @@ static uint64_t virtio_mem_default_block_size(RAMBlock 
>>>> *rb)
>>>>    * The memory block size corresponds mostly to the section size.
>>>>    *
>>>>    * This allows e.g., to add 20MB with a section size of 128MB on x86_64, 
>>>> and
>>>> - * a section size of 1GB on arm64 (as long as the start address is 
>>>> properly
>>>> + * a section size of 512MB on arm64 (as long as the start address is 
>>>> properly
>>>>    * aligned, similar to ordinary DIMMs).
>>>>    *
>>>>    * We can change this at any time and maybe even make it configurable if
>>>> @@ -134,6 +134,8 @@ static uint64_t virtio_mem_default_block_size(RAMBlock 
>>>> *rb)
>>>>    */
>>>>   #if defined(TARGET_X86_64) || defined(TARGET_I386)
>>>>   #define VIRTIO_MEM_USABLE_EXTENT (2 * (128 * MiB))
>>>> +#elif defined(TARGET_ARM)
>>>> +#define VIRTIO_MEM_USABLE_EXTENT (2 * (512 * MiB))
>>>>   #else
>>>>   #error VIRTIO_MEM_USABLE_EXTENT not defined
>>>>   #endif
>>>
>>> Could this comment explain where the 128MB and 512MB come from
>>> and why the value is different for different architectures ?
>>>
>>
>> Yes, the comment already explained it by "section size", which is the
>> minimal hotpluggable unit. It's defined by the linux guest kernel as
>> below. On ARM64, we pick the larger section size without considering
>> the base page size. Besides, the virtio-mem is/will-be enabled on
>> x86_64 and ARM64 guest kernel only.
> 
> Oh, so "section" is a Linux kernel concept? It wasn't clear to me
> that that was a fixed value rather than something we were arbitrarily
> defining in QEMU.

It's somewhat an arbitrary value, as the section size can change in the
future, and there are other memory hotplug granularity restrictions on
some architectures (e.g., x86 with boot memory size of >64GiB can
usually only hotplug in 2 GiB granularity, not 128 MiB granularity). So
what we're doing here is really best-effort for Linux guests we expect.
As the comment states, we can always change that magic value in the
future if there is need to (e.g., increase it to granularity we expect).

If our guesstimate is wrong, the guest won't be able to hotplug all
requested device memory, until we actually increase the requested size
such that it gets again possible for the VM.

We'd be running into similar issues when trying hotplug of a 128MiB DIMM
to an arm64 64k guest: Linux guests can currently only hotplug 512 MiB
granularity in such a setup and smaller DIMMs can simply not be exposed
to the page alloator and remain essentially unusable. But in contrast to
DIMMs, with virtio-mem we can actually detect that the guest cannot make
use of that memory, figure out why, and optimize.

-- 
Thanks,

David / dhildenb