Re: [PATCH v14 00/14] Support blob memory and venus on qemu

[Alex Bennée]

Dmitry Osipenko <dmitry.osipenko@collabora.com> writes:

> On 6/21/24 11:59, Alex Bennée wrote:
>> Dmitry Osipenko <dmitry.osipenko@collabora.com> writes:
>> 
>>> On 6/19/24 20:37, Alex Bennée wrote:
>>>> So I've been experimenting with Aarch64 TCG with an Intel backend like
>>>> this:
>>>>
>>>> ./qemu-system-aarch64 \
>>>>            -M virt -cpu cortex-a76 \
>>>>            -device virtio-net-pci,netdev=unet \
>>>>            -netdev user,id=unet,hostfwd=tcp::2222-:22 \
>>>>            -m 8192 \
>>>>            -object memory-backend-memfd,id=mem,size=8G,share=on \
>>>>            -serial mon:stdio \
>>>>            -kernel 
>>>> ~/lsrc/linux.git/builds/arm64.initramfs/arch/arm64/boot/Image \
>>>>            -append "console=ttyAMA0" \
>>>>            -device qemu-xhci -device usb-kbd -device usb-tablet \
>>>>            -device virtio-gpu-gl-pci,blob=true,venus=true,hostmem=4G \
>>>>            -display sdl,gl=on -d 
>>>> plugin,guest_errors,trace:virtio_gpu_cmd_res_create_blob,trace:virtio_gpu_cmd_res_back_\*,trace:virtio_gpu_cmd_res_xfer_toh_3d,trace:virtio_gpu_cmd_res_xfer_fromh_3d,trace:address_space_map
>>>>  
>>>>
>>>> And I've noticed a couple of things. First trying to launch vkmark to
>>>> run a KMS mode test fails with:
>>>>
>>> ...
>>>>   virgl_render_server[1875931]: vkr: failed to import resource: invalid 
>>>> res_id 5
>>>>   virgl_render_server[1875931]: vkr: vkAllocateMemory resulted in CS error 
>>>>   virgl_render_server[1875931]: vkr: ring_submit_cmd: vn_dispatch_command 
>>>> failed
>>>>
>>>> More interestingly when shutting stuff down we see weirdness like:
>>>>
>>>>   address_space_map as:0x561b48ec48c0 addr 0x1008ac4b0:18 write:1 
>>>> attrs:0x1                                                                  
>>>>                                   
>>>>   virgl_render_server[1875931]: vkr: destroying context 3 (vkmark) with a 
>>>> valid instance                                                             
>>>>                           
>>>>   virgl_render_server[1875931]: vkr: destroying device with valid objects  
>>>>                                                                            
>>>>                          
>>>>   vkr_context_remove_object: -7438602987017907480                          
>>>>                                                                            
>>>>                          
>>>>   vkr_context_remove_object: 7                                             
>>>>                                                                            
>>>>                          
>>>>   vkr_context_remove_object: 5       
>>>>
>>>> which indicates something has gone very wrong. I'm not super familiar
>>>> with the memory allocation patterns but should stuff that is done as
>>>> virtio_gpu_cmd_res_back_attach() be find-able in the list of resources?
>>>
>>> This is expected to fail. Vkmark creates shmem virgl GBM FB BO on guest
>>> that isn't exportable on host. AFAICT, more code changes should be
>>> needed to support this case.
>> 
>> There are a lot of acronyms there. If this is pure guest memory why
>> isn't it exportable to the host? Or should the underlying mesa library
>> be making sure the allocation happens from the shared region?
>> 
>> Is vkmark particularly special here?
>
> Actually, you could get it to work to a some degree if you'll compile
> virglrenderer with -Dminigbm_allocation=true. On host use GTK/Wayland
> display.

I'll give that a go.

> Vkmark isn't special. It's virglrenderer that has a room for
> improvement. ChromeOS doesn't use KMS in VMs, proper KMS support was
> never a priority for Venus.

Is there a tracking bug for KMS support for Venus? Or Venus should work
fine if virglrenderer can export the buffer to the host?

<snip>
>>>>
>>>> This could be a false positive or it could be a race between the guest
>>>> kernel clearing memory while we are still doing
>>>> virtio_gpu_ctrl_response.
>>>>
>>>> What do you think?
>>>
>>> The memcpy warning looks a bit suspicion, but likely is harmless. I
>>> don't see such warning with TSAN and x86 VM.
>> 
>> TSAN can only pick up these interactions with TCG guests because it can
>> track guest memory accesses. With a KVM guest we have no visibility of
>> the guest accesses. 
>
> I couldn't reproduce this issue with my KVM/TCG/ARM64 setups. Fox x86 I
> checked both KVM and TCG, TSAN only warns about vitio-net memcpy's for
> me.

Hmm OK. I'll keep an eye out as I test the next version.

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro