Re: Why does the vmovdqu works for passthrough device but crashes for emulated device with "illegal operand" error (in x86_64 QEMU, -accel = kvm) ?

[Xu Liu]

Thanks Jim!

> On Mar 4, 2024, at 10:10 PM, Jim Mattson <jmattson@google.com> wrote:
> 
> !-------------------------------------------------------------------|
>  This Message Is From an External Sender
> 
> |-------------------------------------------------------------------!
> 
> On Mon, Mar 4, 2024 at 6:11 PM Xu Liu <liuxu@meta.com> wrote:
>> 
>> Hey Alex and Paolo,
>> 
>> I saw there is some code related to AVX  
>> https://elixir.bootlin.com/linux/latest/source/arch/x86/kvm/emulate.c#L668 
>> 
>> Does that mean in some special cases, kvm supports AVX instructions ?
>> I didn’t really know the big picture, so just guess what it is doing .
> 
> The Avx bit was added in commit 1c11b37669a5 ("KVM: x86 emulator: add
> support for vector alignment"). It is not used.
> 
>> Thanks,
>> Xu
>> 
>>> On Mar 4, 2024, at 6:39 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>> 
>>> !-------------------------------------------------------------------|
>>> This Message Is From an External Sender
>>> 
>>> |-------------------------------------------------------------------!
>>> 
>>> On 3/4/24 22:59, Alex Williamson wrote:
>>>> Since you're not seeing a KVM_EXIT_MMIO I'd guess this is more of a KVM
>>>> issue than QEMU (Cc kvm list).  Possibly KVM doesn't emulate vmovdqu
>>>> relative to an MMIO access, but honestly I'm not positive that AVX
>>>> instructions are meant to work on MMIO space.  I'll let x86 KVM experts
>>>> more familiar with specific opcode semantics weigh in on that.
>>> 
>>> Indeed, KVM's instruction emulator supports some SSE MOV instructions but 
>>> not the corresponding AVX instructions.
>>> 
>>> Vector instructions however do work on MMIO space, and they are used 
>>> occasionally especially in combination with write-combining memory.  SSE 
>>> support was added to KVM because some operating systems used SSE 
>>> instructions to read and write to VRAM.  However, so far we've never 
>>> received any reports of OSes using AVX instructions on devices that QEMU 
>>> can emulate (as opposed to, for example, GPU VRAM that is passed through).
>>> 
>>> Thanks,
>>> 
>>> Paolo
>>> 
>>>> Is your "program" just doing a memcpy() with an mmap() of the PCI BAR
>>>> acquired through pci-sysfs or a userspace vfio-pci driver within the
>>>> guest?
>>>> In QEMU 4a2e242bbb30 ("memory: Don't use memcpy for ram_device
>>>> regions") we resolved an issue[1] where QEMU itself was doing a memcpy()
>>>> to assigned device MMIO space resulting in breaking functionality of
>>>> the device.  IIRC memcpy() was using an SSE instruction that didn't
>>>> fault, but didn't work correctly relative to MMIO space either.
>>>> So I also wouldn't rule out that the program isn't inherently
>>>> misbehaving by using memcpy() and thereby ignoring the nature of the
>>>> device MMIO access semantics.  Thanks,
>>>> Alex
>>>> [1]https://bugs.launchpad.net/qemu/+bug/1384892
>>> 
>>