Re: Why does the vmovdqu works for passthrough device but crashes for emulated device with "illegal operand" error (in x86_64 QEMU, -accel = kvm) ?

[Jim Mattson]

On Mon, Mar 4, 2024 at 6:11 PM Xu Liu <liuxu@meta.com> wrote:
>
> Hey Alex and Paolo,
>
> I saw there is some code related to AVX  
> https://elixir.bootlin.com/linux/latest/source/arch/x86/kvm/emulate.c#L668
>
> Does that mean in some special cases, kvm supports AVX instructions ?
> I didn’t really know the big picture, so just guess what it is doing .

The Avx bit was added in commit 1c11b37669a5 ("KVM: x86 emulator: add
support for vector alignment"). It is not used.

> Thanks,
> Xu
>
> > On Mar 4, 2024, at 6:39 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> >
> > !-------------------------------------------------------------------|
> > This Message Is From an External Sender
> >
> > |-------------------------------------------------------------------!
> >
> > On 3/4/24 22:59, Alex Williamson wrote:
> >> Since you're not seeing a KVM_EXIT_MMIO I'd guess this is more of a KVM
> >> issue than QEMU (Cc kvm list).  Possibly KVM doesn't emulate vmovdqu
> >> relative to an MMIO access, but honestly I'm not positive that AVX
> >> instructions are meant to work on MMIO space.  I'll let x86 KVM experts
> >> more familiar with specific opcode semantics weigh in on that.
> >
> > Indeed, KVM's instruction emulator supports some SSE MOV instructions but 
> > not the corresponding AVX instructions.
> >
> > Vector instructions however do work on MMIO space, and they are used 
> > occasionally especially in combination with write-combining memory.  SSE 
> > support was added to KVM because some operating systems used SSE 
> > instructions to read and write to VRAM.  However, so far we've never 
> > received any reports of OSes using AVX instructions on devices that QEMU 
> > can emulate (as opposed to, for example, GPU VRAM that is passed through).
> >
> > Thanks,
> >
> > Paolo
> >
> >> Is your "program" just doing a memcpy() with an mmap() of the PCI BAR
> >> acquired through pci-sysfs or a userspace vfio-pci driver within the
> >> guest?
> >> In QEMU 4a2e242bbb30 ("memory: Don't use memcpy for ram_device
> >> regions") we resolved an issue[1] where QEMU itself was doing a memcpy()
> >> to assigned device MMIO space resulting in breaking functionality of
> >> the device.  IIRC memcpy() was using an SSE instruction that didn't
> >> fault, but didn't work correctly relative to MMIO space either.
> >> So I also wouldn't rule out that the program isn't inherently
> >> misbehaving by using memcpy() and thereby ignoring the nature of the
> >> device MMIO access semantics.  Thanks,
> >> Alex
> >> [1]https://bugs.launchpad.net/qemu/+bug/1384892
> >
>