How do I make my emulated device's DMA go through viommu ?

[Xu Liu]

Hello, All

I am working a project, which has a custom emulated PCI device for the QEMU.

In side the emulated device,  I call pci_dma_read/write to write to some IOVA address, for example  0X400000030000.

After start the QEMU, I bind the emulated device to vfio-pci driver .

It always complains:  

Invalid access at addr 0x400000030000, size 4, region '(null)', reason: rejected




From  my application log, I can see the IOVA is mapped:

{ iova_range: RangeInclusive { start: 70368744374272, end: 70368744378367 }, memfd: Memfd { file: File { fd: 167, path: "/memfd:dma_memory_0x400000030000 (deleted)", read: true, write: true } }, mmap: MmapRaw { ptr: 0x7feb9bc7a000, len: 4096 } }




From the QEMU OS kernel  trace, I can see the mapping is there:  

workload-697 [000] ..... 31.476109: map: IOMMU: iova=0x0000400000030000 - 0x0000400000031000 paddr=0x0000000109eca000 size=4096

From QEMU trace event, I can see the IOMMU is enabled.  the  02:00.0 is a PCI bridge, and 03:00.0 is my emulated device  

pci_update_mappings_add d=0x56461fcf1a40 03:00.0 0,0xfe800000+0x100000 

pci_update_mappings_add d=0x56461fcf1a40 03:00.0 1,0xfe900000+0x100000 

pci_update_mappings_add d=0x56461fcf1a40 03:00.0 2,0x8000000000+0x1000000 

pci_update_mappings_add d=0x56461fcf1a40 03:00.0 4,0x4000000000+0x4000000000 

vtd_switch_address_space Device 02:00.0 switching address space (iommu enabled=1) 

vtd_switch_address_space Device 02:00.0 switching address space (iommu enabled=1) 

vtd_switch_address_space Device 03:00.0 switching address space (iommu enabled=1) 

vtd_switch_address_space Device 03:00.0 switching address space (iommu enabled=1) 

vtd_switch_address_space Device 02:00.0 switching address space (iommu enabled=1) 

vtd_switch_address_space Device 02:00.0 switching address space (iommu enabled=1) 

vtd_switch_address_space Device 03:00.0 switching address space (iommu enabled=1) 

vtd_switch_address_space Device 03:00.0 switching address space (iommu enabled=1)

From GDB, looks like the IOVA is io_mem_unassigned:

$20 = {size = 18446744073709551616, mr = 0x5613a6519840 <io_mem_unassigned>, fv = 0x5613a67ab3e0, offset_within_region = 0, offset_within_address_space = 0, readonly = false, nonvolatile = false}










Any ideas about this ?

Thanks.

Xu