[Qemu-devel] vfio failure with intel 760p 128GB nvme

[Dongli Zhang]

Hi,

I obtained below error when assigning an intel 760p 128GB nvme to guest via
vfio on my desktop:

qemu-system-x86_64: -device vfio-pci,host=0000:01:00.0: vfio 0000:01:00.0: 
failed to add PCI capability address@hidden: table & pba overlap, or they don't 
fit in BARs, or don't align


This is because the msix table is overlapping with pba. According to below
'lspci -vv' from host, the distance between msix table offset and pba offset is
only 0x100, although there are 22 entries supported (22 entries need 0x160).
Looks qemu supports at most 0x800.

# sudo lspci -vv
... ...
01:00.0 Non-Volatile memory controller: Intel Corporation Device f1a6 (rev 03) 
(prog-if 02 [NVM Express])
        Subsystem: Intel Corporation Device 390b
... ...
        Capabilities: [b0] MSI-X: Enable- Count=22 Masked-
                Vector table: BAR=0 offset=00002000
                PBA: BAR=0 offset=00002100



A patch below could workaround the issue and passthrough nvme successfully.

diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index 5c7bd96..54fc25e 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -1510,6 +1510,11 @@ static void vfio_msix_early_setup(VFIOPCIDevice *vdev, 
Error **errp)
     msix->pba_offset = pba & ~PCI_MSIX_FLAGS_BIRMASK;
     msix->entries = (ctrl & PCI_MSIX_FLAGS_QSIZE) + 1;
 
+    if (msix->table_bar == msix->pba_bar &&
+        msix->table_offset + msix->entries * PCI_MSIX_ENTRY_SIZE > 
msix->pba_offset) {
+        msix->entries = (msix->pba_offset - msix->table_offset) / 
PCI_MSIX_ENTRY_SIZE;
+    }
+
     /*
      * Test the size of the pba_offset variable and catch if it extends outside
      * of the specified BAR. If it is the case, we need to apply a hardware


Would you please help confirm if this can be regarded as bug in qemu, or issue
with nvme hardware? Should we fix thin in qemu, or we should never use such 
buggy
hardware with vfio?

Thank you very much!

Dongli Zhang