Re: [Qemu-devel] [RFC v5 0/6] pci_expander_brdige: support separate

[Julia Suvorova]

On Mon, 2018-09-17 at 22:57 +0800, Zihan Yang wrote:
> Hi all
> 
> Here is a minimal working version of supporting multiple pci domains.

Hi Zihan,
Do you plan to continue working on this project?

I am interested in it, and if you do not mind I would like to finish
the project, retaining your authorship. I am going to take care
of this patch set, and the one in SeaBios too.
How does it sound to you?

Any tips and clues are welcomed.

Best regards, Julia Suvorova.

> The next a few paragraphs will illustrate the purpose and use
> example.
> Current issue and limitations will be in last 2 paragraphs, followed
> by the changelog of each verison.
> 
> Currently only q35 host bridge is allocated an item in MCFG table,
> all
> pxb-pcie host bridges stay within pci domain 0. This series of
> patches
> allow each pxb-pcie to be put in separate pci domain, allocating a
> new
> MCFG table item for it.
> 
> Users can configure whether to put pxb host bridge into separate
> domain
> by specifying property 'domain_nr' of pxb-pcie device. 'bus_nr'
> property
> indicates the Base Bus Number(BBN) of the pxb-pcie host bridge.
> Another
> property max_bus specifies the maximum desired bus numbers to reduce
> MCFG space cost. Example command is
> 
>     -device pxb-pcie,id=bridge3,bus="pcie.0",domain_nr=1,max_bus=15
> 
> Then this pxb-pcie host bridge is placed at pci domain 1, and only 
> reserve
> (15+1)=16 buses, which is much smaller than the default 256 buses.
> 
> Compared with previous version, this version is much simpler because
> mcfg of extra domain now has a relatively fixed address, as suggested
> by Marcel and Gerd. Putting extra mmconfig above 4G and let seabios
> leave them for guest os will be expected in next version. The range
> is
> [0x80000000, 0xb0000000), which allows us to hold 4x busses compared
> with before.
> 
> A complete command line for test is follows, you need to replace 
> GUEST_IMAGE,
> DATA_IMAGE and SEABIOS_BIN with proper environment variable
> 
> ./x86_64-softmmu/qemu-system-x86_64 \
>     -machine q35,accel=kvm -smp 2 -m 2048 \
>     -drive file=${GUEST_IMAGE}  -netdev user,id=realnet0 \
>     -device e1000e,netdev=realnet0,mac=52:54:00:12:34:56 \
>     -device pxb-pcie,id=bridge3,bus="pcie.0",domain_nr=1 \
>     -device pcie-root-
> port,id=rp1,bus=bridge3,addr=1c.0,port=8,chassis=8 
> \
>     -drive if=none,id=drive0,file=${DATA_IMAGE} \
>     -device virtio-scsi-pci,id=scsi,bus=rp1,addr=00.0 \
>     -bios ${SEABIOS_BIN}
> 
> There are a few limitations, though
> 1. Legacy interrupt routing is not dealt with yet. There is only
> support 
> for
>    devices using MSI/MSIX
> 2. Only 4x devices is supported, you need to be careful not to
> overuse
> 3. I have not fully tested the functionality of devices under
> separate 
> domain
>    yet, but Linux can recognize then when typing `lspci`
> 
> Current issue:
> * SCSI storage device will be recognized twice, one in domain 0 as 
> 0000:01.0,
>   the other in domain 1 as 0001:01.0. I will try to fix it in next 
> version
> 
> v5 <- v4:
> - Refactor the design and place pxb-pcie's mcfg in [0x80000000, 
> 0xb0000000)
> - QEMU only decides the desired mcfg_size and leaves mcfg_base for 
> seabios
> - Does not connect PXBDev and PXBPCIEHost with link property
> anymore, 
> but
>   with the pcibus under them, which makes code simpler.
> 
> v4 <- v3:
> - Fix bug in setting mcfg table
> - bus_nr is not used when pxb-pcie is in a new pci domain
> 
> v3 <- v2:
> - Replace duplicate properties in pxb pcie host with link property
> to 
> PXBDev
> - Allow seabios to access config space and data space of expander
> bridge
>   through a different ioport, because 0xcf8 is attached only to
> sysbus.
> - Add a new property start_bus to indicate the BBN of pxb host
> bridge. 
> The
>   bus_nr property is used as the bus number of pxb-pcie device on
> pcie.0 
> bus
> 
> v2 <- v1:
> - Allow user to configure whether to put pxb-pcie into seperate
> domain
> - Add AML description part of each host bridge
> - Modify the location of MCFG space to between RAM hotplug and pci 
> hole64
> 
> Many thanks to 
> Please let me know if you have any suggestions.
> 
> Zihan Yang (6):
>   pci_expander_bridge: add type TYPE_PXB_PCIE_HOST
>   pci_expander_bridge: add domain_nr and max_bus property for pxb-
> pcie
>   acpi-build: allocate mcfg for pxb-pcie host bridges
>   i386/acpi-build: describe new pci domain in AML
>   pci_expander_bridge: add config_write callback for pxb-pcie
>   pci_expander_bridge: inform seabios of desired mcfg size via hidden
>     bar
> 
>  hw/i386/acpi-build.c                        | 162 
> ++++++++++++++++++--------
>  hw/pci-bridge/pci_expander_bridge.c         | 172 
> +++++++++++++++++++++++++++-
>  hw/pci/pci.c                                |  30 ++++-
>  include/hw/pci-bridge/pci_expander_bridge.h |  25 ++++
>  include/hw/pci/pci.h                        |   2 +
>  include/hw/pci/pci_bus.h                    |   2 +
>  include/hw/pci/pci_host.h                   |   2 +-
>  7 files changed, 336 insertions(+), 59 deletions(-)
>  create mode 100644 include/hw/pci-bridge/pci_expander_bridge.h
>