Re: [Qemu-ppc] [PATCH|RFC 1/1] pci: allow 0 address for PCI IO/MEM regions

[David Gibson]

On Wed, Jul 22, 2015 at 07:17:16PM +0200, Alexander Graf wrote:
> On 07/22/15 19:08, Laurent Vivier wrote:
> >
> >On 22/07/2015 18:47, Alexander Graf wrote:
> >>On 07/22/15 18:17, Michael S. Tsirkin wrote:
> >>>On Wed, Jul 22, 2015 at 01:54:59PM +0200, Laurent Vivier wrote:
> >>>>From: Michael Roth <address@hidden>
> >>>>
> >>>>Some kernels program a 0 address for io regions. PCI 3.0 spec
> >>>>section 6.2.5.1 doesn't seem to disallow this.
> >>>>
> >>>>Signed-off-by: Michael Roth <address@hidden>
> >>>>[lvivier: add accept_addr_0 in PCIBus to conditionally allow addr 0,
> >>>>   as this can break other architectures]
> >>>>Signed-off-by: Laurent Vivier <address@hidden>
> >>>I guess it's a solution - though I'd rather fix up priorities
> >>>for PC and others.
> >>Won't this break Linux? IIRC there was a check in Linux that declares
> >>IO==0 addresses as invalid on PCI enumeration.
> >Well, without this patch, it doesn't work (try 2.4.0-rcX -M pseries and
> >add a virtio-net-pci card).
> >
> >But the address 0 appears only on PCI hotplug. The first card we add
> >"lively" is always inserted with address 0 for BAR0 (even if we have
> >already a PCI card on the bus).
> >
> >If we add a second card, it works well (as it can't have the address 0
> >again).
> >
> >If we reboot the machine, the cards are reordered and have "valid" (!=
> >0) addresses.
> >
> >Do you suspect a bug in the kernel PCI hotplug functions ?
> 
> I'm not quite sure whether there is an actually visible bug, but I wanted to
> make sure you're aware that I've run into horrible issues on real hardware
> that exposed IO BARs at address 0 and then told Linux to not reconfigure
> BARs. Because then some code somewhere hidden deep inside Linux just assumes
> that the BAR is invalid.
> 
> IIRC BAR mapping for sPAPR hotplug happens in QEMU, no? So you can just
> guarantee that it never hits 0.

It should happen in qemu, but it doesn't, yet.  

For boot-time devices, BAR allocation is done in SLOF, even though bus
enumeration is done in qemu, and qemu does have code to publish
assigned-addresses if anything had set them.

For hotplugged devices, the code is currently relying on some weird
behaviour in Linux guests that causes them to fall back to assigning
BARs themselves, even though they shouldn't under PAPR.  It's the
guest itself attempting to assign address 0 and failing.

I understand from Michael Roth that there are other bugs floating
about that make moving to qemu based BAR assignment non-trivial,
although that's certainly the way it should be done long term. :/

Like you, I was under the impression that we were already doing BAR
assignment in qemu, but I recently discovered I had misunderstood what
Michael's and Nikunj's code was doing.

-- 
David Gibson                    | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
                                | _way_ _around_!
http://www.ozlabs.org/~dgibson

pgpcWG73hPkAW.pgp
Description: PGP signature