Re: making a qdev bus available from a (non-qtree?) device

[Markus Armbruster]

Klaus Jensen <its@irrelevant.dk> writes:

> Hi all,
>
> I need some help with grok'ing qdev busses. Stefan, Michael - David
> suggested on IRC that I CC'ed you guys since you might have solved a 
> similar issue with virtio devices. I've tried to study how that works,
> but I'm not exactly sure how to apply it to the issue I'm having.
>
> Currently, to support multiple namespaces on the emulated nvme device,
> one can do something like this:
>
>   -device nvme,id=nvme-ctrl-0,serial=foo,...
>   -device nvme-ns,id=nvme-ns-0,bus=nvme-ctrl-0,...
>   -device nvme-ns,id-nvme-ns-1,bus=nvme-ctrl-0,...
>
> The nvme device creates an 'nvme-bus' and the nvme-ns devices has
> dc->bus_type = TYPE_NVME_BUS. This all works very well and provides a 
> nice overview in `info qtree`:
>
>   bus: main-system-bus
>   type System
>     ...
>     dev: q35-pcihost, id ""
>       ..
>       bus: pcie.0
>       type PCIE
>       ..
>       dev: nvme, id "nvme-ctrl-0"
>         ..
>         bus: nvme-ctrl-0
>           type nvme-bus
>           dev: nvme-ns, id "nvme-ns-0"
>             ..
>           dev: nvme-ns, id "nvme-ns-1"
>             ..
>
>
> Nice and qdevy.
>
> We have since introduced support for NVM Subsystems through an
> nvme-subsys device. The nvme-subsys device is just a TYPE_DEVICE and 
> does not show in `info qtree`

Yes.

Most devices plug into a bus.  DeviceClass member @bus_type specifies
the type of bus they plug into, and DeviceState member @parent_bus
points to the actual BusState.  Example: PCI devices plug into a PCI
bus, and have ->bus_type = TYPE_PCI_BUS.

Some devices don't.  @bus_type and @parent_bus are NULL then.

Most buses are provided by a device.  BusState member @parent points to
the device.

The main-system-bus isn't.  Its @parent is null.

"info qtree" only shows the qtree rooted at main-system-bus.  It doesn't
show qtrees rooted at bus-less devices or device-less buses other than
main-system-bus.  I doubt such buses exist.

>                               (I wonder if this should actually just
> have been an -object?).

Does nvme-subsys expose virtual hardware to the guest?  Memory, IRQs,
...

If yes, it needs to be a device.

If no, object may be more appropriate.  Tell us more about what it does.


>                         Anyway. The nvme device has a 'subsys' link 
> parameter and we use this to manage the namespaces across the
> subsystem that may contain several nvme devices (controllers). The
> problem is that this doesnt work too well with unplugging since if the
> nvme device is `device_del`'ed, the nvme-ns devices on the nvme-bus
> are unrealized which is not what we want. We really want the
> namespaces to linger, preferably on an nvme-bus of the nvme-subsys
> device so they can be attached to other nvme devices that may show up
> (or already exist) in the subsystem.
>
> The core problem I'm having is that I can't seem to create an nvme-bus
> from the nvme-subsys device and make it available to the nvme-ns
> device on the command line:
>
>   -device nvme-subsys,id=nvme-subsys-0,...
>   -device nvme-ns,bus=nvme-subsys-0
>
> The above results in 'No 'nvme-bus' bus found for device 'nvme-ns',
> even though I do `qbus_create_inplace()` just like the nvme
> device. However, I *can* reparent the nvme-ns device in its realize()
> method, so if I instead define it like so:
>
>   -device nvme-subsys,id=nvme-subsys-0,...
>   -device nvme,id=nvme-ctrl-0,subsys=nvme-subsys-0
>   -device nvme-ns,bus=nvme-ctrl-0
>
> I can then call `qdev_set_parent_bus()` and set the parent bus to the
> bus creates in the nvme-subsys device. This solves the problem since
> the namespaces are not "garbage collected" when the nvme device is
> removed, but it just feels wrong you know? Also, if possible, I'd of
> course really like to retain the nice entries in `info qtree`.

I'm afraid I'm too ignorant on NVME to give useful advice.

Can you give us a brief primer on the aspects of physical NVME devices
you'd like to model in QEMU?  What are "controllers", "namespaces", and
"subsystems", and how do they work together?

Once we understand the relevant aspects of physical devices, we can
discuss how to best model them in QEMU.