Re: [RFC 0/3] Introduce a new Qemu machine for RISC-V

[Daniel P . Berrangé]

On Fri, May 06, 2022 at 06:34:47AM +1000, Alistair Francis wrote:
> On Thu, May 5, 2022 at 8:04 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
> >
> > On Thu, May 05, 2022 at 07:36:51PM +1000, Alistair Francis wrote:
> > > On Tue, May 3, 2022 at 5:57 PM Atish Patra <atishp@atishpatra.org> wrote:
> > > >
> > > > On Tue, Apr 19, 2022 at 5:26 PM Atish Patra <atishp@atishpatra.org> 
> > > > wrote:
> > > > >
> > > > > On Tue, Apr 19, 2022 at 9:51 AM Daniel P. Berrangé 
> > > > > <berrange@redhat.com> wrote:
> > > > > >
> > > > > > On Mon, Apr 11, 2022 at 07:10:06PM -0700, Atish Patra wrote:
> > > > > > >
> > > > > > > The RISC-V virt machine has helped RISC-V software eco system to 
> > > > > > > evolve at a
> > > > > > > rapid pace even in absense of the real hardware. It is definitely 
> > > > > > > commendable.
> > > > > > > However, the number of devices & commandline options keeps 
> > > > > > > growing as a result
> > > > > > > of that as well. That adds flexibility but will also become bit 
> > > > > > > difficult
> > > > > > > to manage in the future as more extension support will be added. 
> > > > > > > As it is the
> > > > > > > most commonly used qemu machine, it needs to support all kinds of 
> > > > > > > device and
> > > > > > > interrupts as well. Moreover, virt machine has limitations on the 
> > > > > > > maximum
> > > > > > > number of harts it can support because of all the MMIO devices it 
> > > > > > > has to support.
> > > > > > >
> > > > > > > The RISC-V IMSIC specification allows to develop machines 
> > > > > > > completely relying
> > > > > > > on MSI and don't care about the wired interrupts at all. It just 
> > > > > > > requires
> > > > > > > all the devices to be present behind a PCI bus or present 
> > > > > > > themselves as platform
> > > > > > > MSI device. The former is a more common scenario in x86 world 
> > > > > > > where most
> > > > > > > of the devices are behind PCI bus. As there is very limited MMIO 
> > > > > > > device
> > > > > > > support, it can also scale to very large number of harts.
> > > > > > >
> > > > > > > That's why, this patch series introduces a minimalistic yet very 
> > > > > > > extensible
> > > > > > > forward looking machine called as "RISC-V Mini Computer" or 
> > > > > > > "minic". The
> > > > > > > idea is to build PC or server like systems with this machine. The 
> > > > > > > machine can
> > > > > > > work with or without virtio framework. The current implementation 
> > > > > > > only
> > > > > > > supports RV64. I am not sure if building a RV32 machine would be 
> > > > > > > of interest
> > > > > > > for such machines. The only mmio device it requires is clint to 
> > > > > > > emulate
> > > > > > > the mtimecmp.
> > > > > >
> > > >
> > > > Any other thoughts ?
> > >
> > > I don't *love* this idea. I think the virt machine is useful, but I'm
> > > not convinced we need a second one.
> > >
> > > This feels a little bit more like a "none" machine, as it contains
> > > just the bare minimum to work.
> > >
> > > >
> > > > > > I would ask what you see as the long term future usage for 'virt' vs
> > > > > > 'minic' machine types ? Would you expect all existing users of 
> > > > > > 'virt'
> > > > > > to ultimately switch to 'minic', or are there distinct 
> > > > > > non-overlapping
> > > > > > use cases for 'virt' vs 'minic' such that both end up widely used ?
> > > > > >
> > > > >
> > > > > Nope. I don't expect existing 'virt' users to switch to 'minic' as
> > > > > they aim to cater to different users.
> > > > >
> > > > > Here are the major differences
> > > > > 1. virt machine supports MMIO devices & wired interrupts. Minic 
> > > > > doesn't
> > >
> > > This seems like the main difference
> > >
> > > > > 2. virt machine doesn't support the MSI only option yet (can be added
> > > > > though[1]). Minic does.
> > >
> > > This could be fixed
> > >
> > > > > 3. Number of cpu supported by virt machine are limited because of the
> > > > > MMIO devices. Minic can scale to very
> > > > > large numbers of cpu.
> > >
> > > Similar to 1
> > >
> > > > > 4. 'Minic' only supports PCI based MSI capable devices. Thus, MSI is a
> > > > > mandatory requirement for 'minic' while
> > > > > it is optional for 'virt'.
> > >
> > > I'm not fully convinced we need this, but it also doesn't seem to cost
> > > us a lot in terms of maintenance. It would be beneficial if we could
> > > share a bit more of the code. Can we share the socket creation code as
> > > well?
> > >
> > > I don't like the name minic though. What about something like
> > > `virt-hpc`, `virt-pcie-minimal` or something like that? That way we
> > > indicate it's still a virt board
> >
> > We're not versioning the 'virt' machine type right so. IOW, we've not
> > made any promises about its long term featureset.
> >
> > If the virt machine type isn't the perfect match right now, should
> > we change it, in a potentially incompatible way, to give us the right
> > solution long term, rather than introducing a brand new machine type
> > with significant overlap.
> 
> Even if we didn't worry about backwards compatibility the current virt
> machine would still be what most users want. It's just a small number
> of users who don't want MMIO devices and instead want to use PCIe for
> everything. Realistically it's only HPC users who would want this type
> of machine, at least that's my understanding.

I'm not so sure about that. Every other architecture has ended up
standardizing on PCI for general purpose virtual machines. IIRC,
aarch64 started off with MMIO, but switched to PCI as it matured. 

In terms of having VM mgmt tools "just work" for risc-v, I think
it will be very compelling for the general 'virt' machine to be
PCI based, otherwise all the assumptions about PCI in mgmt apps
are going to break requiring never ending riscv fixes.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|