Re: [RFC 0/3] Introduce a new Qemu machine for RISC-V

[Alistair Francis]

On Wed, May 18, 2022 at 4:38 PM Atish Patra <atishp@atishpatra.org> wrote:
>
> On Tue, May 17, 2022 at 1:54 PM Alistair Francis <alistair23@gmail.com> wrote:
> >
> > On Tue, May 17, 2022 at 6:52 PM Daniel P. Berrangé <berrange@redhat.com> 
> > wrote:
> > >
> > > On Tue, May 17, 2022 at 03:03:38PM +1000, Alistair Francis wrote:
> > > > On Sat, May 7, 2022 at 6:30 AM Atish Kumar Patra <atishp@rivosinc.com> 
> > > > wrote:
> > > > >
> > > > > On Fri, May 6, 2022 at 4:00 AM Peter Maydell 
> > > > > <peter.maydell@linaro.org> wrote:
> > > > > >
> > > > > > On Fri, 6 May 2022 at 09:18, Daniel P. Berrangé 
> > > > > > <berrange@redhat.com> wrote:
> > > > > > >
> > > > > > > On Fri, May 06, 2022 at 06:34:47AM +1000, Alistair Francis wrote:
> > > > > > > > Even if we didn't worry about backwards compatibility the 
> > > > > > > > current virt
> > > > > > > > machine would still be what most users want. It's just a small 
> > > > > > > > number
> > > > > > > > of users who don't want MMIO devices and instead want to use 
> > > > > > > > PCIe for
> > > > > > > > everything. Realistically it's only HPC users who would want 
> > > > > > > > this type
> > > > > > > > of machine, at least that's my understanding.
> > > > > > >
> > > > > > > I'm not so sure about that. Every other architecture has ended up
> > > > > > > standardizing on PCI for general purpose virtual machines. IIRC,
> > > > > > > aarch64 started off with MMIO, but switched to PCI as it matured.
> > > > > > >
> > > > > > > In terms of having VM mgmt tools "just work" for risc-v, I think
> > > > > > > it will be very compelling for the general 'virt' machine to be
> > > > > > > PCI based, otherwise all the assumptions about PCI in mgmt apps
> > > > > > > are going to break requiring never ending riscv fixes.
> > > > > >
> > > > > > Mmm, my experience with aarch64 virt is that PCI is much nicer
> > > > > > as a general preference. aarch64 virt has some MMIO devices
> > > > > > for historical reasons and some because you can't reasonably
> > > > > > do the necessary things with PCI, but I'm actively trying to
> > > > > > push people who submit new MMIO device features for virt to
> > > > > > try to use a PCI-based solution instead if they possibly can.
> > > >
> > > > Interesting...
> > > >
> > > > Ok, maybe calling this "virt-pcie" might be a good start, with the
> > > > expectation to eventually replace the current virt with the new
> > > > virt-pcie at some point.
> > >
> > > Delaying the inevitable by leaving PCIE support in a separate
> > > machine type initially is going to be more painful long term.
> > >
> > > > The other option would be to try and gradually change from the current
> > > > virt machine to this new virt machine
> > >
> > > Yes, I really think the 'virt' machine type needs to aim for PCIE
> > > support sooner rather than later, if RISC-V wants to get on part
> > > with other architectures. The best time to have added PCIE support
> > > to 'virt' was when it was first created, the next best time is now.
> >
> > So maybe instead we lock in the current virt machine as the 7.1 virt
> > machine for QEMU 7.1, then work on migrating to a PCIe only machine
> > with versions (similar to the other archs)
> >
>
> I am not quite sure what exactly you mean here. Do you mean to modify
> the current virt
> machine to be PCIE only after QEMU 7.1 or the new PCIE only machine
> (with the versioning)
> which will be the default machine in the future

I mean that we call the current virt machine the virt machine for QEMU
7.1. Then for future releases we can make breaking changes, where we
have the old 7.1 virt machine for backwards compatibility.

>
> If you intend to say the former, few issues with that approach.
>
> 1. virt machine is not well documented and already bloated. There is
> no specification for virt machine as such.
> Putting restrictions after a certain release will lead to confusion.
> 2. Do we support existing MMIO devices after that specific version or not ?

Yeah, so I guess this doesn't achieve the same outcome you want. I
would say we would still include some MMIO devices, like UART for
example.

But we could simplify things a bit. So for example maybe we could use
AIA by default and then remove the PLIC code. That would help cleanup
the board file. Then we could add a `msi-only` option that would be
similar to 
https://github.com/atishp04/qemu/commit/d7fc1c6aa7855b414b3484672a076140166a2dcd.
But without the PLIC it should hopefully be cleaner

We would need AIA ratified before we could remove the PLIC though.

> 3. The user has to be aware of which version of virt machine it is
> running in order to test specific features (i.e. flash, reset, wired
> interrupts)

That's true, but I think we are going to have this issue someday
anyway. We don't want to use the SiFive CLINT and PLIC forever,
eventually we will want to use the newer hardware.

> 4. Based on the version of the virt machine, the command line options
> will change. This may also be confusing.
>
> On the other hand, the second approach will be much cleaner without
> any baggage. The RISC-V eco-system is still maturing and this is the
> right time
> to start something fresh. Let's call the new machine virt-pcie for
> now. Here are a few things that can be implemented for this machine.
>
> 1. It must support versioning and any changes to the machine code must
> modify the version of the machine.
> 2. It must only support MSI based PCIe devices. No support for wired 
> interrupts.
>     The only allowed MMIO devices are
>            -- mtimer/mtimecmp (there is no other option provided in ISA)
>            -- reset/rtc device (If there is a way we can emulate these
> two over PCIe, that would be great)
>            -- flash
> Beyond this, adding any other MMIO device must be strongly discouraged.
> 3. The virt-pcie machine must be well documented similar to a hardware
> specification (including address range, restrictions and
> implemented/allowed devices)
> 4. No dependence on virtio framework but must work with virtio-pcie backend.
> 5. Migration must be supported.

I'm on board with these! They would all be great to have.

I'm open to a virt-pcie, but as others have pointed out it might be
easier to just make the switch now to the general board.

> 6. No command line option is required.

I don't see this being achievable unfortunately

> 7. Any other ?
>
> Once we have these policies in place, this can be the preferred virt
> machine and the current virt machine can be phased out or continue to
> co-exist
> as a more flexible experimental platform.
>
> Thoughts ?
>
> > Alistair
> >
> > >
> > > With regards,
> > > Daniel
> > > --
> > > |: https://berrange.com      -o-    
> > > https://www.flickr.com/photos/dberrange :|
> > > |: https://libvirt.org         -o-            
> > > https://fstop138.berrange.com :|
> > > |: https://entangle-photo.org    -o-    
> > > https://www.instagram.com/dberrange :|
> > >
>
>
>
> --
> Regards,
> Atish