Re: [RFC PATCH 0/9] Initial support for machine creation via QMP

[Mirela Grujic]

Hi Paolo,


Thanks for the feedback!


On 5/13/21 7:52 PM, Paolo Bonzini wrote:
> Hi Mirela, this is very interesting!
>
> It's unfortunate that I completely missed the discussions in 
> January/February.  You might have noticed that in the 5.2/6.0 
> timeframe I worked on cleaning up the machine initialization phases 
> and qemu_init.  The idea behind the cleanup was to identify clearly 
> the steps from one phase to the next.  I am very happy that you are 
> already benefitting from that work in this series and you were able to 
> make a prototype with so little code.


I was really happy to see your changes in 6.0, they simplified the 
implementation a lot. It looked like you're up to something bigger, and 
I'm glad that we can sync up now.


> My plan was a bit more ambitious though :) and it is laid out at 
> https://wiki.qemu.org/User:Paolo_Bonzini/Machine_init_sequence.
>
> My plan was to have completely different binaries than the current 
> qemu-system-*.  These would only have a handful of command line 
> options (such as -name, -sandbox, -trace, -L) and would open a QMP 
> connection on stdio.
>
> All other command line option would be either considered legacy or 
> adjusted to be part of two new QMP commands, machine-set and 
> accel-set, which would advance through the phases like this:
>
> PHASE_NO_MACHINE
>    -> machine-set -> PHASE_MACHINE_CREATED ->
>    -> accel-set -> PHASE_ACCEL_CREATED -> PHASE_MACHINE_INITIALIZED ->
>    -> finish-machine-init -> PHASE_MACHINE_READY
>    -> cont
>
> I think this idea would work well with your plan below, because the 
> preconfiguration that you need to do happens mostly at 
> PHASE_MACHINE_INITIALIZED.
>
> Of course, the disadvantage of my approach is that, in the initial 
> version, a lot of capabilities of QEMU are not available (especially 
> with respect to the UI, since there's no "display-add" command). 
> However, the basic implementation of machine-set and accel-set should 
> not really be *that* much more work, and it should be doable in 
> parallel with the conversion efforts for those options.  For example, 
> today I posted a series that maps -smp to -M (and then, SMP 
> configuration would automatically become available in machine-set).


With our approach, transitioning to the QMP configuration suppose to 
happen gradually, i.e. we still specify some configuration options via 
command line. For your approach to be applicable to our use case we 
would at least need a QMP equivalent for the following:

qemu-system-riscv64 \
    -M sifive_dt \
    -cpu 
rv64,i=true,g=false,m=true,a=true,f=true,d=true,c=true,s=false,u=false,x-b=true,pmp=true,mmu=false,num-pmp-regions=8 
\
    -smp 1 \
    -device ...

AFAIU from the materials you shared, we would need to add -cpu and 
-device, but I don't see any reason why we wouldn't do this.


> I have placed the skeleton of the work I was doing at 
> https://gitlab.com/bonzini/qemu/ in the branch qemu-qmp-targets. You 
> can see a sample execution at 
> https://asciinema.org/a/TXMX8EZh8Md0fZnuE7uhfv6cO.  I have not touched 
> some of the patches in a long time, but I plan to give them a quick 
> test tomorrow.  Starting from the code in that branch, it should not 
> be hard to implement the machine-set and accel-set commands in the 
> same fashion as QEMU 5.2's implementation of object-add.

Ok, please let me know once you test, then I would run your code and 
play with it to better understand what needs to be done. Then I might 
come back with a couple of questions, so that we align on the TODOs. Is 
that ok with you?

Btw, when (in which version) did you plan to integrate the qemu-qmp-* 
support? I guess once machine-set/accel-set is implemented, but maybe 
I'm wrong...


> Thanks for posting these patches, I have started a light review of them.

If we would add the support for our use case to your approach, then this 
series would likely be split into a couple of patches that are 
applicable and the rest that is obsolete.


In summary, we believe it would be great to join efforts, please let us 
know how can we help.


Thanks,

Mirela


> Paolo
>
> On 13/05/21 10:25, Mirela Grujic wrote:
> > The direction for this work has been set in the discussion thread:
> > "About creating machines on the command line" in January/February 2021:
> > https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg01839.html
> > https://lists.gnu.org/archive/html/qemu-devel/2021-02/msg01070.html
> >
> > To customize a machine via QMP we need the ability to stop QEMU at a 
> > specific
> > machine initialization phase.
> >
> > Currently, machine initialization phases are:
> > 1) no-machine: machine does not exist yet (current_machine == NULL)
> > 2) machine-created: machine exists, but its accelerator does not
> >     (current_machine->accelerator == NULL)
> > 3) accel-created: machine's accelerator is configured
> >     (current_machine->accelerator != NULL), but machine class's 
> > init() has not
> >     been called (no properties validated, machine_init_done notifiers 
> > not
> >     registered, no sysbus, etc.)
> > 4) initialized: machine class's init() has been called, thus machine 
> > properties
> >     are validated, machine_init_done notifiers registered, sysbus 
> > realized, etc.
> >     Devices added at this phase are considered to be cold-plugged.
> > 5) ready: machine_init_done notifiers are called, then QEMU is ready 
> > to start
> >     CPUs. Devices added at this phase are considered to be hot-plugged.
> >
> > QEMU can be stopped today using the -preconfig CLI option at phase 3
> > (accel-created). This option was introduced to enable the QMP 
> > configuration of
> > parameters that affect the machine initialization. We cannot add 
> > devices at
> > this point because the machine class's init() has not been called, 
> > thus sysbus
> > does not exist yet (a device cannot be added because there is no bus 
> > to attach
> > it to).
> >
> > QEMU can be also stopped using the -S CLI option at the machine ready 
> > phase.
> > However, it is too late to add devices at this phase because the 
> > machine is
> > already configured, and any devices added at this point are 
> > considered to be
> > hot-plugged.
> >
> > Since the existing -preconfig CLI option stops QEMU too early, and 
> > the -S option
> > stops too late, we need a way to stop QEMU in between (after the 
> > machine is
> > initialized and before it becomes ready).