Rust in QEMU update, January 2025

[Paolo Bonzini]

It's been roughly two months since my previous posting of a roadmap
for Rust in QEMU, so it's time for an update.  While the project is
still at an experimental phase, the amount of functionality available
from safe Rust is enough that it could be considered for new devices.

As before, this mostly covers what I have looked at, which is making
it possible to write devices in safe Rust.  Topics such as QAPI and
async (block devices) are missing for this reason. An "⚠️" icon marks
the most important missing features.

Table of contents
'''''''''''''''''

* Status in QEMU 10.0
* Build system
* Feature parity for devices
* Avoiding undefined behavior
* Bindings to C code
* New utility code
* Rust version requirements
* Next steps


Status in QEMU 10.0
'''''''''''''''''''

QEMU when built with ``--enable-rust`` compiles on all supported
build platforms.  It passes CI and ``make check-unit`` runs tests for
rust/qemu-api.  ``make check-qtests`` covers the Rust pl011 device model,
including migration; a Rust conversion of the HPET device is available
on the mailing list, and will probably be merged in the next few weeks.

The amount of unsafe code needed to define classes, define callbacks,
and invoke core QEMU functionality has gone down a lot since 9.2,
especially if you count a few more pieces that are posted and pending
review.  QOM objects can be created and their lifetime managed using
entirely safe code; unlike C code, Rust is able to differentiate embedded
objects (which do not have an independent reference count) from those
that are allocated in their own heap block.  Available bindings include
sysbus devices, GPIO pins, clocks and MemoryRegionOps and timers;
Kevin Wolf is working on the block layer and on using Rust in tools.
There are also core building blocks that make it much easier to extend
the C<->Rust bindings.

Last December I posted a comparison between the pl011 C implementation
and the "intended" shape of the Rust code; at the time of the posting
(https://lists.nongnu.org/archive/html/qemu-rust/2024-12/msg00006.html)
the code compiled but was far from being usable.  The in-tree Rust
implementation of pl011 is now very similar to that intended shape,
the main exception being character devices and safe ``instance_init``.


Build system
''''''''''''

Developers can use ninja to easily access clippy, rustfmt and rustdoc.
Meson 1.7 supports clippy natively, with a rustdoc pull request
in progress.  This is important because long term it would be nice to
avoid having a proliferation of src/ directories since QEMU has hundreds
of devices.

A Meson pull request is also in progress that add support for doctests,
integrating them into ``make check`` and allowing them (unlike ``cargo test
--doc``) to link with C code[1]_.

⚠️ Modules do not work with Rust code yet due to a limitation of
rust/qemu-api/meson.build.  The fix for this is also in Meson and there
is a pull request open for this as well[2]_.

Rust is still not enabled and its presence is not checked for by
default.  If any devices are contributed that are written in Rust and
do not have a C counterpart, it may be worth splitting "enable Rust"
from "enable all devices written in Rust".

.. [1] https://github.com/mesonbuild/meson/pull/13933
.. [2] https://github.com/mesonbuild/meson/pull/14031


Feature parity for devices
''''''''''''''''''''''''''

⚠️ Some recent pl011 commits are missing in the Rust version.  Philippe
volunteered to port them to teach himself Rust.  Some issues in the
ported device are now fixed and the migration stream is compatible
between the Rust and C versions.

⚠️ Philippe also has a series to implement flow control and better use of
the PL011 FIFO (IIUC).  Porting this to Rust would be hard right now,
so its Rust implementation blocked on having chardev bindings.

⚠️ HPET is lacking support for live migration.

⚠️ As before, also missing is logging and tracing.


Avoiding undefined behavior
'''''''''''''''''''''''''''

Interior mutability types specific to QEMU---homegrown variants of Cell
and RefCell that support the "big QEMU lock", and therefore are usable
in a multi-threaded context---are included in tree, and used by the
pl011 and HPET device models.  This means that Rust code is now
respecting the invariant that mutable references must be either
unique, or obtained from an ``UnsafeCell<>``.

In some cases it is still creating references that point to invalid
data before they are initialized; this will be solved by using Linux's
pinned-init crate to implement the QOM ``instance_init`` method.
``pinned-init`` is available for use in userspace on crates.io and
eliminates the need to expose partially-initialized objects to safe
Rust code.

John Baublitz pointed out more cases of fields that should also be
included in a cell, because they are modified by C code.  These
include fields for qdev properties, and the C structs for QOM
classes.  The issues here are a lot more theoretical than the
rest, and Linux provides examples of how to deal with them too.


Bindings to C code
''''''''''''''''''

Even though QOM's object-oriented, inheritance heavy design is a challenge
for Rust, qdev bindings in QEMU 10.0 are going to be relatively complete,
covering both classes and interfaces.  Thanks to a new callback
functionality, devices have access to GPIO pins, timers, clocks and
MemoryRegionOps.  This should be enough to implement devices with very
limited use of unsafe code.

One area of development that was not mentioned in the previous roadmap
is VMState.  While a direct port of the C macros was part of 9.2, it
required a large amount of macros, and especially it was not type safe.
The latter actually made the Rust version worse than the C code and,
while fixable, rewriting it to take advantage of Rust's type system
and const evaluation was a better plan.  The VMState API is not yetfinal
(for example it still needs unsafe callbacks for pre_save/post_load)
but the new version is already a lot easier to use.

The remaining instances of `unsafe` blocks in the pl011 and HPET code
are as follows:

* pl011 needs character device bindings; the code for this is almost
  ready, though it may not necessarily be in 10.0

* HPET does some very simple memory accesses; a good safe solution
  for this may be the ``vm-memory`` crate.  I have not looked into
  using it, but ``vm-memory`` and ``vm-virtio`` were written with
  QEMU's use cases in mind

* as mentioned above, bindings for VMState definitions are also in
  progress and still partly unsafe

* the ``instance_init`` method is using unsafe code until the
  ``pinned-init`` crate is introduced.  ``pinned-init`` will also
  help with creating cyclical data structures

While this may seem a lot, the fact that we can enumerate the uses of
``unsafe`` and plan their removal is already a substantial achievement!

The amount of functionality available from safe Rust is enough that
including new devices should be possible, even if they need some unsafe
code for parts of QEMU that do not have bindings yet.

Most devices added to QEMU are simple and do not do any complex DMA.
In the previous roadmap I mentioned that such simple devices have very
little benefit from rewriting them in Rust.  While this is probably
true for devices that exist as C code, there is in my opinion already a
benefit to writing *new* devices in Rust.  The code is generally clearer
and so are some aspects of the semantics (e.g. which fields are mutable);
the main limitation is lack of support for tracing and logging.


New utility code
''''''''''''''''

The qemu-api and qemu-api-macros crates now include more utility code
for use in both bindings and devices, including:

* bit-twiddling extensions for integers

* a procedural macro to convert between enums and integers

* callback support

Callbacks are the best and most unexpected news since the last roadmap
post.  The solution implemented in the qemu-api crate makes it possible
to convert a Rust function and a reference into a function pointer and
an opaque value; generics are used to create separate trampolines (i.e.
separate function pointers) for each Rust function.  This mechanism
is used by clocks, timers, memory regions and character devices; so
this part can be considered solved.

The qemu-api-macros crates has also grown a small set of utility functions
that can be reused by more procedural macros in the future.

One utility that I'm missing is bit-flag enums.  The most
commonly used crate in the Rust ecosystem is bitflags
(https://docs.rs/bitflags/2.6.0/bitflags/); it does not have any
dependencies so it is easy to incorporate in QEMU if desired.


Rust version requirements
'''''''''''''''''''''''''

Compared to the previous roadmap, the main change is that three features
have become a bit more pressing:

* while pinned-init's code only needs small changes to support Rust 1.63.0,
  it relies heavily on ``impl Trait`` return types; trait functions however
  can only return ``impl Trait`` since Rust 1.75.0.  Because instance_init
  is itself in a trait, this is an important limitation which might delay the
  inclusion of pinned-init; the only supported distro with an older rustc
  is Debian bookworm.

* references to statics in constants (stable in 1.83.0); the main place
  where ugly workarounds are needed is VMState.

While none of these are blockers for enabling Rust, the first two add a
small amount of technical debt for each Rust device that is added to QEMU.
It may be appealing to use the ``rustc-web`` package on Debian bookworm,
which provides the 1.78.0 version of rustc as of this writing.


Next steps
''''''''''

Two areas that were mentioned in earlier discussions have seen no
activity yet:

* Tracepoints and logging are the highest-priority missing feature, perhaps
  together with DMA; while the current set of device only has a limited set
  of tracepoints, observability is obviously important as a debugging tool.
  This is the main blocker before implementing new devices in Rust can be
  encouraged.

* interoperability between QEMU's C data structure and Rust counterparts
  has no news either.  We'll figure it out as soon as we need a realize()
  implementation that can fail, or when tackling QAPI.

On top of this it may be worthwhile to investigate again using procedural
macros to simplify declaring QOM/qdev classes, such as for qdev properties.