Re: [PATCH v2 0/3 qemu] tcg/i386: Page tables in MMIO memory fixes (CXL)

[Jonathan Cameron]

On Thu, 22 Feb 2024 09:07:03 -1000
Richard Henderson <richard.henderson@linaro.org> wrote:

> On 2/19/24 07:31, Jonathan Cameron wrote:
> > v2: Changes documented in patch 3.
> >     - I have not addressed Richard's comment on recursive locks as that
> >       seems to be a more general issue not specific to this patch set.
> > 
> > CXL memory is interleaved at granularities as fine as 64 bytes.
> > To emulate this each read and write access undergoes address translation
> > similar to that used in physical hardware. This is done using
> > cfmws_ops for a memory region per CXL Fixed Memory Window (the PA address
> > range in the host that is interleaved across host bridges and beyond.
> > The OS programs interleaved decoders in the CXL Root Bridges, switch
> > upstream ports and the corresponding decoders CXL type 3 devices who
> > have to know the Host PA to Device PA mappings).
> > 
> > Unfortunately this CXL memory may be used as normal memory and anything
> > that can end up in RAM can be placed within it. As Linux has become
> > more capable of handling this memory we've started to get quite a few
> > bug reports for the QEMU support. However terrible the performance is
> > people seem to like running actual software stacks on it :(
> > 
> > This doesn't work for KVM - so for now CXL emulation remains TCG only.
> > (unless you are very careful on how it is used!)  I plan to add some
> > safety guards at a later date to make it slightly harder for people
> > to shoot themselves in the foot + a more limited set of CXL functionality
> > that is safe (no interleaving!)
> > 
> > Previously we had some issues with TCG reading instructions from CXL
> > memory but that is now all working. This time the issues are around
> > the Page Tables being in the CXL memory + DMA buffers being placed in it.
> > 
> > The test setup I've been using is simple 2 way interleave via 2 root
> > ports below a single CXL root complex.  After configuration in Linux
> > these are mapped to their own Numa Node and
> > numactl --membind=1 ls
> > followed by powering down the machine is sufficient to hit all the bugs
> > addressed in this series.
> > 
> > Thanks to Gregory, Peter and Alex for their help figuring this lot
> > out. Note that I've included one patch from Peter without a SoB
> > because so far it has only be posted in the discussion thread.
> > 
> > Whilst thread started back at:
> > https://lore.kernel.org/all/CAAg4PaqsGZvkDk_=PH+Oz-yeEUVcVsrumncAgegRKuxe_YoFhA@mail.gmail.com/
> > The QEMU part is from.
> > https://lore.kernel.org/all/20240201130438.00001384@Huawei.com/
> > 
> > 
> > Gregory Price (1):
> >    target/i386: Enable page walking from MMIO memory
> > 
> > Jonathan Cameron (1):
> >    tcg: Avoid double lock if page tables happen to be in mmio memory.
> > 
> > Peter Maydell (1):
> >    accel/tcg: Set can_do_io at at start of lookup_tb_ptr helper  
> 
> Thanks.  Queued patches 1 and 3 to tcg-next, and adjusted patch 3 to use 
> BQL_LOCK_GUARD as 
> suggested by Alex.
> 
Thanks.

For patch 2, I assume this goes via an x86 specific path.

Paolo?  Looks like most such patches go through you.

Jonathan


> 
> r~
>