Re: Unexpected error in rme_configure_one() at ../target/arm/kvm-rme.c:159

[Gavin Shan]

Hi Jean,

On 6/3/24 18:24, Jean-Philippe Brucker wrote:
> On Sat, Jun 01, 2024 at 08:14:46PM +1000, Gavin Shan wrote:
> > ---> guest edk2
> >
> > # git clone https://git.codelinaro.org/linaro/dcap/edk2.git edk2-guest
> > # cd edk2-guest; git checkout origin/cca/v2 -b cca/v2
> > # git submodule update --init --recursive;  \
> >    source edksetup.sh; make -j -C BaseTools; \
> >    export GCC5_AARCH64_PREFIX=;              \
>
> Doesn't this needs a cross-compiler, something like "aarch64-linux-gnu-" ?

No, I was building everything using a native compiler instead of a cross 
compiler.
All packages were compiled on a NVidia's grace-hopper machine.

[root@nvidia-grace-hopper-05 ~]# cat /etc/system-release
Red Hat Enterprise Linux release 9.5 Beta (Plow)
[root@nvidia-grace-hopper-05 ~]# uname -r
6.7.0-rc2-gavin+
[root@nvidia-grace-hopper-05 ~]# gcc --version
gcc (GCC) 11.4.1 20231218 (Red Hat 11.4.1-3)
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

I tried the cross compiler and encountered the same build error.

[root@nvidia-grace-hopper-05 edk2-guest]# export | grep GCC5_AARCH64_PREFIX
declare -x GCC5_AARCH64_PREFIX="aarch64-linux-gnu-"
[root@nvidia-grace-hopper-05 edk2-guest]# build -b DEBUG -a AARCH64 -t GCC5 -p 
ArmVirtPkg/ArmVirtQemu.dsc
  :
--add-gnu-debuglink=/home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore/DEBUG/ArmPlatformPrePeiCore.debug
 
/home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore/DEBUG/ArmPlatformPrePeiCore.dll
cp -p -f 
/home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore/DEBUG/ArmPlatformPrePeiCore.debug
 
/home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/ArmPlatformPrePeiCore.debug
"GenFw" -e SEC -o 
/home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore/OUTPUT/ArmPlatformPrePeiCore.efi
 
/home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore/DEBUG/ArmPlatformPrePeiCore.dll
GenFw: ERROR 3000: Invalid
  WriteSections64(): 
/home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore/DEBUG/ArmPlatformPrePeiCore.dll
 AARCH64 small code model requires identical ELF and PE/COFF section offsets 
modulo 4 KB.
GenFw: ERROR 3000: Invalid
  :

> >    build -b DEBUG -a AARCH64 -t GCC5 -p ArmVirtPkg/ArmVirtQemu.dsc
> >     :
> >    WriteSections64(): 
> > /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore/DEBUG/ArmPlatformPrePeiCore.dll
> >  AARCH64 small code model requires identical ELF and PE/COFF section offsets 
> > modulo 4 KB.
> > cp -p -f 
> > /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/OvmfPkg/VirtioFsDxe/VirtioFsDxe/DEBUG/VirtioFsDxe.dll
> >  
> > /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/OvmfPkg/VirtioFsDxe/VirtioFsDxe/DEBUG/VirtioFsDxe.debug
> > cp -p -f 
> > /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/MdeModulePkg/Universal/Disk/PartitionDxe/PartitionDxe/DEBUG/PartitionDxe.debug
> >  
> > /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/PartitionDxe.debug
> > "gcc" -MMD -MF 
> > /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/CryptoPkg/Library/OpensslLib/OpensslLibCrypto/OUTPUT/openssl/crypto/asn1/x_sig.obj.deps
> >  
> > @/home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/CryptoPkg/Library/OpensslLib/OpensslLibCrypto/OUTPUT/cc_resp.txt
> >   -c -o 
> > /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/CryptoPkg/Library/OpensslLib/OpensslLibCrypto/OUTPUT/openssl/crypto/asn1/x_sig.obj
> >   
> > /home/gavin/sandbox/CCA/edk2-guest/CryptoPkg/Library/OpensslLib/openssl/crypto/asn1/x_sig.c
> > "GenFw" -e DXE_CORE -o 
> > /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/MdeModulePkg/Core/Dxe/DxeMain/OUTPUT/DxeCore.efi
> >  
> > /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll
> > GenSec -s EFI_SECTION_USER_INTERFACE -n ArmCpuDxe -o 
> > /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/FV/Ffs/B8D9777E-D72A-451F-9BDB-BAFB52A68415ArmCpuDxe/B8D9777E-D72A-451F-9BDB-BAFB52A68415SEC3.ui
> > cp -p -f 
> > /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/MdeModulePkg/Universal/DisplayEngineDxe/DisplayEngineDxe/DEBUG/*.map
> >  
> > /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/MdeModulePkg/Universal/DisplayEngineDxe/DisplayEngineDxe/OUTPUT
> > cp -p -f 
> > /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/MdeModulePkg/Universal/Disk/UdfDxe/UdfDxe/OUTPUT/UdfDxe.efi
> >  
> > /home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/MdeModulePkg/Universal/Disk/UdfDxe/UdfDxe/DEBUG
> > GenFw: ERROR 3000: Invalid
> >    :
> > build.py...
> >   : error 7000: Failed to execute command
> >         make tbuild 
> > [/home/gavin/sandbox/CCA/edk2-guest/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore]
> >
> >
> > build.py...
> >   : error F002: Failed to build module
> >         
> > /home/gavin/sandbox/CCA/edk2-guest/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore.inf
> >  [AARCH64, GCC5, DEBUG]
> >
> > - Failed -
> > Build end time: 05:42:19, Jun.01 2024
> > Build total time: 00:00:31
> >
> > ---> Use the edk2 image from the latest QEMU source
>
> Unfortunately this can't work at the moment because edk2 needs several
> changes in order to run in a Realm
>
> > # cd /home/gavin/sandbox/CCA
> > # cp /home/gavin/sandbox/qemu.main/build/pc-bios/edk2-aarch64-code.fd ./
> >
> > ---> Start the guest and no output from the console
> >
> > host# mount | grep 9p
> > shr0 on /mnt/shr0 type 9p (rw,relatime,access=client,trans=virtio)
> > host# export SHR_DIR="/mnt/shr0"
> > host# qemu-system-aarch64 -accel kvm                              \
> >        -machine virt,gic-version=3,confidential-guest-support=rme0 \
> >        -cpu host -smp 2 -m 512M                                    \
> >        -object 'rme-guest,id=rme0,measurement-algo=sha512'         \
> >        -monitor none -serial mon:stdio -nographic                  \
> >        -bios /mnt/edk2-aarch64-code.fd                             \
> >        -kernel ${SHR_DIR}/linux/arch/arm64/boot/Image              \
> >        -initrd ${SHR_DIR}/buildroot/output/images/rootfs.cpio      \
> >        -append 'console=ttyAMA0'
> >          :
> >        <no output from the console>
> >          :
> >        (QEMU) q
> >
> > There are some messages from host's console indicating RMI/RMM servicing
> > states when the guest is running at background. After the guest is terminated,
> > the host crashes.
> >
> > SMC_RMM_RTT_CREATE            102dff000 122c2e000 1e00000 3 > RMI_SUCCESS
> > SMC_RMM_RTT_CREATE            102dff000 1234a7000 2000000 3 > RMI_SUCCESS
> > SMC_RMM_RTT_CREATE            102dff000 1235bd000 2200000 3 > RMI_SUCCESS
> > SMC_RMM_RTT_CREATE            102dff000 12387c000 2400000 3 > RMI_SUCCESS
> > SMC_RMM_RTT_CREATE            102dff000 123a5a000 2600000 3 > RMI_SUCCESS
> > SMC_RMM_RTT_CREATE            102dff000 12407d000 2800000 3 > RMI_SUCCESS
> > SMC_RMM_RTT_CREATE            102dff000 124109000 2a00000 3 > RMI_SUCCESS
> > SMC_RMM_RTT_CREATE            102dff000 123e49000 2c00000 3 > RMI_SUCCESS
> > SMC_RMM_RTT_CREATE            102dff000 124275000 2e00000 3 > RMI_SUCCESS
> > SMC_RMM_RTT_CREATE            102dff000 123138000 3000000 3 > RMI_SUCCESS
> > SMC_RMM_RTT_CREATE            102dff000 124d07000 3200000 3 > RMI_SUCCESS
> >   :
> >   :
> > [22768.994481] rcu: INFO: rcu_preempt self-detected stall on CPU
>
> This is a warning rather than a crash. The current KVM patches spend too
> much time tearing down guest page tables (SMC calls to RMM) while holding
> the mmu lock. Not very nice but harmless.

Ok, I can look into this deeply after I can bring up the guest successfully.

Thanks,
Gavin