[Bug 1920784] Re: qemu-system-ppc64le fails with kvm acceleration

[Laurent Vivier]

You need a kernel with a the following fix for POWER9:

commit 25edcc50d76c834479d11fcc7de46f3da4d95121
Author: Fabiano Rosas <farosas@linux.ibm.com>
Date:   Thu Feb 4 17:05:17 2021 -0300

    KVM: PPC: Book3S HV: Save and restore FSCR in the P9 path
    
    The Facility Status and Control Register is a privileged SPR that
    defines the availability of some features in problem state. Since it
    can be written by the guest, we must restore it to the previous host
    value after guest exit.
    
    This restoration is currently done by taking the value from
    current->thread.fscr, which in the P9 path is not enough anymore
    because the guest could context switch the QEMU thread, causing the
    guest-current value to be saved into the thread struct.
    
    The above situation manifested when running a QEMU linked against a
    libc with System Call Vectored support, which causes scv
    instructions to be run by QEMU early during the guest boot (during
    SLOF), at which point the FSCR is 0 due to guest entry. After a few
    scv calls (1 to a couple hundred), the context switching happens and
    the QEMU thread runs with the guest value, resulting in a Facility
    Unavailable interrupt.
    
    This patch saves and restores the host value of FSCR in the inner
    guest entry loop in a way independent of current->thread.fscr. The old
    way of doing it is still kept in place because it works for the old
    entry path.
    
    Signed-off-by: Fabiano Rosas <farosas@linux.ibm.com>
    Signed-off-by: Paul Mackerras <paulus@ozlabs.org>

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1920784

Title:
  qemu-system-ppc64le fails with kvm acceleration

Status in QEMU:
  New
Status in The Ubuntu-power-systems project:
  New
Status in glibc package in Ubuntu:
  New
Status in qemu package in Ubuntu:
  Confirmed

Bug description:
  (Suspected glibc issue!)

  qemu-system-ppc64(le) fails when invoked with kvm acceleration with
  error "illegal instruction"

  > qemu-system-ppc64(le) -M pseries,accel=kvm

  Illegal instruction (core dumped)

  In dmesg:

  Facility 'SCV' unavailable (12), exception at 0x7624f8134c0c,
  MSR=900000000280f033

  
  Version-Release number of selected component (if applicable):
  qemu 5.2.0 
  Linux kernel 5.11
  glibc 2.33
  all latest updates as of submitting the bug report

  How reproducible:
  Always

  Steps to Reproduce:
  1. Run qemu with kvm acceleration

  Actual results:
  Illegal instruction

  Expected results:
  Normal VM execution

  Additional info:
  The machine is a Raptor Talos II Lite with a Sforza V1 8-core, but was also 
observed on a Raptor Blackbird with the same processor.

  This was also observed on Fedora 34 beta, which uses glibc 2.33
  Also tested on ArchPOWER (unofficial port of Arch Linux for ppc64le) with 
glibc 2.33
  Fedora 33 and Ubuntu 20.10, both using glibc 2.32 do not have this issue, and 
downgrading the Linux kernel from 5.11 to 5.4 LTS on ArchPOWER solved the 
problem. Kernel 5.9 and 5.10 have the same issue when combined with glibc2.33

  ProblemType: Bug
  DistroRelease: Ubuntu 21.04
  Package: qemu-system 1:5.2+dfsg-6ubuntu2
  ProcVersionSignature: Ubuntu 5.11.0-11.12-generic 5.11.0
  Uname: Linux 5.11.0-11-generic ppc64le
  .sys.firmware.opal.msglog: Error: [Errno 13] Permission denied: 
'/sys/firmware/opal/msglog'
  ApportVersion: 2.20.11-0ubuntu60
  Architecture: ppc64el
  CasperMD5CheckResult: pass
  CurrentDesktop: Unity:Unity7:ubuntu
  Date: Mon Mar 22 14:48:39 2021
  InstallationDate: Installed on 2021-03-22 (0 days ago)
  InstallationMedia: Ubuntu-Server 21.04 "Hirsute Hippo" - Alpha ppc64el 
(20210321)
  KvmCmdLine: COMMAND         STAT  EUID  RUID     PID    PPID %CPU COMMAND
  ProcKernelCmdLine: root=UUID=f3d03315-0944-4a02-9c87-09c00eba9fa1 ro
  ProcLoadAvg: 1.20 0.73 0.46 1/1054 6071
  ProcSwaps:
   Filename                             Type            Size            Used    
        Priority
   /swap.img                               file         8388544         0       
        -2
  ProcVersion: Linux version 5.11.0-11-generic (buildd@bos02-ppc64el-002) (gcc 
(Ubuntu 10.2.1-20ubuntu1) 10.2.1 20210220, GNU ld (GNU Binutils for Ubuntu) 
2.36.1) #12-Ubuntu SMP Mon Mar 1 19:26:20 UTC 2021
  SourcePackage: qemu
  UpgradeStatus: No upgrade log present (probably fresh install)
  VarLogDump_list: total 0
  acpidump:
   
  cpu_cores: Number of cores present = 8
  cpu_coreson: Number of cores online = 8
  cpu_smt: SMT=4

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1920784/+subscriptions