[Qemu-discuss] High host CPU load and slow Windows 10 vm after upgrade to 1803

[Giovanni Panozzo]

Hi to all, I'm new in this ML.

After upgrading some VM from Windows 10 1709 to Windows 10 1803, the VMs 
runs slower. And when VM is almost idle, host CPU load is quite high.

It happens on 4 different hardware platform (AMD FX 4300 and intel core 
i3/i5), with Arch and ubuntu 16.04/18.04 with libvirt. I already asked 
help on askubuntu.com and opened a bug to quemu with non answers.

So I continued my investigation, but it's very difficult for me to 
investigate, having limited time and knowledge on Kvm/Qemu.

perf kvm --host stat live reports:

Analyze events for all VMs, all VCPUs:

             VM-EXIT    Samples  Samples%     Time% Min Time    Max 
Time         Avg time

      IO_INSTRUCTION      17379    54.45%    49.37% 4.73us   
5274.05us     40.94us ( +-   1.89% )
            MSR_READ       5382    16.86%     1.56% 2.24us   
2126.01us      4.17us ( +-  12.26% )
       EPT_VIOLATION       3183     9.97%     3.63% 2.83us   
8829.17us     16.44us ( +-  24.23% )
           MSR_WRITE       2425     7.60%     0.80% 3.12us    
220.26us      4.77us ( +-   1.96% )
  EXTERNAL_INTERRUPT       1464     4.59%     3.05% 1.99us   
7080.61us     29.98us ( +-  26.92% )
   PENDING_INTERRUPT        999     3.13%     0.29% 2.87us      
7.12us      4.13us ( +-   0.31% )
                 HLT        662     2.07%    41.16% 2.75us   
7956.90us    895.99us ( +-   2.27% )
 TPR_BELOW_THRESHOLD        220     0.69%     0.08% 3.61us     
94.16us      5.55us ( +-   7.33% )
              VMCALL        171     0.54%     0.05% 2.30us     
58.09us      4.20us ( +-   8.11% )
               CPUID         24     0.08%     0.00% 2.05us      
4.12us      2.84us ( +-   4.05% )
       EPT_MISCONFIG          7     0.02%     0.01% 23.05us     
32.87us     25.97us ( +-   4.98% )

Total Samples:31916, Total events handled time:1441189.06us.

And perf kvm --host stat live --event=ioport


Analyze events for all VMs, all VCPUs:

      IO Port Access    Samples  Samples%     Time% Min Time    Max 
Time         Avg time

           0x70:POUT      11138    49.69%    85.13% 8.60us    
392.56us     30.34us ( +-   0.99% )
            0x71:PIN      11138    49.69%    14.63% 3.80us     
58.95us      5.21us ( +-   0.16% )
         0xc010:POUT        110     0.49%     0.12% 2.65us     
11.67us      4.15us ( +-   4.13% )
          0x1f0:POUT          6     0.03%     0.06% 13.81us     
82.45us     37.23us ( +-  34.19% )
           0x1f7:PIN          4     0.02%     0.01% 4.99us      
6.12us      5.62us ( +-   4.36% )
[...]

On another virtualization host, I noticed also traffic on io port 0x0608:

Analyze events for all VMs, all VCPUs:

      IO Port Access    Samples  Samples%     Time% Min Time    Max 
Time         Avg time

           0x70:POUT       4220    40.00%    90.93% 3.42us   
2023.80us     12.61us ( +-   3.97% )
            0x71:PIN       4220    40.00%     5.70% 0.53us      
9.23us      0.79us ( +-   0.65% )
           0x608:PIN       2074    19.66%     3.15% 0.55us     
10.62us      0.89us ( +-   0.95% )
          0x1f0:POUT          6     0.06%     0.06% 4.64us      
8.51us      5.54us ( +-  11.23% )
         0xc070:POUT          5     0.05%     0.05% 4.06us      
7.09us      5.49us ( +-  10.68% )
           0x1f7:PIN          4     0.04%     0.00% 0.65us      
0.70us      0.68us ( +-   2.10% )
         0xc010:POUT          3     0.03%     0.04% 6.26us      
9.15us      7.40us ( +-  11.99% )

when running older Windows 10 1709 VM, 0x70 and 0x71 IO port Samples 
value are around 120-130 with spikes to 500, on Windows 10 1803, as you 
can see, 4000 to 11000 samples.

Thank you in advance for any help.

Giovanni