Re: About the performance of hyper-v

[Vitaly Kuznetsov]

Liang Li <liliang324@gmail.com> writes:

>> > Hi Vitaly,
>> >
>> > I found a case that the virtualization overhead was almost doubled
>> > when turning on Hper-v related features compared to that without any
>> > no hyper-v feature.  It happens when running a 3D game in windows
>> > guest in qemu kvm environment.
>> >
>> > By investigation, I found there are a lot of IPIs triggered by guest,
>> > when turning on the hyer-v related features including stimer, for the
>> > apicv is turned off, at least two vm exits are needed for processing a
>> > single IPI.
>> >
>> >
>> > perf stat will show something like below [recorded for 5 seconds]
>> >
>> > ---------
>> >
>> > Analyze events for all VMs, all VCPUs:
>> >              VM-EXIT    Samples  Samples%     Time%    Min Time    Max
>> > Time         Avg time
>> >   EXTERNAL_INTERRUPT     471831    59.89%    68.58%      0.64us
>> > 65.42us      2.34us ( +-   0.11% )
>> >            MSR_WRITE     238932    30.33%    23.07%      0.48us
>> > 41.05us      1.56us ( +-   0.14% )
>> >
>> > Total Samples:787803, Total events handled time:1611193.84us.
>> >
>> > I tried turning off hyper-v for the same workload and repeat the test,
>> > the overall virtualization overhead reduced by about of 50%:
>> >
>> > -------
>> >
>> > Analyze events for all VMs, all VCPUs:
>> >
>> >              VM-EXIT    Samples  Samples%     Time%    Min Time    Max
>> > Time         Avg time
>> >           APIC_WRITE     255152    74.43%    50.72%      0.49us
>> > 50.01us      1.42us ( +-   0.14% )
>> >        EPT_MISCONFIG      39967    11.66%    40.58%      1.55us
>> > 686.05us      7.27us ( +-   0.43% )
>> >            DR_ACCESS      35003    10.21%     4.64%      0.32us
>> > 40.03us      0.95us ( +-   0.32% )
>> >   EXTERNAL_INTERRUPT       6622     1.93%     2.08%      0.70us
>> > 57.38us      2.25us ( +-   1.42% )
>> >
>> > Total Samples:342788, Total events handled time:715695.62us.
>> >
>> > For this scenario,  hyper-v works really bad.  stimer works better
>> > than hpet, but on the other hand, it relies on SynIC which has
>> > negative effects for IPI intensive workloads.
>> > Do you have any plans for improvement?
>> >
>>
>> Hey,
>>
>> the above can be caused by the fact that when 'hv-synic' is enabled, KVM
>> automatically disables APICv and this can explain the overhead and the
>> fact that you're seeing more vmexits. KVM disables APICv because SynIC's
>> 'AutoEOI' feature is incompatible with it. We can, however, tell Windows
>> to not use AutoEOI ('Recommend deprecating AutoEOI' bit) and only
>> inhibit APICv if the recommendation was ignored. This is implemented in
>> the following KVM patch series:
>> https://lore.kernel.org/kvm/20210518144339.1987982-1-vkuznets@redhat.com/
>>
>> It will, however, require a new 'hv-something' flag to QEMU. For now, it
>> can be tested with 'hv-passthrough'.
>>
>> It would be great if you could give it a spin!
>>
>> --
>> Vitaly
>
> It's great to know that you already have a solution for this. :)
>
> By the way,  is there any requirement for the version of windows or
> windows updates for the new feature to work?

AFAIR, 'Recommend deprecating AutoEOI' bit appeared in WS2012 so I'd
expect WS2008 to ignore it completely (and thus SynIC will always be
disabling APICv for it).

-- 
Vitaly