On Thu, Oct 17, 2024 at 04:18:06PM +0800, Xiaoyao Li wrote:
Date: Thu, 17 Oct 2024 16:18:06 +0800
From: Xiaoyao Li <xiaoyao.li@intel.com>
Subject: Re: [PATCH v6] i386/cpu: fixup number of addressable IDs for
logical processors in the physical package
On 10/14/2024 11:36 AM, Zhao Liu wrote:
On 10/9/2024 11:56 AM, Chuang Xu wrote:
When QEMU is started with:
-cpu host,migratable=on,host-cache-info=on,l3-cache=off
-smp 180,sockets=2,dies=1,cores=45,threads=2
On Intel platform:
CPUID.01H.EBX[23:16] is defined as "max number of addressable IDs for
logical processors in the physical package".
When executing "cpuid -1 -l 1 -r" in the guest, we obtain a
value of 90 for
CPUID.01H.EBX[23:16], whereas the expected value is 128. Additionally,
executing "cpuid -1 -l 4 -r" in the guest yields a value of 63 for
CPUID.04H.EAX[31:26], which matches the expected result.
As (1+CPUID.04H.EAX[31:26]) rounds up to the nearest power-of-2 integer,
we'd beter round up CPUID.01H.EBX[23:16] to the nearest power-of-2
integer too. Otherwise we may encounter unexpected results in guest.
For example, when QEMU is started with CLI above and xtopology
is disabled,
guest kernel 5.15.120 uses CPUID.01H.EBX[23:16]/
(1+CPUID.04H.EAX[31:26]) to
calculate threads-per-core in detect_ht(). Then guest will get
"90/ (1+63)=1"
as the result, even though threads-per-core should actually be 2.
It's kernel's bug instead.
In 1.5.3 "Sub ID Extraction Parameters for initial APIC ID" of
"Intel 64 Architecture Processor Topology Enumeration" [1], it is
- SMT_Mask_Width = Log2(RoundToNearestPof2(CPUID.1:EBX[23:16])/
(CPUID.(EAX=4,ECX=0):EAX[31:26]) + 1))
The value of CPUID.1:EBX[23:16] needs to be *rounded* to the
neartest power-of-two integer instead of itself being the
power-of-two.
This also is consistency with the SDM, where the comment for bit
23-16 of CPUID.1:EBX is:
The nearest power-of-2 integer that is not smaller than EBX[23:16] is
the number of unique initial APIC IDs reserved for addressing
different logical processors in a physical package.
What I read from this is, the nearest power-of-2 integer that is not
smaller than EBX[23:16] is a different thing than EBX[23:16]. i.e.,
Yes, when I read sdm, I also thought it was a kernel bug. But on my
192ht spr host, the value of CPUID.1:EBX[23:16] was indeed rounded up
to the nearest power of 2 by the hardware. After communicating with
Intel technical support staff, we thought that perhaps the description
in sdm
is not so accurate, and rounding up CPUID.1:EBX[23:16] to the power of 2
in qemu may be more in line with the hardware behavior.
I think above justification is important. We need to justify our changes
with the fact and correct reason.
I somehow agree to set EBX[23:16] to a value of power-of-2, because the
1.5.3 "Sub ID Extraction Parameters for initial APIC ID" of "Intel 64
Architecture Processor Topology Enumeration" spec says
CPUID.1:EBX[23:16] represents the maximum number of addressable IDs
(initial APIC ID) that can be assigned to logical processors in a
physical package. The value may not be the same as the number of
logical processors that are present in the hardware of a physical
package.
It uses the word "may not".
IMO, I don't quite understand your confusion regarding this. I've already
explained the meaning of addressable ID, and the spec you referenced also
clarifies its significance. The reason for this modification is not
because of the two words "may not".
Whether it is "be" or "not be" the same as the number of logical
processors, the essence is that due to topology, the actual number of
initial IDs that can be accommodated in the APIC ID may exceed the number
of logical processors.
I have the confusion because no matter from SDM:
Bit 23-16: Maximum number of addressable IDs for logical processors in
this physical package*
* The nearest power-of-2 integer that is not smaller than EBX[23:16]
is the number of unique initial APIC IDs reserved for addressing
different logical processors in a physical package.
or from "Intel 64 Architecture Processor Topology Enumeration" spec，(Jan
2018, revision 1.1), 1.5.3 "sub ID Extraction Parameters for Inital APIC ID"
RoundToNearestPof2(CPUID.1:EBX[23:16])
or from "Intel 64 Architecture Processor Topology Enumeration" spec，(April
2023, revision 2.0), 1.6.1 Legacy Extraction Algorithm
https://cdrdv2-public.intel.com/775917/intel-64-architecture-processor-topology-enumeration.pdf
"MaximumLogicalProcessorIDsPerPackage" is calculated by rounding
CPUID.01H.EBX[23:16] to nearest power of 2.
what I read from them is that EBX[23:16] is a different thing than
pow2ceil(EBX[23:16]) and EBX[23:16] doesn't need to be power-of-2, but the
patch are trying to make it power-of-2.
Yes, no one requires it must be power-of-2. But power-of-2 is just
the result, not the reason.
The core point is not power-of-2, but is the meaning of EBX[23:16].
Sorry, I have to re-emphasize:
Pls remember it's not real number of logical processors per package,
and it's "addressable ID", which is the initial APIC ID. The maximum
capacity of addressable ID is calculated by the APIC layout, and the
final value is “power-of-2”. The calculation by APIC ID or pow2ceil()
are mathematically equivalent. That's the way to get addressable IDs.
The spec is expressed in such a way to help software understands this
value, while the QEMU is designed to emulate hardware behavior.
Then I consult it with Intel internal architect. I was told that EBX[23:16]
used to be that software was to round to the next power of 2. However,
software had issues a long time ago because applications could then compute
the wrong power of 2 based on APIC ID holes or some applications would use
it directly (without round it up to power-of-2).
So intel became to report exact power-of-2 and this behavior is not
documented.
Again, I suggest you think in terms of the meaning of number of
addressable IDs, it's not a matter of how power-of-2 is calculated, you
can choose to calculate number of addressable IDs in other ways, but
the final value is still a power of 2.
-Zhao