On Fri, Aug 21, 2020 at 09:47:47AM -0300, Daniel Henrique Barboza wrote:
On 8/21/20 5:55 AM, Igor Mammedov wrote:
On Thu, 20 Aug 2020 12:51:03 -0400
Eduardo Habkost <ehabkost@redhat.com> wrote:
On Thu, Aug 20, 2020 at 02:15:04PM +1000, David Gibson wrote:
On Wed, Aug 19, 2020 at 10:11:28PM -0400, Eduardo Habkost wrote:
On Thu, Aug 20, 2020 at 11:17:26AM +1000, David Gibson wrote:
On Fri, Aug 14, 2020 at 05:54:16PM -0300, Daniel Henrique Barboza wrote:
The pSeries machine does not support asymmetrical NUMA
configurations.
This seems a bit oddly specific to have as a global machine class
property.
Would it make more sense for machines with specific NUMA constraints
to just verify those during their initialization?
This would be much simpler. However, I like the idea of
representing machine-specific configuration validation rules as
data that can eventually be exported to management software.
Ah, ok, so basically the usual tradeoff between flexibility and
advertisability.
So, in that case, I guess the question is whether we envisage "no
assymmetry" as a constraint common enough that it's worth creating an
advertisable rule or not. If we only ever have one user, then we
haven't really done any better than hard coding the constraint in the
manageent software.
Of course to complicate matters, in the longer term we're looking at
removing that constraint from pseries - but doing so will be dependent
on the guest kernel understanding a new format for the NUMA
information in the device tree. So qemu alone won't have enough
information to tell if such a configuration is possible or not.
Requiring both QEMU (and possibly management software) to be
patched again after the guest kernel is fixed sounds undesirable.
If we drop this restriction, then we don't need to touch QEMU when
guest kernel is ready.
Btw, what spapr spec says about the matter?
LOPAPR support a somewhat asymmetrical NUMA setup in its current
form,
Huh, I didn't even realize that. What's the mechanism?
but
the Linux kernel doesn't support it. The effort to implement it in the current
spapr machine code, given that Linux wouldn't mind it, is not worth it. This
is why I chose to invalidate it for pseries.
Igor,
It's kind of difficult to answer that question - PAPR doesn't
specifically describe limitations, it's just that the representation
it uses is inherently limited. Instead of the obvious, simple and
pretty much universal method (used in the generic kernel and qemu) of
having a matrix of distance between all the nodes, it instead
describes the hierarchy of components that give rise to the different
distances.
So, for each NUMA relevant object (cpu, memory block, host bridge,
etc.) there is a vector of IDs. Each number in the vector gives one
level of the objects location in the heirarchy.
So, for example the first number might be the physical chip/socket.
the second one which group of cores & memory interfaces sharing an Ln
cache, the third one the specific core number. So to work out how far
objects are from each other you essentially look at how long a prefix
of their vector they share, which tells you how far above in the
hierarchy you have to go to reach it.
There's a bunch of complicating details, but that's the gist of it.
Perhaps a warning would be better in this case?
In either case, it sounds like this won't be a common constraint
and I now agree with your original suggestion of doing this in
machine initialization code.
Agreed, if it goes to spapr specific machine code I will not object much.
(it will burden just spapr maintainers, so it's about convincing
David in the end)
I believe he's ok with it given that he suggested it in his first reply.
I'll move this verification to spapr machine_init in the next version.
Thanks,
DHB