[Qemu-commits] [qemu/qemu] ed26b9: numa: Fix off-by-one error at MAX_CPUMASK_BITS che...

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: ed26b92290768818371fbfd4317988eab6009ad5
      
https://github.com/qemu/qemu/commit/ed26b92290768818371fbfd4317988eab6009ad5
  Author: Eduardo Habkost <address@hidden>
  Date:   2015-03-19 (Thu, 19 Mar 2015)

  Changed paths:
    M numa.c

  Log Message:
  -----------
  numa: Fix off-by-one error at MAX_CPUMASK_BITS check

Fix the CPU index check to ensure we don't go beyond the size of the
node_cpu bitmap.

CPU index is always less than MAX_CPUMASK_BITS, as documented at
sysemu.h:

> The following shall be true for all CPUs:
>   cpu->cpu_index < max_cpus <= MAX_CPUMASK_BITS

Reviewed-by: Igor Mammedov <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 8979c945c1a7ffd20edbd5da2513c04baccfd7de
      
https://github.com/qemu/qemu/commit/8979c945c1a7ffd20edbd5da2513c04baccfd7de
  Author: Eduardo Habkost <address@hidden>
  Date:   2015-03-19 (Thu, 19 Mar 2015)

  Changed paths:
    M numa.c

  Log Message:
  -----------
  numa: Reject CPU indexes > max_cpus

CPU index is always less than max_cpus, as documented at sysemu.h:

> The following shall be true for all CPUs:
>   cpu->cpu_index < max_cpus <= MAX_CPUMASK_BITS

Reject configuration which uses invalid CPU indexes.

Reviewed-by: Igor Mammedov <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 3ef7197505e483e2f28c5fbd6ed54b4061221200
      
https://github.com/qemu/qemu/commit/3ef7197505e483e2f28c5fbd6ed54b4061221200
  Author: Eduardo Habkost <address@hidden>
  Date:   2015-03-19 (Thu, 19 Mar 2015)

  Changed paths:
    M numa.c

  Log Message:
  -----------
  numa: Reject configuration if CPU appears on multiple nodes

Each CPU can appear in only one NUMA node on the NUMA config. Reject
configuration if a CPU appears in multiple nodes.

Reviewed-by: Igor Mammedov <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 57924bcd87cb03cc21ebd7efed880d16ca048dce
      
https://github.com/qemu/qemu/commit/57924bcd87cb03cc21ebd7efed880d16ca048dce
  Author: Igor Mammedov <address@hidden>
  Date:   2015-03-19 (Thu, 19 Mar 2015)

  Changed paths:
    M include/hw/boards.h
    M include/sysemu/numa.h
    M numa.c
    M vl.c

  Log Message:
  -----------
  numa: introduce machine callback for VCPU to node mapping

Current default round-robin way of distributing VCPUs among
NUMA nodes might be wrong in case on multi-core/threads
CPUs. Making guests confused wrt topology where cores from
the same socket are on different nodes.

Allow a machine to override default mapping by providing
 MachineClass::cpu_index_to_socket_id()
callback which would allow it group VCPUs from a socket
on the same NUMA node.

Signed-off-by: Igor Mammedov <address@hidden>
Reviewed-by: Andreas Färber <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: fb43b73b9225ff2d19cf5350c68112aade7eec13
      
https://github.com/qemu/qemu/commit/fb43b73b9225ff2d19cf5350c68112aade7eec13
  Author: Igor Mammedov <address@hidden>
  Date:   2015-03-19 (Thu, 19 Mar 2015)

  Changed paths:
    M hw/i386/pc.c

  Log Message:
  -----------
  pc: fix default VCPU to NUMA node mapping

Since commit
   dd0247e0 pc: acpi: mark all possible CPUs as enabled in SRAT
Linux kernel actually tries to use CPU to Node mapping from
QEMU provided SRAT table instead of discarding it, and that
in some cases breaks build_sched_domains() which expects
sane mapping where cores/threads belonging to the same socket
are on the same NUMA node.

With current default round-robin mapping of VCPUs to nodes
guest ends-up with cores/threads belonging to the same socket
being on different NUMA nodes.

For example with following CLI:

   qemu-system-x86_64 -m 4G \
   -cpu Opteron_G3,vendor=AuthenticAMD \
   -smp 5,sockets=1,cores=4,threads=1,maxcpus=8 \
   -numa node,nodeid=0 -numa node,nodeid=1

2.6.32 based kernels will hang on boot due to incorrectly built
sched_group-s list in update_sd_lb_stats()

Replacing default mapping with a manual, where VCPUs belonging to
the same socket are on the same NUMA node, fixes the issue for
guests which can't handle nonsense topology i.e. changing CLI to:
  -numa node,nodeid=0,cpus=0-3 -numa node,nodeid=1,cpus=4-7

So instead of simply scattering VCPUs around nodes, provide
callback to map the same socket VCPUs to the same NUMA node,
which is what guests would expect from a sane hardware/BIOS.

Signed-off-by: Igor Mammedov <address@hidden>
Reviewed-by: Andreas Färber <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 549fc54b8cfe16a475d8f6b8f838e53b45452b4a
      
https://github.com/qemu/qemu/commit/549fc54b8cfe16a475d8f6b8f838e53b45452b4a
  Author: Eduardo Habkost <address@hidden>
  Date:   2015-03-19 (Thu, 19 Mar 2015)

  Changed paths:
    M numa.c

  Log Message:
  -----------
  numa: Print warning if no node is assigned to a CPU

We need all possible CPUs (including hotplug ones) to be present in the
SRAT when QEMU starts. QEMU already does that correctly today, the only
problem is that when a CPU is omitted from the NUMA configuration, it is
silently assigned to node 0.

Check if all CPUs up to max_cpus are present in the NUMA configuration
and warn about missing CPUs.

Make it just a warning, to allow management software to be updated if
necessary. In the future we may make it a fatal error instead.

Command-line examples:

* Correct, no warning:

  $ qemu-system-x86_64 -smp 2,maxcpus=4
  $ qemu-system-x86_64 -smp 2,maxcpus=4 -numa node,cpus=0-3

* Incomplete, with warnings:

  $ qemu-system-x86_64 -smp 2,maxcpus=4 -numa node,cpus=0
  qemu-system-x86_64: warning: CPU(s) not present in any NUMA nodes: 1 2 3
  qemu-system-x86_64: warning: All CPU(s) up to maxcpus should be described in 
NUMA config

  $ qemu-system-x86_64 -smp 2,maxcpus=4 -numa node,cpus=0-2
  qemu-system-x86_64: warning: CPU(s) not present in any NUMA nodes: 3
  qemu-system-x86_64: warning: All CPU(s) up to maxcpus should be described in 
NUMA config

Signed-off-by: Eduardo Habkost <address@hidden>
---
v1 -> v2: (no changes)

v2 -> v3:
 * Use enumerate_cpus() and error_report() for error message
 * Simplify logic using bitmap_full()

v3 -> v4:
 * Clarify error message, mention that all CPUs up to
   maxcpus need to be described in NUMA config

v4 -> v5:
 * Commit log update, to make problem description clearer


  Commit: 4eef86486d4090d7587e94d3f1a2203b94899989
      
https://github.com/qemu/qemu/commit/4eef86486d4090d7587e94d3f1a2203b94899989
  Author: Peter Maydell <address@hidden>
  Date:   2015-03-20 (Fri, 20 Mar 2015)

  Changed paths:
    M hw/i386/pc.c
    M include/hw/boards.h
    M include/sysemu/numa.h
    M numa.c
    M vl.c

  Log Message:
  -----------
  Merge remote-tracking branch 
'remotes/ehabkost/tags/work/numa-verify-cpus-pull-request' into staging

NUMA queue 2015-03-19

# gpg: Signature made Thu Mar 19 19:25:53 2015 GMT using RSA key ID 984DC5A6
# gpg: Can't check signature: public key not found

* remotes/ehabkost/tags/work/numa-verify-cpus-pull-request:
  numa: Print warning if no node is assigned to a CPU
  pc: fix default VCPU to NUMA node mapping
  numa: introduce machine callback for VCPU to node mapping
  numa: Reject configuration if CPU appears on multiple nodes
  numa: Reject CPU indexes > max_cpus
  numa: Fix off-by-one error at MAX_CPUMASK_BITS check

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/e7e9b49f8e9e...4eef86486d40