[Fsuk-manchester] parallel systems (was: Open ARM GPU drivers)

[Dave Love]

Bob Ham <address@hidden> writes:

> Also interesting, and could be useful.  I find it noteworthy that the
> library targets multi-core CPUs, rather than GPGPUs.  I've viewed OpenMP as
> the de facto method for parallelism on CPUs and OpenCL for GPGPUs but on
> reflection, OpenMP seems only like a quick and easy way to get instant
> parallelism.

If only!

Most HPC is actually done with MPI -- systems are distributed, and it's
typically competitive on shared memory anyway.  MPI implementations are
getting GGPU support, but currently over proprietary libraries, of
course, though there are good free implementations of MPI itself, and
other support.  OpenMP is apparently heading for attached processors
too, but is still shared-memory.

> It makes me wonder what advantages there are to implementing
> algorithms in OpenCL kernels rather than in parallel OpenMP loops, for a
> CPU.  They're quite different environments and OpenCL has a lot of overhead
> just to get a kernel to run.

The systems relevant for HPC are increasingly heterogeneous,
particularly as they get added to over time with the silly rate of
hardware development.  We have three different CPU architectures even in
a new cluster, and you can expect to see a variety of different GGPUs
(even in the same node), Phi, multiple network islands, &c in a running
system.  Even if a particular cluster currently has uniform nodes, in a
flat network, you don't want to write hardware-specific software.

Basically you don't want to work at the CUDA/OpenCL, or even OpenMP,
level if you can reasonably avoid it, and the interesting systems are
networked at multiple levels.  One free software system addressing that
is <http://runtime.bordeaux.inria.fr/StarPU/>.  [Why don't we have an
INRIA?]