Re: GNU Parallel Bug Reports Auto-detection of available CPU cores

[Johannes Dröge]

Am 23.02.2015 um 21:48 schrieb Ole Tange:
> I just tested this on a 32-core and a 64-core machine:
>
> $ ssh cpu32 cat  /proc/self/status |grep -m 1 '^Cpus_allowed:'
> Cpus_allowed:   ffffffff,ffffffff
> $ ssh cpu64 cat  /proc/self/status |grep -m 1 '^Cpus_allowed:'
> Cpus_allowed:   ffffffff,ffffffff
>
> It does the right thing if you prepend with 'taskset ffff...', so it
> should be possible to do min(existing_code,your_code).

I don't quite get what you mean but the calculation of the mask should 
be somewhere in the kernel docs and it should be valid for all 
architectures. You could probably port this code to Perl easily.

> Is this a real problem or is it a theoretical problem? 

When you are using CPU pinning with taskset, which seems to be the new 
standard, this will be a problem. For example, I believe containers like 
Docker see the full set of cores but will be restricted to a subset 
using the cpuset functionality. So if you run parallel in such 
containers, auto-detection will fail.

Gruß Johannes

--
Johannes Dröge, M.Sc.
Algorithmic Bioinformatics, Heinrich Heine University Düsseldorf
25.12.01.50, Universitätsstraße 1, 40225 Düsseldorf, Germany
PGP: http://keys.fungs.de/6ea5e4.asc (55F2720303A7F236A94666F20E2360727A6EA5E4)
Web: algbio.cs.uni-duesseldorf.de | Tel/Fax: +49 211 81-12644/13464