Re: GNU Parallel Bug Reports Auto-detection of available CPU cores

[Ole Tange]

On Mon, Feb 23, 2015 at 4:58 PM, Johannes Dröge
<address@hidden> wrote:

> I found another issue with the CPU detection on recent Linux:
> The number of jobs should be set to the number of _usable_ cores, which does 
> not seem to be detected correctly on recent Linux systems. For instance, try 
> the following:
>
> while true; do echo; done | taskset -c 1 parallel 'cat /dev/zero > /dev/null'&

Yep. I can see that will go wrong.

> Here is a command line which is not really nice but works for the detection 
> of allowed CPU cores:
>
> grep -m 1 '^Cpus_allowed:' /proc/self/status | cut -f 2 | tr -d ',' | tr 
> '[:lower:]' '[:upper:]' | xargs echo "obase=2; ibase=16;" | bc | tr -d -c '1' 
> | wc -c

I just tested this on a 32-core and a 64-core machine:

$ ssh cpu32 cat  /proc/self/status |grep -m 1 '^Cpus_allowed:'
Cpus_allowed:   ffffffff,ffffffff
$ ssh cpu64 cat  /proc/self/status |grep -m 1 '^Cpus_allowed:'
Cpus_allowed:   ffffffff,ffffffff

It does the right thing if you prepend with 'taskset ffff...', so it
should be possible to do min(existing_code,your_code).

Is this a real problem or is it a theoretical problem?


/Ole