Re: Getting For Loop to use 7 of my 8 cores

[Olaf Till]

On Mon, Jun 22, 2015 at 08:29:42AM +0200, Olaf Till wrote:
> On Sun, Jun 21, 2015 at 11:05:29PM -0400, RT wrote:
> > I have 8 cores and I'm trying to use 7 of them to compute a For loop
> > I know parfor is just a for loop command in 3.8.1 Octave and I looked at
> > the examples but they are very sparse.
> > http://wiki.octave.org/Parallel_package.
> > 
> > What I've tried:
> > 1) I've tried vectorizing the for loop which runs out of memory due to the
> > large arrays. (100000x6)
> > 2) I've tried vectorizing the loop and splitting them into smaller sections
> > and appending the arrays together as not to run out of memory but the for
> > loop ends up being faster.
> > 
> > So I'm trying to see how I can use the Parallel Package and use 7 of my 8
> > cores to increase speed.
> > Any ideas how to convert this code into a form that can work with the
> > Parallel Package?
> 
> I'm afraid there is no way at the moment. 'parcellfun' or
> 'pararrayfun' would compute each term and you would have to sum them
> up afterwards. This causes the same memory problem as vectorization.
> 
> Such a problem came up on this list before, I believe. Maybe we should
> enhance 'parcellfun' to optionally perform some accumulating operation
> on the results, like adding, instead of returning them all.

I've patched parcellfun/pararrayfun to optionally compute only one
cumulative result. No release yet, the change is in the
repository. Tell me if you need instructions how to get it.

Olaf

-- 
public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net

signature.asc
Description: Digital signature