Re: Parallel octave ??

[Javier Fernández]

address@hidden wrote:
> Today's Topics:
> [...]
>    4. Re: Parallel octave ?? (Bill Denney)
> [...]
>
> ------------------------------------------------------------------------
>
> Subject:
> Re: Parallel octave ??
> From:
> Bill Denney <address@hidden>
> =?ISO-8859-1?Q?Javier_Fern=E1ndez_ wrote:
>   
> > On the other hand, I found mpirun easier to use with Open-MPI, since no 
> > daemon has to be previously started [...]
> >     
>
> Hi Javier,
>
> I'm curious how easy it would be to modify this to write a parfor
> command?  I'm probably completely off on this, but it sounds like this
> could translate into parfor without huge issues.
>
> Thanks,
>
> Bill
>   
Well, Octave is GPL'ed so anybody has the freedom to do that, if they 
want. FWIW, my advice would be not to. I have Googled for parfor, and 
after reading the first two pages, I see some issues :-)

0.- parfor is a grammatical structure of the language. It's not a 
command. You cannot implement it as a function. It's the kind of 
reserved word that matches a closing "end". You better talk to JWE and 
collaborators to get your parser patch applied. For me, that qualifies 
as "huge" issue, ie: I won't do the job :-)

1.- The very moment you provide a parfor, you'll have users asking for 
the corresponding matlabpool. I'm not sure they would be satisfied with 
an octavepool command, probably they'll require matlabpool :-)

2.- Particularly, the idea of having an XXX-pool command would be more 
appropriate if/when using LAM/MPI which could easily lamboot and mpirun 
on the fly from a pure sequential Octave session. When you replied, I 
was just explaining that I found easier not to do that under OMPI. 
Probably now OMPI can do that too, after the 1.3.x series and the list 
discussion I showed last week, but I have not tried it yet.

3.- OMPI is the way to go. In MPITB "readme" file I commented on the 
difficulties I had trying to convince users to setup their clusters so 
that they could send back terms to debug their parallel applications... 
it soon becomes a nightmare, trying to MPI_Comm_spawn("xterm",...), it's 
easier to mpirun. I even deleted the "tutorial" file, where no mpirun 
was used, everything was done with MPI_Comm_spawn("xterm",...). Now 
MPITB has demos, which are to be executed using mpirun. OMPI is the way 
to go. mpirun is the way to go. Let "spawning on the fly" for advanced 
users. In fact, MPI_Comm_spawn was only included in the standard after 
many many discussions (and there are those who believe that it was due 
to pvm_spawn() and bad user habits, not to a sound rationale :-)

4.- I hope I'm not doing anything illegal if I quote the following from 
the second link to TMW in the Google search, which better illustrates 
the grammatical issues of parfor (and its semantics :-)
> parfor i = 1:n
>    if p(i)   % assume p is a function
>       s = s + 1;
>    end
> end
>
> The key point of this example is that the conditional adding of 1 to s 
> can be done in any order. After the parfor statement has finished 
> executing, the value of s depends only on the number of iterations for 
> which p(i) is true. As long as p(i) depends only upon i, the value of 
> s is deterministic. This technique generalizes to functions other than 
> plus (+).
I suppose that "s=s+int(p(i))" does not generalize, ie, the technique 
refuses to trying to understand arbitrary code, and has a pre-defined 
casuistic of what can and cannot be done.

The example above this one (in the original TMW page) is also 
interesting. They both prove that some (non-obvious) gramatical analysis 
is required to implement the desired functionality. Some part inside 
Octave (the one analyzing the parfor command) will need to know and 
decide a lot of things:
a) decide the "control index variable" values to be assigned to each host
b) do some index translation (better understood in the other example) so 
you won't need to repeat the same storage allocation on each host. Ie, 
if you are computing a 1GB array in parallel on N "workers", it would be 
nice you needed 1GB/N storage on each, not 1GB on each

parfor i = 1:n;   t = f(A(i));   u = g(B(i));   C(i) = h(t, u); end

c) do some Frank-ssembly to get the results back, and according to this 
other example, 3 cases are in order
- scalars assigned (before used: i,t,u) inside the loop: no need to send 
or receive them. That includes the index.
- arrays with indexing based in the control var (C(i), or C(3*i^2)): 
They must be assembled back.
- scalars (like s in the 1st example) "known" to have a predictable 
final value: They must be somehow reduced back.

The idea behind MPITB is that the final users knows better.
The idea behind parfor seems to be that final users don't want to know 
there is a cluster (or multiple workers) behind parfor... well, they 
only need to know that indexes must be ascending row vector of 
consecutive indexes, if they are to be iterated with parfor. And that 
they shouldn't expect scalars to keep their "last assigned value" in the 
loop (which is rather sensible :-). And other things I'm not sure I 
foresee them all ;-)

Again, that's my 2p. Octave is GPL, so anyone can play with the code. A 
different issue is getting the patch commited, which renders JWE and/or 
collaborators responsible for its maintenance. parfor has more to do 
with Octave parser than with MPITB, really.

-javier