[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: distributed octave
From: |
JD Cole |
Subject: |
Re: distributed octave |
Date: |
Sun, 16 Nov 2003 21:34:31 -0800 |
User-agent: |
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.5) Gecko/20031007 |
Hey Jason,
The quick answer is, stick to using the MPI-API as of know. That
code is the least likely to change in the course of developing the high
level parallel commands. It also more stable. (The high level stuff is
very proof of concept right now.)
Below is a more detailed description of what's supposed to happen.
Aso you have probably noticed, there are two types of parallel
interfaces going on here.
(1) The high level interface using the parallel/parallelsection
commands. These are currently intended to give the common user access to
basic parallelism. The idea being that each section of octave script
placed between a parallel/parallelsection command will be distributed to
the next available processing node. For example, let's say I want to
covolve my rather large, big_image, independently with three different
kernels k1, k2, and k3. The script might look something like this:
parallel
k1_out = conv2 (big_image, k1);
parallelsection
k2_out = conv2 (big_image, k2);
parallelsection
k3_out = conv2 (big_image, k3);
end
final_output = k1_out + k2_out + k3_out;
Depending on the number or parallel nodes you have, in this case
specified by MPI, each "section" is sent to a different node to be
processed. Of course, this is how its SUPPOSED to work, however, there
has been very little testing performed on this functionality and it
definitely has some errors.
(2) The low-level API interface. Since the above functionality provides
a level of abstraction to a lower level IPC library, I thought it
appropriate to provide access to that library also. In this case you are
using MPI, so, at present, basic MPI API functions are available. In
the case of writing an MPI-API based script, it should be basically the
same structure as in C, i.e. MPI_Init, _send/recieve, _finish. But
currently, if you want to run that script over multiple nodes, you need
to pass it as a command line parameter. (And if I remember correctly,
this only works with certain flavors of MPI like LAM which pass the
command line parameters to all nodes at startup. There is a script file
for starting MPICH nodes.) I am currently looking at adding a command
which would allow you to call a function or file on all nodes
simultaneously, something like paralleleval, something like this was
done in the octave-mpi distribution.
(NOTE: Although there have been basic tests of mixing the API calls and
high-level parallel commands, I wouldn't encourage it at this point.)
Best,
JD
I have built a small cluster. RedHat 9. LAM 7.0.2. Your patched
Octave-2.1.45. Your mpi_pil...
I am able lamboot the cluster.
I am able to start Octave with the --parallel option.
The processes show up on the other nodes in the cluster
an I get a [node-0] prompt. woohoo. Thanks again.
I am able to run ptest.m (which presumably you wrote) but it ends with a
message
"error: can't perform indexing operations for <unknown type> type" (not sure
what this means yet)
When ptest.m is running, I have confirmed cpu usage using top on the other
nodes in the cluster.
I am unable to figure out how to integrate mpi_send/receive into the
framework.
I also don't understand yet the proper usage of the parallel/parallelsection
keywords.
I did find api_example.m in the mpi_pil source tree, but have not been able
to
distribute it over the nodes in the cluster.
Any hints would be greaty appreciative before I head to the source code.
Thank you,
Jason
-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.
Octave's home on the web: http://www.octave.org
How to fund new projects: http://www.octave.org/funding.html
Subscription information: http://www.octave.org/archive.html
-------------------------------------------------------------
- distributed octave, Jason Tillett, 2003/11/06
- Re: distributed octave, JD Cole, 2003/11/07
- Message not available
- Message not available
- Message not available
- Message not available
- Re: distributed octave,
JD Cole <=
Message not available