Re: data structure for simulation

[Michael Creel]

Francesco Potorti`-2 wrote:
> 
>>> The general problem is that I am trying to find a good and simple way to
>>> use a cluster for Montecarlo simulation.  Ideally, I'd like to do that
>>> from inside Octave, without resorting to an external program like a bash
>>> script.
>>
>>Maybe I can help with that. I have a function of the form
>># montecarlo.m: generates a specified number of replications of a
function's
>>output and
>># writes them to a user-specified output file.
>>#
>># USAGE: montecarlo(f, f_args, reps, outfile, nslaves, n_pooled, 
n_returns,
>>debug)
>>#
>># IMPORTANT: f should return a row vector of output from feval(f,f_args)
>>#
>># For normal evaluation on one machine, only the first 4 arguments are
>>required.
>># * Arg 1: (required) the function that generates a row vector of output
>># * Arg 2: (required) the arguments of the function, in a cell
>># * Arg 3: (required) the number of replications to generate
>># * Arg 4: (required) the output file name
>># * Arg 5: (optional) number of compute nodes for parallel runs
>># * Arg 6 (optional) number of replications to be pooled together before
>>they are sent
>># back to the master node.
>># * Arg 7 (optional) the number of columns of the output returned by f.
When
>>f is very costly
>># to evaluate, use this to get a parallel run started immediately.
Otherwise
>>single run must
>># be finished before the size is determined. This is bad, since many nodes
>>will be idle.
>># * Arg 8 (optional) verbose: 1 for on, 0 for off
>># * Arg 9 (optional) 0/1 switch controlling debug mode, off by default.
>>
>>
>>This will Monte Carlo any function that takes arguments as a cell, and
>>writes output as a row vector. I think that in most cases a wrapper can be
>>written to satisfy this requirement. To do this in parallel, you must have
>>MPITB installed. I can provide more details if you're interested.
> 
> That would be very good, more than I expected :)
> 
> To be true, when I had written "simple" I meant that no recompilation of
> Octave was needed, and no fiddling with external libraries :(
> 
> The fact is, Montecarlo is a very particular and simple case of parallel
> execution, which could be done by using Octave's popen command, fread,
> fwrite and little else, all in a .m file without the help of external
> libraries, taking Juroslav's parcellfun as a starting point.  If I am
> not oversimplifying things...
> 
> -- 
> Francesco Potortì (ricercatore)        Voice: +39 050 315 3058 (op.2111)
> ISTI - Area della ricerca CNR          Fax:   +39 050 315 2040
> via G. Moruzzi 1, I-56124 Pisa         Email: address@hidden
> (entrance 20, 1st floor, room C71)     Web:   http://fly.isti.cnr.it/
> _______________________________________________
> Help-octave mailing list
> address@hidden
> https://www-old.cae.wisc.edu/mailman/listinfo/help-octave
> 
> 


monte_carlo.m and monte_carlo_nodes.m are available on the PelicanHPC Linux
distro, along with some examples, ready to run. "Ready to run" is an
important qualifier here, because to use these functions on more than one
core, you need to have LAM/MPI and MPITB installed. For clusters, you need
to have password-less ssh set up, etc. Due to all of that, I recommend
getting PelicanHPC, and running it virtualized. That will let you check out
the examples. If you like what you see, PelicanHPC will allow you to set up
a cluster very quickly. It is possible to set up a real cluster while
running PelicanHPC virtualized on the platform of your choice, so this works
for Windows and MacOS users, too.

Doing Monte Carlo work in this way scales linearly, as you would expect. I
have had a 16 core cluster doing this almost nonstop for the last year and a
half :-)

PelicanHPC is available at http://pelicanhpc.org
You can get a journal article that uses montecarlo.m as an example at
http://econpapers.repec.org/article/jaejapmet/v_3a22_3ay_3a2007_3ai_3a1_3ap_3a215-223.htm

The files are directly available at
http://pareto.uab.es/mcreel/Econometrics/MyOctaveFiles/Econometrics/MonteCarlo/

I'd be happy to help you get started if you're interested in using this
scheme.

Michael
-- 
View this message in context: 
http://www.nabble.com/data-structure-for-simulation-tp22560278p22584071.html
Sent from the Octave - General mailing list archive at Nabble.com.