Re: Parallelization with live stream of data

[Jose]

On 16/09/12 20:17, Olaf Till wrote:
> Jose, you should always CC the list so that others can comment ...

I ment to, but pressed reply blindly and did not check the recipients, 
sorry (and thanks for forwarding the message to the list).

> For a realtime task, it might be better to do the scheduling manually
> (but thats not quite easy).
>
> There is an openmpi_ext package, but I'm not the right person to
> comment on this.
>
> Alternatively, one can use Octaves fork(), waitpid(), pipe(), and
> close() functions. The packages 'general' and 'parallel' both provide
> functions for sending and receiving Octave variables over pipes
> (fsave()/fload() and __bw_psend__()/__bw_prcv__(), respectively). They
> require the additional use of Octaves fflush(). Both packages provide
> a wrapper around Unix' _exit for child termination (__exit__() and
> __internal_exit__(), respectively). The package 'parallel' also
> provides a function select(), a wrapper around Unix' select. All these
> together allow parallelization with Octave without writing C or C++
> code.
>
> The latter way, and possibly also the former (if it should be feasible
> for multicore/single-machine), require you to write a scheduler.

Probably I will try going with the parallel package, as it seems to 
provide a quite comfortable API for high level remote evaluations. I 
will have to, as you suggest, create my own scheduler.

Thanks again for the comments.
Jose