Re: [igraph] iGraph functions in parallel?

[Gábor Csárdi]

I don't know much about snowfall, but if that does not work for you,
then the 'snow' package should, it works above MPI or even without it.
It work on multicore or SMP machines or even on clusters. It is very
high level, parallel diameter calculation can be programmed in 10
minutes and 10 lines of code, I would guess.

Best,
Gabor

On Tue, Sep 15, 2009 at 12:56 PM, Tamas Nepusz <address@hidden> wrote:
> Hi Juan,
>
> igraph does not contain native support for openMP or any other
> parallelization solution yet. However, an obvious way to distribute the
> computational load of diameter calculations (if your computer has
> multiple CPU cores) is to start independent shortest path calculations
> from each of the nodes and aggregate the results later. For instance, if
> you have 4 CPU cores, start four R instances, partition the set of
> vertices into four roughly equal sets and calculate shortest paths
> individually from each of the vertices s.t. the first R instance
> processes paths starting from the first quarter of the vertices, the
> second R instance processes paths starting from the second quarter and
> so on. Unfortunately this means that you have to load your graph into
> the memory four times and if it doesn't fit, you will have problems as
> the OS will keep on swapping all the time.
>
> As for snowfall, I haven't tried it and I'm not an R expert, so I won't
> try to comment on that part. Since most of igraph's code is in C, igraph
> calls are sort of a black box for R: some R objects go in there and some
> other R objects come out, but R knows nothing about what happens inside.
> Therefore, I don't think that snowfall could magically parallelize
> igraph's diameter() call, but it might be possible that the solution I
> mentioned above is feasible using snowfall and a _single_ R instance
> where snowfall will take care of distributing the individual shortest
> path calls.
>
> --
> Tamas
>
> On Tue, Sep 15, 2009 at 12:41:54PM +0200, Juan Manuel Tirado wrote:
>> Hi everybody,
>>
>>    I'm currently working with large graphs (about 250,000 nodes and
>> 1,500,000 edges). This size seems to be too large for igraph, and some
>> operations such as diameter() take more than one day (I stopped the
>> operation before it finished), the same thing with the average path length.
>>
>>    My question is does exist any way to execute igraph in parallel (using
>> openMP for example)? I have found an R package called snowfall (
>> http://www.imbi.uni-freiburg.de/parallel/) but I'm not sure if it is
>> possible to run igraph with this package.
>>
>>    Regards,
>>
>>    Juan
>
>> _______________________________________________
>> igraph-help mailing list
>> address@hidden
>> http://lists.nongnu.org/mailman/listinfo/igraph-help
>
>
>
> _______________________________________________
> igraph-help mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/igraph-help
>



-- 
Gabor Csardi <address@hidden>     UNIL DGM