Re: [igraph] igraph (R) get.all.shortest.paths memory issue

[Pascal Jürgens]

Hello Dazhi,

not long ago, I asked a similar question; here are some part of Tamas' Answer:

...

Are there any practical limits to using shortest_paths() with much larger networks?

*Theoretically*, the only practical limit should be the amount of memory you have available :) (Of course, your OS has to support allocating huge chunks of memory to the same process; for instance, on 32-bit machines, there is an address space limit which cannot be circumvented without switching to a 64-bit platform). If you found some limit other than the memory requirements or the time complexity, then that's a bug in igraph and we'll try to fix it.

...

What I can tell you is that I successfully managed to calculate the *diameter* of an undirected network with 2.1 million vertices and 7.5 million edges -- this took about 11 days on a single core of an Intel Xeon X3360 running at 2.83 GHz. So, time complexity should not be a problem for your network. I'm still worried about the storage requirements, because when using the Python interface, the result matrix is temporarily stored *twice*, once as a regular igraph_matrix_t and once as a Python list of lists which is returned to Python. Of course the igraph_matrix_t is destroyed as soon as the Python representation is ready, but still. I would probably try to be on the safe side and build up the shortest path matrix line by line and store the calculated lines on disk, so even if Python runs out of memory for some reason, nothing is lost because all the earlier calculations are still saved.

...

Tamas

This should give you some background. So as long as you are using a 64bit system with more than twice the memory that the matrix itself needs, you should be fine.

Pascal

On Aug 4, 2010, at 10:12 PM, Dazhi Jiao wrote:

Hi,

I am a new user of igraph. I am currently use it for a relative large directed network (~7.5K vertices, ~20K edges).

I need to calculate the shortest paths bewteen the vertices, and I found that it throws an error:

Error in get.all.shortest.paths(g, i, to = j, mode = "out") :
  At vector.pmt:972 : canot copy vector, Out of memory

Is there any way to work around this? The computer I am running this on has 8G memory.

Thanks!

Dazhi
_______________________________________________
igraph-help mailing list
address@hidden
http://lists.nongnu.org/mailman/listinfo/igraph-help