Re: [igraph] releasing the GIL in the python extension

[Chris Wj]

So, would the following scenario result in memory corruption if the GIL was released?:

in python, create graph,
make a full copy of the graph (not a weakref),
in thread 1, with first graph, calculate something
in thread 2 with the copy of the graph, calculate something
thread 1 calculation fails,
error function/destructor is called, ...

thread 2's graph (the copy) gets freed?

-Chris

On Sun, Sep 20, 2009 at 10:18 AM, Tamas Nepusz <address@hidden> wrote:

Tamas, in the python extension, why do you not release the GIL when performing calculations on graphs? Was there a specific reason not to do so?

Yes, there was. The igraph C core is not thread-safe:

http://igraph.sourceforge.net/doc/html/igraph-errorhandlingthreads.html

In a nutshell: whenever igraph allocates memory for some purpose, it pushes the address of the allocated memory block to a global stack along with a destructor function that should be called should an error occur. If the calculation that required the memory block runs to completion without errors, it cleans up the allocated memory itself and then unrolls the stack. If an error happens, the calculation calls an error handler routine which unrolls the stack (calling the destructors associated to the memory addresses). Since there is only a single global stack, igraph functions that are run in parallel in different threads might end up freeing each other's memory should an error occur in one of them. Therefore there's no use in running multiple igraph functions in separate threads at the same time. That's why I keep the GIL locked -- even if you create multiple threads in Python and start calling igraph functions from them, the GIL will ensure that igraph's internal stack remains in a valid state.

--
Tamas



_______________________________________________
igraph-help mailing list
address@hidden
http://lists.nongnu.org/mailman/listinfo/igraph-help