Re: wip-threads-and-fork

[Andy Wingo]

On Sat 03 Mar 2012 22:20, address@hidden (Ludovic Courtès) writes:

> I’d prefer a solution where libguile-internal threads and locks are
> suitably handled upon fork (basically what wip-threads-and-fork does),
> and where users are provided with mechanisms to do the same at their
> level–which boils down to exposing pthread_atfork.
>
> WDYT?

I would have preferred this, but I came to the conclusion that this
approach is not sound.

Did you see that I merged the atfork bits into master?
(wip-threads-and-fork also had some CLOEXEC bits that needed more
baking).  They worked... sorta.  They had a few problems:

  1) It's impossible to work around the lack of atfork() in libraries
     that you depend on.

     For example, wip-threads-and-fork called fork() within the GC alloc
     lock, to get around the lack of a pthread_atfork() in libgc.  But
     then I submitted a patch to make libgc do this itself:

       
http://thread.gmane.org/gmane.comp.programming.garbage-collection.boehmgc/4940

     It's pretty difficult to tell which version of libgc you would
     have.  There is no workaround that is sufficient.

  2) POSIX explicitly disclaims the result of calling non-signal-safe
     primitives after a fork() of a multithreaded program.

  3) Nobody cares about these bugs.  See e.g. the lack of response at
     http://sourceware.org/bugzilla/show_bug.cgi?id=13725.  Even Bruno
     didn't reply to the Cc.  See point (2).

  4) The atfork mechanism imposes a total ordering on locks.  This is
     possible for static locks, but difficult for locks on collectable
     Scheme objects.

  5) Relatedly, just to be able to lock all weak tables at a fork, we
     had to create a new weak table-of-tables and add the tables to it.
     This is needless complication and overhead.

  6) scm_c_atfork() is a broken interface.  Because it hangs its hooks
     off of one pthread_atfork() invocation, it can cause newer locks to
     insert themselves in the wrong position relative to
     pthread_atfork() calls made between when Guile installed the
     scm_c_atfork handler, and the call to scm_c_atfork.

     There can be only one pthread_atfork() list, in a correct program.

In the end I reverted those patches because they were just complication
that didn't solve any fundamental problems.

I came to the opinion, having run a threaded, forking program, that we
would be much better off if we provided good abstractions to spawn
processes, but that expecting fork() to work in a multithreaded program
is not realistic.

Still, there is one other thing that perhaps we could do to shut down
the signal handling thread around a fork().  Dunno, perhaps it is worth
looking into.

Andy
-- 
http://wingolog.org/