BDW-GC-Guile incompatibilities

[Ludovic Courtès]

Hi,

"Neil Jerram" <address@hidden> writes:

> 2009/1/5 Ludovic Courtès <address@hidden>:

>>  1. The lack of `gc-live-object-stats'.
>
> I doubt it's a stopper, but current `gc-live-object-stats' is quite
> nice.  Doesn't libgc have a lightweight object-collected hook that
> would allow us to implement this?  Or is it the problem that would be
> too bad for performance?

We'd need to write something similar to `scm_i_card_statistics ()',
which basically traverses a card and accumulates live object info in a
hash table.  The difficulty is that, while Guile's GC is only able to
scan the cell heap (which is sufficient), BDW-GC doesn't know what a
cell is, so we'd have to find tricks to somehow not traverse the whole
heap but only cells.

I can look into it into more details to see whether/how that can be done
before we settle on a decision.

It'd be a potential source of incompatibility if `gc-live-object-stats'
behaved differently.  For instance, `gc.test' in `master' uses it to see
whether a given object was reclaimed, so the same test was rewritten in
the BDW-GC branch.

>>  3. Different behavior of weak hash tables, see
>>     http://lists.gnu.org/archive/html/guile-devel/2008-11/msg00015.html .
>>     This can be fixed, but I'm unclear whether it's worth it (comments
>>     welcome!).
>
> IIUC, your example to the GC list can be fixed (in the application) by
> using a (BGC) doubly weak hash table instead.

Yes, but that's a visible change for the application.

> I wonder what is the simplest example of a current Guile weak-key hash
> table that can't be mapped to a BGC doubly weak one?

The test that led me to find it is "Unused modules are removed" in
`gc.test'.  The culprit is `set-module-eval-closure!', which creates a
circular reference between CLOSURE and MODULE via the procedure property
weak-key hash table.

Here are the details (from an old message I never posted since I
understood what happens, in the meantime...):

Basically, `make-module' invokes a record constructor, after which it
does:

  (set-module-eval-closure! module (standard-eval-closure module))

Note that `standard-eval-closure' returns a fresh SMOB that points to
MODULE.

In turn, `set-module-eval-closure!' invokes the record setter and then
does this:

  (set-procedure-property! closure 'module module)

Here, CLOSURE is actually the SMOB returned by `standard-eval-closure'.
Procedure properties of "real" closures are stored in the closure itself
(see `SCM_SETPROCPROPS ()').  But for non-closure objects such as subrs
or SMOBs, an indirection is used: a phony closure is created and
associated with the subr/SMOB via a weakly-keyed hash table (see
`scm_stand_in_scm_proc ()').  Still here?

So, in the case of `make-module', the object graph is as follows:


    .---------------------------------.
    |                                 |
    V                                 |
.--------.       .--------------.  .----------------------------.
| module |<----->| eval-closure |  | closure                    |
`--------'       `--------------'  | (from                      |
                   ^               |  `scm_stand_in_scm_proc ())|
                   :               `----------------------------'
                   :                  ^                          
                   :                  :                          
                   :..................'                         
                   :                    
                 .---------------------.
                 | scm_stand_in_procs  |
                 | (weak-key hash tab.)|
                 `---------------------'

(Dotted arrows denote weak references, while solid arrows denote
"regular" references.)

In the BDW-GC-based Guile, the reference from the phony closure to the
module prevents them from being collected.

Cheers,
Ludo'.