module GC bug

[Han-Wen Nienhuys]

Hi,

I've found a memory leak in GUILE.

The contents of modules are not garbage collected.

This seems to be related with two errors:

 - scm_stand_in_procs is a hashtable. I believe it should be weak_key
hashtable, just like the scm_object_whash table. For, if a closure is
GC'd, so should it properties.

 - in boot-9.scm, set-module-eval-closure! does

     (set-procedure-property! closure 'module module))

So the closure is a key in a weak hash-table, pointing to the module
as a value (using scm_stand_in_procs), the module is always marked
during GC. However, since the module points back to the closure via
the 'eval-closure slot, the key is always marked. Consequently,
neither closure nor module are ever GC'd.

I've fixed this by introducing a new function (eval-closure-module)
which returns the module of a closure via the eval-closure smob.  Find 
the patch (including test) attached.

I don't really know what I am doing, so please comment. FWIW, the test 
suite compiled without complaints. If noone objects, I will merge this 
patch tomorrow night.  Furthermore, I believe the same bug is in GUILE 
1.6, so a 1.6.8 release is warranted IMO.

Greetings,

Han-Wen


PS. I feel like a broken record, but when will GUILE 1.8 be out?


--
 Han-Wen Nienhuys - address@hidden - http://www.xs4all.nl/~hanwen

? b
? core.11847
? core.8858
? evalclos
? gettext.m4
? libltdl
? t.scm
Index: INSTALL
===================================================================
RCS file: /cvsroot/guile/guile/guile-core/INSTALL,v
retrieving revision 1.40
diff -u -r1.40 INSTALL
--- INSTALL     28 Sep 2004 19:33:40 -0000      1.40
+++ INSTALL     9 Jun 2005 23:27:25 -0000
@@ -1,7 +1,7 @@
 Installation Instructions
 *************************
 
-Copyright (C) 1994, 1995, 1996, 1999, 2000, 2001, 2002, 2004 Free
+Copyright (C) 1994, 1995, 1996, 1999, 2000, 2001, 2002, 2004, 2005 Free
 Software Foundation, Inc.
 
 This file is free documentation; the Free Software Foundation gives
@@ -189,8 +189,13 @@
 
      ./configure CC=/usr/local2/bin/gcc
 
-will cause the specified gcc to be used as the C compiler (unless it is
-overridden in the site shell script).
+causes the specified `gcc' to be used as the C compiler (unless it is
+overridden in the site shell script).  Here is a another example:
+
+     /bin/bash ./configure CONFIG_SHELL=/bin/bash
+
+Here the `CONFIG_SHELL=/bin/bash' operand causes subsequent
+configuration-related scripts to be executed by `/bin/bash'.
 
 `configure' Invocation
 ======================
Index: ice-9/ChangeLog
===================================================================
RCS file: /cvsroot/guile/guile/guile-core/ice-9/ChangeLog,v
retrieving revision 1.650
diff -u -r1.650 ChangeLog
--- ice-9/ChangeLog     5 Jun 2005 17:26:07 -0000       1.650
+++ ice-9/ChangeLog     9 Jun 2005 23:27:27 -0000
@@ -1,3 +1,9 @@
+2005-06-10  Han-Wen Nienhuys  <address@hidden>
+
+       * boot-9.scm (set-module-eval-closure!): remove
+       set-procedure-property! closure 'module. Setting this property
+       causes un-gc-able modules.
+
 2005-06-05  Marius Vollmer  <address@hidden>
 
        * boot-9.scm (substring-fill!): New, for compatability.
Index: ice-9/boot-9.scm
===================================================================
RCS file: /cvsroot/guile/guile/guile-core/ice-9/boot-9.scm,v
retrieving revision 1.349
diff -u -r1.349 boot-9.scm
--- ice-9/boot-9.scm    5 Jun 2005 17:24:52 -0000       1.349
+++ ice-9/boot-9.scm    9 Jun 2005 23:27:28 -0000
@@ -339,7 +339,7 @@
 
 (define (environment-module env)
   (let ((closure (and (pair? env) (car (last-pair env)))))
-    (and closure (procedure-property closure 'module))))
+    (and closure (eval-closure-module closure))))
 
 
 
@@ -1266,10 +1266,12 @@
   (let ((setter (record-modifier module-type 'eval-closure)))
     (lambda (module closure)
       (setter module closure)
-      ;; Make it possible to lookup the module from the environment.
-      ;; This implementation is correct since an eval closure can belong
-      ;; to maximally one module.
-      (set-procedure-property! closure 'module module))))
+      
+
+      ;; do not set procedure properties on closures.
+      ;; since procedure properties are weak-hashes, they cannot
+      ;; have cyclical data, otherwise the data cannot be GC-ed.
+      )))
 
 
 
Index: libguile/ChangeLog
===================================================================
RCS file: /cvsroot/guile/guile/guile-core/libguile/ChangeLog,v
retrieving revision 1.2281
diff -u -r1.2281 ChangeLog
--- libguile/ChangeLog  9 Jun 2005 19:33:38 -0000       1.2281
+++ libguile/ChangeLog  9 Jun 2005 23:27:35 -0000
@@ -1,3 +1,12 @@
+2005-06-10  Han-Wen Nienhuys  <address@hidden>
+
+       * modules.c (s_scm_eval_closure_module): new function. Return the
+       module inside an eval-closure.
+
+       * gc.c (scm_init_storage): make scm_stand_in_procs a weak_key hash
+       table. This means that procedure properties are GC'd if the
+       procedure dies.
+
 2005-06-09  Han-Wen Nienhuys  <address@hidden>
 
        * gc.c (tag_table_to_type_alist): convert tag number to "tag %d"
Index: libguile/gc.c
===================================================================
RCS file: /cvsroot/guile/guile/guile-core/libguile/gc.c,v
retrieving revision 1.268
diff -u -r1.268 gc.c
--- libguile/gc.c       9 Jun 2005 19:33:38 -0000       1.268
+++ libguile/gc.c       9 Jun 2005 23:27:35 -0000
@@ -935,7 +935,7 @@
 
 #endif
 
-  scm_stand_in_procs = scm_c_make_hash_table (257);
+  scm_stand_in_procs = scm_make_weak_key_hash_table (scm_from_int (257));
   scm_permobjs = SCM_EOL;
   scm_protects = scm_c_make_hash_table (31);
   scm_gc_registered_roots = scm_c_make_hash_table (31);
Index: libguile/modules.c
===================================================================
RCS file: /cvsroot/guile/guile/guile-core/libguile/modules.c,v
retrieving revision 1.59
diff -u -r1.59 modules.c
--- libguile/modules.c  23 May 2005 19:57:20 -0000      1.59
+++ libguile/modules.c  9 Jun 2005 23:27:36 -0000
@@ -346,6 +346,19 @@
 }
 #undef FUNC_NAME
 
+
+SCM_DEFINE (scm_eval_closure_module, "eval-closure-module", 1, 0, 0,
+           (SCM closure),
+           "Return the module for @var{closure}.")
+#define FUNC_NAME s_scm_eval_closure_module
+{
+  SCM_ASSERT_TYPE(SCM_EVAL_CLOSURE_P (closure), closure, SCM_ARG1, FUNC_NAME, 
"eval-closure");
+  return SCM_PACK (SCM_CELL_WORD_1(closure));
+}
+#undef FUNC_NAME
+ 
+
+
 SCM_DEFINE (scm_standard_interface_eval_closure,
            "standard-interface-eval-closure", 1, 0, 0,
            (SCM module),
Index: libguile/modules.h
===================================================================
RCS file: /cvsroot/guile/guile/guile-core/libguile/modules.h,v
retrieving revision 1.26
diff -u -r1.26 modules.h
--- libguile/modules.h  23 May 2005 19:57:20 -0000      1.26
+++ libguile/modules.h  9 Jun 2005 23:27:36 -0000
@@ -98,6 +98,7 @@
 SCM_API SCM scm_current_module_transformer (void);
 SCM_API SCM scm_eval_closure_lookup (SCM eclo, SCM sym, SCM definep);
 SCM_API SCM scm_standard_eval_closure (SCM module);
+SCM_API SCM scm_eval_closure_module (SCM closure);
 SCM_API SCM scm_standard_interface_eval_closure (SCM module);
 SCM_API SCM scm_get_pre_modules_obarray (void);
 SCM_API SCM scm_lookup_closure_module (SCM proc);
Index: test-suite/ChangeLog
===================================================================
RCS file: /cvsroot/guile/guile/guile-core/test-suite/ChangeLog,v
retrieving revision 1.350
diff -u -r1.350 ChangeLog
--- test-suite/ChangeLog        5 Jun 2005 21:38:22 -0000       1.350
+++ test-suite/ChangeLog        9 Jun 2005 23:27:37 -0000
@@ -1,3 +1,8 @@
+2005-06-10  Han-Wen Nienhuys  <address@hidden>
+
+       * tests/gc.test ("gc"): add a test to verify that modules are
+       garbage collected.
+
 2005-06-06  Kevin Ryde  <address@hidden>
 
        * tests/strings.test (string-split): Try splitting on an 8-bit char.
Index: test-suite/tests/gc.test
===================================================================
RCS file: /cvsroot/guile/guile/guile-core/test-suite/tests/gc.test,v
retrieving revision 1.6
diff -u -r1.6 gc.test
--- test-suite/tests/gc.test    23 May 2005 19:57:22 -0000      1.6
+++ test-suite/tests/gc.test    9 Jun 2005 23:27:37 -0000
@@ -55,3 +55,17 @@
       (gc)
       (remove-hook! after-gc-hook thunk)
       foo)))
+
+
+(with-test-prefix "gc"
+  (pass-if "Unused modules are removed"
+          (let*
+              ((dummy (gc))
+               (last-count (cdr (assoc
+                                 "eval-closure" (gc-live-object-stats)))))
+              
+            (for-each (lambda (x) (make-module)) (iota 1000))
+            (gc)
+            (gc) ;; twice: have to kill the weak vectors.
+            (= last-count (cdr (assoc "eval-closure" (gc-live-object-stats)))))
+          ))
Index: test-suite/tests/hash.test
===================================================================
RCS file: /cvsroot/guile/guile/guile-core/test-suite/tests/hash.test,v
retrieving revision 1.3
diff -u -r1.3 hash.test
--- test-suite/tests/hash.test  23 May 2005 19:57:22 -0000      1.3
+++ test-suite/tests/hash.test  9 Jun 2005 23:27:37 -0000
@@ -65,7 +65,6 @@
 ;;;
 ;;; hashx-remove!
 ;;;
-
 (with-test-prefix "hashx-remove!"
   (pass-if (->bool (object-documentation hashx-remove!)))