Re: cell heap usage in 1.8 vs 1.6

[Ludovic Courtès]

Hi,

I hopefully managed to characterize the problem.  I instrumented HEAD
with the attached patch that exposes the current "min yield" of both
freelists.  Then, I ran the attached script that does the following:

  * at regular intervals, plot the total cell heap, alive cell heap
    (using your computation method), yield of the single-cell freelist,
    yield of the double-cell freelist;

  * every 1000 iteration, perform a large memory allocation (single
    cells), and then discard the allocated cells.

The resulting file for around 5000 iterations is attached.  It can be
plotted with GNUplot using:

  plot ',,head.sporadic.gp' \
       using 1:($2/$3) with lines title "total/alive", \
    '' using 1:4 with lines axes x1y2 title "yield1",  \
    '' using 1:5 with lines axes x1y2 title "yield2"

This shows the oscillation of the total to alive cell heap ratio, as
well as the variations of the "min yield" of the freelists (x axis
represents the iteration number):

The following observations can be made:

  * the heap gets more and more underutilized, although it seems to
    eventually stabilize somewhere around 5% (!) utilization (i.e.,
    total/alive = 20);

  * the "min yield" of the single-cell freelist grows way too fast as a
    reaction to the first memory allocation (at iteration 1000), which
    consequently leads to the increase of the total heap size.

Keep in mind that currently heap is never reclaimed, so its total size
can only increase.  Likewise, "min yield" can only increase, as its
typically a fraction (40% by default) of the total heap size.

Strangely enough, with 1.8, `alive-cell-heap' remains exactly constant
until iteration 1000.  I suspect something wrong with
`gc-live-object-stats' there...

Thanks,
Ludovic.

--- orig/libguile/gc.c
+++ mod/libguile/gc.c
@@ -408,6 +408,28 @@
 }
 #undef FUNC_NAME
 
+SCM scm_freelist1_yield (void);
+SCM scm_freelist2_yield (void);
+
+SCM_DEFINE (scm_freelist1_yield, "freelist1-yield", 0, 0, 0,
+           (void),
+           "")
+#define FUNC_NAME s_scm_freelist1_yield
+{
+  return (scm_from_long (scm_i_master_freelist.min_yield));
+}
+#undef FUNC_NAME
+
+SCM_DEFINE (scm_freelist2_yield, "freelist2-yield", 0, 0, 0,
+           (void),
+           "")
+#define FUNC_NAME s_scm_freelist2_yield
+{
+  return (scm_from_long (scm_i_master_freelist2.min_yield));
+}
+#undef FUNC_NAME
+
+
 /* Update the global sweeping/collection statistics by adding SWEEP_STATS to
    SCM_I_GC_SWEEP_STATS and updating related variables.  */
 static inline void

gc-stress.scm
Description: The script

binGSR6x5Ya7A.bin
Description: The GNUplot file