[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Do we trust the man on the GC trigger?
From: |
Marius Vollmer |
Subject: |
Do we trust the man on the GC trigger? |
Date: |
28 Aug 2001 01:01:56 +0200 |
User-agent: |
Gnus/5.09 (Gnus v5.9.0) Emacs/21.0.102 |
Hi,
I think we have a problem with the `mallocated' GC trigger. It is not
maintained reliably and I'm afraid we need to have everybody review
their code to get it right.
I think the current interface with scm_must_malloc, scm_must_free,
scm_done_malloc, scm_done_free is too difficult to use right and too
hard to debug.
Guile itself is full of mtrigger related bugs, I'm afraid. A typical
one is in fports.c: the buffers for a fport are allocated with
scm_must_malloc and freed with scm_must_free. The allocation is
reported to the GC, but the freeing never is. The result is that the
GC thinks that more and more memory is being allocated that it should
be able to free, but that never actually gets freed (although in
reality the program is very well behaved). As a counter measure to
constant GC, the GC raises its mtrigger setting in a frenzy until it
wraps around, causing a `hallucinating GC' syndrome, effectively
stopping the program dead.
(Watch scm_mtrigger while your favorite long-running Guile program
executes, it will continuously rise.)
The problem is that scm_must_malloc registers the allocated amount
with the GC, but scm_must_free does not de-register it. For that, one
would currently needs to use scm_done_free, or return an appropriate
number from a smob free routine.
Another problem is that scm_must_malloc is used in places where it is
probably not appropriate since the caller does not know whether that
block memory is really ending up under the control of the GC, or not.
For example scm_do_read_line in rdelim.c uses scm_must_malloc to
allocate a buffer that it returns, and scm_read_line passes this to
scm_take_string. scm_take_string assumes that the memory has not been
under GC control previously and calls scm_done_malloc to account for
the fact that it now is. But scm_must_malloc has _already_ increased
scm_mallocated by the proper amount. Thus, it is now doubly
reflected.
Since the current interface is unsymmetrical (scm_must_malloc
registers, but scm_must_free doesn't de-register), I propose to change
it as follows. Switching to this new interface will force us and
everybody else to systematically review their code.
- the smob free routine does no longer return the number of bytes
that have been freed. For the transition period, free routines
are first encourged to return 0, then required, and then their
return type changes to void.
- scm_must_malloc, scm_must_free are deprecated.
- in their place, we have
Function: void *scm_malloc (size_t size, const char *what);
Allocate SIZE bytes of memory. When not enough memory is
available, signal an error. This function runs the GC to free
up some memory when it deems it appropriate.
The WHAT argument is used for statistical purposes and for
error reporting. It should describe the type of object that
the memory will be used for so that users can identify just
what strange objects are eating up their memory.
[ Note: this function will not consider the memory block to be
under GC control. ]
Function: void scm_free (void *mem, size_t size, const char *what);
Free the memory at MEM. MEM must have been returned by
scm_malloc and SIZE and WHAT should match what has been passed
to that scm_malloc call. MEM might be NULL in which case
nothing happens.
When you don't have an accurate value for SIZE, pass 0. This
is not encouraged however, as it will distort the statistics.
Function: void scm_gc_show (void *mem, size_t size, const char *what);
[ Need a better name. ]
Informs the GC that the memory at MEM of size SIZE can
potentially be freed during a GC. That is, MEM is part of a
GC controlled object and when the GC happens to free that
object, SIZE bytes will be freed along with it. The GC will
_not_ free the memory itself, it will just know that so-and-so
much bytes of memory are associated with GC controlled objects
and the memory system figures this into its decisions when to
run a GC.
MEM does not need to come from scm_malloc.
Function: void scm_gc_hide (void *mem, size_t size, const char *what);
[ Need a better name. ]
Inform the GC that the memory at MEM of size SIZE has been
freed is no longer associated with a GC controlled object.
Function: void *scm_gc_malloc (size_t size, const char *what);
Like scm_malloc, but also call scm_gc_show on the result.
Function: void scm_gc_free (void *mem, size_t size,
const char *what);
Like scm_free, but also call scm_gc_hide.
The normal thing to use is scm_gc_malloc / scm_gc_free.
Comments?
Re: Do we trust the man on the GC trigger?, Dirk Herrmann, 2001/08/28