Re: GC

[Stefan Monnier]

> think, but then again it might not: IIRC, Emacs always does a GC
> before it asks the OS for more heap.  So you might see the message

I don't think this is true.  GC should only ever be called from eval
or funcall.  Several parts of the code assume that Fcons cannot call the GC,
for example.

In any case, here is my understanding of the situation:

The time taken by a single GC is roughly proportional to the total
(live+dead) heap size: the mark phase is proportional to the live-data size,
but the subsequent sweep phase is proportional to the total heap size.

Total heap size at the time of a GC is roughly equal to live-data +
gc-cons-threshold + unused-allocated.  The unused-allocated part of the
memory is basically due to fragmentation.

The frequency of GC is inversely proportional to gc-cons-threshold.
So the total portion of time used up by GC (the GC-overhead) is basically
proportional to:

   live-data + gc-cons-threshold + fragmentation
   ---------------------------------------------
               gc-cons-threshold

with a fixed gc-cons-threshold (as we have now), this boils down to

     live-data + fragmentation
     ------------------------- + 1
         gc-cons-threshold

So the GC-overhead currently grows with the live-data and with the
fragmentation.  This is one of the reasons why with a large heap, Emacs
tends to slow down.

Looking at the above equation one might think "let's bump gc-cons-threshold
way up" to make GC much cheaper.  The problem with it is that it tends to
increase fragmentation by delaying the reclaiming of memory (most serious
studies of memory fragmentation with non-moving memory allocator indicate
that an important factor to reduce fragmentation is prompt reclamation of
memory).

Making gc-cons-threshold proportional to the installed RAM sounds like a bad
idea to me: it's bound to be too small for some cases and much too large
for others.

The normal way to keep GC-overhead under control is to grow
gc-cons-threshold together with the heap size, such that the GC-overhead
stays constant (by making GCs less frequent when they get more
time-consuming).  Of course this may not always be best because by growing
gc-cons-threshold we may increase fragmentation, but "the best" is simply
not doable (not with a simple mark&sweep anyway).

I'd had already suggested a change to grow gc-cons-threshold as the heap
grows (a long time ago), and I see that XEmacs's gc-cons-percentage is
clean interface to such a feature.  I think we should introduce this
variable and give it a good non-zero default value.


        Stefan