Re: [PATCH] Configure GMP to use GC allocation functions, remove bignum finalizers

[Ludovic Courtès]

Hi Andy!

Andy Wingo <address@hidden> skribis:

> On Tue 29 Nov 2011 00:10, address@hidden (Ludovic Courtès) writes:
>
>> Andy Wingo <address@hidden> skribis:
>>
>>> On Sun 27 Nov 2011 22:25, address@hidden (Ludovic Courtès) writes:
>>>
>>>> The problem is that this measurement doesn’t allow us to differentiate
>>>> between a growing heap with objects that may be freed as a result of
>>>> running the GC, and a growing heap just because the application needs
>>>> more malloc’d objects.
>>>
>>> This is true, but typically the heap stabilizes at some point.
>>
>> Ooh, re-reading your previous message, I now see what you mean, and it
>> makes sense to me.
>
> WDYT?

Nice!

> +static size_t
> +get_image_size (void)
> +{
> +  unsigned long size, resident, share;
> +  size_t ret;
> +
> +  FILE *fp = fopen ("/proc/self/statm", "r");
> +
> +  if (fp && fscanf (fp, "%lu %lu %lu", &size, &resident, &share) == 3)
> +    ret = resident * 4096;
> +
> +  if (fp)
> +    fclose (fp);
> +
> +  return ret;
> +}

RET should never be left uninitialized, and the caller should be
prepared to deal with RET == 0.

(The code itself doesn’t have to #ifdef __linux__ because other OSes
provides an implementation that mimics Linux’s /proc.)

> +static void *
> +adjust_gc_frequency (void * hook_data SCM_UNUSED,
> +                     void *fn_data SCM_UNUSED,
> +                     void *data SCM_UNUSED)
> +{

[...]

> +      if (free_space_divisor + 0.5 + hysteresis < target_free_space_divisor
> +          || free_space_divisor - 0.5 - hysteresis > 
> target_free_space_divisor)
> +        {
> +          free_space_divisor = lround (target_free_space_divisor);
> +#if HEURISTICS_DEBUG
> +          fprintf (stderr, "new divisor %lu\n", free_space_divisor);
> +#endif
> +          GC_set_free_space_divisor (free_space_divisor);
> +        }
> +    }
> +

My impression was that FREE_SPACE_DIVISOR was quite coarse-grain.
What’s your experience with that?

Could you run the stuff under gc-benchmarks/ with and without the
heuristic, as in [0]?

Also, could you check with a program that actually mixes malloc +
GC-alloc how it behaves?

I tried to reproduce the infamous iconv issue, which would have been a
simple way to check, but failed:

  (with-fluids ((%default-port-encoding "UTF-16BE"))
    (let loop () (open-output-string ) (loop)))

Memory growth appears to be bounded here.

Thanks,
Ludo’.

[0] http://thread.gmane.org/gmane.lisp.guile.devel/7803