Re: “Too many root sets” when calling compile frequently

[Jean Abou Samra]

Le 19/08/2022 à 00:33, Jean Abou Samra a écrit :
> Le 19/08/2022 à 00:18, Jean Abou Samra a écrit :
> > Calling the Guile compiler often causes this BDWGC error: “Too
> > many root sets”.
> >
> > scheme@(guile-user)> (define-syntax-rule (repeat n expr expr* ...) 
> > (do ((i 0 (1+ i))) ((eqv? i n)) expr expr* ...))
> > scheme@(guile-user)> (use-modules (system base compile))
> > scheme@(guile-user)> (repeat 10000 (compile 5))
> > Too many root sets
> > Abandon (core dumped)
> >
> > Any idea what is going on here? Should I report it as a bug?
> > Is there a workaround?
>
>
> Interestingly:
>
> scheme@(guile-user)> (define-syntax-rule (repeat n expr expr* ...) (do 
> ((i 0 (1+ i))) ((eqv? i n)) expr expr* ...))
> scheme@(guile-user)> (use-modules (system base compile))
> scheme@(guile-user)> (repeat 10000 (compile 5 #:to 'bytecode))
> scheme@(guile-user)> (use-modules (system vm loader))
> scheme@(guile-user)> (repeat 10000 (load-thunk-from-memory (compile 5 
> #:to 'bytecode)))
> Too many root sets
> Abandon (core dumped)
>
>
> So the problem lies in the VM loading infrastructure. (This is
> as far as I can investigate for now.)



I tried this code:

(use-modules (system vm loader))

(define-syntax-rule (repeat n-expr expr expr* ...)
  (let ((n n-expr))
    (do ((i 0 (1+ i)))
      ((eqv? i n))
      expr expr* ...)))

(let ((code (compile 5 #:to 'bytecode)))
  (repeat 10000
    (load-thunk-from-memory code)
    (display (length (all-mapped-elf-images))) (newline)))


For me, the output ends with

8174
8175
8176
8177
8178
8179
8180
8181
Too many root sets
Abandon (core dumped)


Now, BDWGC defines MAX_ROOT_SETS in include/private/gc_priv.h as

/* Root sets.  Logically private to mark_rts.c.  But we don't want the  */
/* tables scanned, so we put them here.                                 */
/* MAX_ROOT_SETS is the maximum number of ranges that can be    */
/* registered as static roots.                                  */
# ifdef LARGE_CONFIG
#   define MAX_ROOT_SETS 8192
# elif !defined(SMALL_CONFIG)
#   define MAX_ROOT_SETS 2048
# else
#   define MAX_ROOT_SETS 512
# endif


I am using Fedora, where BDWGC is compiled with --enable-large-config. When
the loader ingests a VM code chunk, it does (loader.c)


  if (gc_root)
    GC_add_roots (gc_root, gc_root + gc_root_size);


So each load is really adding a root until this threshold of 8192
is crossed.

I have no idea if it is possible to fix and/or work around this
other than by not calling compile often like this.

(Yes, I know, I should be using eval for one-off code evaluation,
but it discards source locations, which is what brought me here
in the first place.)

Jean