Re: Weak tables harmful to GC?

[Ricardo Wurmus]

Hi Ludo,

I tried building Guile with the following Guix package definition:

--8<---------------cut here---------------start------------->8---
(define-public guile-2.2-awesome
  (package (inherit guile-2.2)
    (name "guile-awesome")
    (source (origin (inherit (package-source guile-2.2))
       (patches (list 
"/home/rwurmus/0001-Remove-weak-tables-and-revert-to-weak-hash-tables.patch"
                      
"/home/rwurmus/0002-Keep-weak-hash-table-item-count-consistent.patch"))))
    (arguments
      (substitute-keyword-arguments (package-arguments guile-2.2)
        ((#:phases phases)
         `(modify-phases ,phases
            (add-before 'pre-configure 'bootstrap
              (lambda _
                (zero? (system* "autoreconf" "-vif"))))))))
    (native-inputs
     `(("autoconf" ,autoconf)
       ("automake" ,automake)
       ("libtool" ,libtool)
       ("flex" ,flex)
       ("texinfo" ,texinfo)
       ("gettext" ,gettext-minimal)
       ,@(package-native-inputs guile-2.2)))))
--8<---------------cut here---------------end--------------->8---

Unfortunately, I cannot bootstrap Guile on this 1.5 TB RAM server:

--8<---------------cut here---------------start------------->8---
…
  BOOTSTRAP GUILEC system/vm/program.go
  BOOTSTRAP GUILEC system/vm/vm.go
  BOOTSTRAP GUILEC system/foreign.go
GC Warning: Repeated allocation of very large block (appr. size 230096896):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 230096896):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 230096896):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 230096896):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 230096896):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 230096896):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 230096896):
        May lead to memory leak and poor performance
Too many heap sections: Increase MAXHINCR or MAX_HEAP_SECTS
/gnu/store/kpxi8h3669afr9r1bgvaf9ij3y4wdyyn-bash-minimal-4.4.12/bin/bash: line 
6: 30796 Aborted                 GUILE_AUTO_COMPILE=0 ../meta/build-env guild 
compile --target="x86_64-unknown-linux-gnu" -O1 -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/module" -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/guile-readline" -o 
"language/scheme/compile-tree-il.go" 
"../module/language/scheme/compile-tree-il.scm"
make[2]: *** [Makefile:1928: language/scheme/compile-tree-il.go] Error 134
make[2]: *** Waiting for unfinished jobs....
^GGC Warning: Repeated allocation of very large block (appr. size 230096896):
        May lead to memory leak and poor performance
Too many heap sections: Increase MAXHINCR or MAX_HEAP_SECTS
/gnu/store/kpxi8h3669afr9r1bgvaf9ij3y4wdyyn-bash-minimal-4.4.12/bin/bash: line 
6: 30386 Aborted                 GUILE_AUTO_COMPILE=0 ../meta/build-env guild 
compile --target="x86_64-unknown-linux-gnu" -O1 -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/module" -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/guile-readline" -o 
"language/tree-il/fix-letrec.go" "../module/language/tree-il/fix-letrec.scm"
make[2]: *** [Makefile:1928: language/tree-il/fix-letrec.go] Error 134
GC Warning: Repeated allocation of very large block (appr. size 230096896):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 230096896):
        May lead to memory leak and poor performance
Too many heap sections: Increase MAXHINCR or MAX_HEAP_SECTS
Too many heap sections: Increase MAXHINCR or MAX_HEAP_SECTS
/gnu/store/kpxi8h3669afr9r1bgvaf9ij3y4wdyyn-bash-minimal-4.4.12/bin/bash: line 
6: 30839 Aborted                 GUILE_AUTO_COMPILE=0 ../meta/build-env guild 
compile --target="x86_64-unknown-linux-gnu" -O1 -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/module" -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/guile-readline" -o 
"language/value/spec.go" "../module/language/value/spec.scm"
make[2]: *** [Makefile:1928: language/value/spec.go] Error 134
/gnu/store/kpxi8h3669afr9r1bgvaf9ij3y4wdyyn-bash-minimal-4.4.12/bin/bash: line 
6: 30917 Aborted                 GUILE_AUTO_COMPILE=0 ../meta/build-env guild 
compile --target="x86_64-unknown-linux-gnu" -O1 -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/module" -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/guile-readline" -o 
"system/base/syntax.go" "../module/system/base/syntax.scm"
make[2]: *** [Makefile:1928: system/base/syntax.go] Error 134
GC Warning: Repeated allocation of very large block (appr. size 230096896):
        May lead to memory leak and poor performance
Too many heap sections: Increase MAXHINCR or MAX_HEAP_SECTS
Too many heap sections: Increase MAXHINCR or MAX_HEAP_SECTS
GC Warning: Repeated allocation of very large block (appr. size 230096896):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 230096896):
        May lead to memory leak and poor performance
/gnu/store/kpxi8h3669afr9r1bgvaf9ij3y4wdyyn-bash-minimal-4.4.12/bin/bash: line 
6: 30344 Aborted                 GUILE_AUTO_COMPILE=0 ../meta/build-env guild 
compile --target="x86_64-unknown-linux-gnu" -O1 -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/module" -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/guile-readline" -o 
"ice-9/psyntax-pp.go" "../module/ice-9/psyntax-pp.scm"
make[2]: *** [Makefile:1928: ice-9/psyntax-pp.go] Error 134
/gnu/store/kpxi8h3669afr9r1bgvaf9ij3y4wdyyn-bash-minimal-4.4.12/bin/bash: line 
6: 30354 Aborted                 GUILE_AUTO_COMPILE=0 ../meta/build-env guild 
compile --target="x86_64-unknown-linux-gnu" -O1 -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/module" -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/guile-readline" -o 
"srfi/srfi-1.go" "../module/srfi/srfi-1.scm"
make[2]: *** [Makefile:1928: srfi/srfi-1.go] Error 134
Too many heap sections: Increase MAXHINCR or MAX_HEAP_SECTS
GC Warning: Repeated allocation of very large block (appr. size 230096896):
        May lead to memory leak and poor performance
/gnu/store/kpxi8h3669afr9r1bgvaf9ij3y4wdyyn-bash-minimal-4.4.12/bin/bash: line 
6: 30548 Aborted                 GUILE_AUTO_COMPILE=0 ../meta/build-env guild 
compile --target="x86_64-unknown-linux-gnu" -O1 -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/module" -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/guile-readline" -o 
"language/cps/peel-loops.go" "../module/language/cps/peel-loops.scm"
make[2]: *** [Makefile:1928: language/cps/peel-loops.go] Error 134
GC Warning: Repeated allocation of very large block (appr. size 230096896):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 230096896):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 230096896):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 230096896):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 230096896):
        May lead to memory leak and poor performance
Too many heap sections: Increase MAXHINCR or MAX_HEAP_SECTS
/gnu/store/kpxi8h3669afr9r1bgvaf9ij3y4wdyyn-bash-minimal-4.4.12/bin/bash: line 
6: 31410 Aborted                 GUILE_AUTO_COMPILE=0 ../meta/build-env guild 
compile --target="x86_64-unknown-linux-gnu" -O1 -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/module" -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/guile-readline" -o 
"system/vm/dwarf.go" "../module/system/vm/dwarf.scm"
make[2]: *** [Makefile:1928: system/vm/dwarf.go] Error 134
Too many heap sections: Increase MAXHINCR or MAX_HEAP_SECTS
GC Warning: Repeated allocation of very large block (appr. size 230096896):
        May lead to memory leak and poor performance
/gnu/store/kpxi8h3669afr9r1bgvaf9ij3y4wdyyn-bash-minimal-4.4.12/bin/bash: line 
6: 31415 Aborted                 GUILE_AUTO_COMPILE=0 ../meta/build-env guild 
compile --target="x86_64-unknown-linux-gnu" -O1 -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/module" -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/guile-readline" -o 
"system/vm/elf.go" "../module/system/vm/elf.scm"
make[2]: *** [Makefile:1928: system/vm/elf.go] Error 134
Too many heap sections: Increase MAXHINCR or MAX_HEAP_SECTS
/gnu/store/kpxi8h3669afr9r1bgvaf9ij3y4wdyyn-bash-minimal-4.4.12/bin/bash: line 
6: 31028 Aborted                 GUILE_AUTO_COMPILE=0 ../meta/build-env guild 
compile --target="x86_64-unknown-linux-gnu" -O1 -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/module" -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/guile-readline" -o 
"ice-9/boot-9.go" "../module/ice-9/boot-9.scm"
make[2]: *** [Makefile:1928: ice-9/boot-9.go] Error 134
GC Warning: Repeated allocation of very large block (appr. size 230096896):
        May lead to memory leak and poor performance
Too many heap sections: Increase MAXHINCR or MAX_HEAP_SECTS
/gnu/store/kpxi8h3669afr9r1bgvaf9ij3y4wdyyn-bash-minimal-4.4.12/bin/bash: line 
6: 30348 Aborted                 GUILE_AUTO_COMPILE=0 ../meta/build-env guild 
compile --target="x86_64-unknown-linux-gnu" -O1 -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/module" -L 
"/tmp/guix-build-guile-awesome-2.2.2.drv-0/guile-2.2.2/guile-readline" -o 
"language/cps/intset.go" "../module/language/cps/intset.scm"
make[2]: *** [Makefile:1928: language/cps/intset.go] Error 134
…
--8<---------------cut here---------------end--------------->8---

I will try this on a workstation with fewer cores and less memory later.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net