bug#47543: “Repeated allocation of very large block” during ‘guix pull’

[John Kehayias]

Hello,

Just hit this warning and went searching to find this report.

On Thu, Nov 23, 2023 at 12:42 PM, Ludovic Courtès wrote:

> Ludovic Courtès <ludovic.courtes@inria.fr> skribis:
>
>> While running ‘guix pull’, sometime between the actual ‘git pull’ (via
>> Guile-Git) and channel authentication, I saw the dreaded libgc warning:
>>
>>   Repeated allocation of very large block
>
> [...]
>
>> So it might rather be ‘update-cached-checkout’ that triggers it.
>
> I wonder if this was due to the big checkouts we ended up with so far:
>
>   https://issues.guix.gnu.org/65720
>
> Ludo’.

In my case, just did a guix pull so I am on
2126dab4cd81db4cbde4566d8c638e45a4c0077c.

Pull was fine, but this warning was repeatable on a system reconfigure:

--8<---------------cut here---------------start------------->8---
❯ sudo guix system reconfigure ~/.config/guix/config.scm --max-jobs=4 -n
GC Warning: Repeated allocation of very large block (appr. size 69632):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 69632):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 69632):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 135168):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 131072):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 69632):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 102400):
        May lead to memory leak and poor performance
substitute: updating substitutes from 
'https://bordeaux-us-east-mirror.cbaines.net/'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://substitutes.nonguix.org'... 
100.0%
The following derivation would be built:
  /gnu/store/r9n8vz4s1n7v3551cphrqy0pqyl8wcv0-corectrl-1.3.5.drv

340.3 MB would be downloaded
❯ sudo guix system reconfigure ~/.config/guix/config.scm -n
GC Warning: Repeated allocation of very large block (appr. size 69632):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 122880):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 69632):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 69632):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 69632):
        May lead to memory leak and poor performance
The following derivation would be built:
  /gnu/store/r9n8vz4s1n7v3551cphrqy0pqyl8wcv0-corectrl-1.3.5.drv

340.3 MB would be downloaded
❯ sudo guix system reconfigure ~/.config/guix/config.scm --max-jobs=4
GC Warning: Repeated allocation of very large block (appr. size 69632):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 69632):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 69632):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 131072):
        May lead to memory leak and poor performance
The following derivation will be built:
  /gnu/store/r9n8vz4s1n7v3551cphrqy0pqyl8wcv0-corectrl-1.3.5.drv

340.3 MB will be downloaded
 adwaita-icon-theme-44.0  904KiB                                                
                    108.0MiB/s 00:00 ▕██████████████████▏ 100.0%
guix substitute: warning: download from 
'https://bordeaux-us-east-mirror.cbaines.net/nar/lzip/c2brbsvn75866l81rswb7z8sdpw2dvjr-font-abattis-cantarell-0.303-0.e049149'
 failed: 404, "Not Found"
retrying download of 
'/gnu/store/c2brbsvn75866l81rswb7z8sdpw2dvjr-font-abattis-cantarell-0.303-0.e049149'
 with other substitute URLs...

substitution of
/gnu/store/5in68ily7wjj8jy67kyj8sikwmci894j-ath9k-htc-ar7010-firmware-1.4.0
complete
--8<---------------cut here---------------end--------------->8---

this last one continued until I killed it to send capture the previous
output and send it.

--8<---------------cut here---------------start------------->8---
❯ free -h
               total        used        free      shared  buff/cache   available
Mem:            15Gi        10Gi       3.4Gi       706Mi       2.4Gi       4.8Gi
Swap:           15Gi       4.1Gi        11Gi
--8<---------------cut here---------------end--------------->8---

But since interrupting the reconfigure, it doesn't give the warning
again. So something happened by actually starting the reconfigure
process...?

I don't think I've ever seen this before. Hope this helps!

John