[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[bug#68266] [PATCH v2] guix: store: Add report-object-cache-duplication.
From: |
Ludovic Courtès |
Subject: |
[bug#68266] [PATCH v2] guix: store: Add report-object-cache-duplication. |
Date: |
Fri, 12 Jan 2024 15:22:07 +0100 |
User-agent: |
Gnus/5.13 (Gnus v5.13) |
Christopher Baines <mail@cbaines.net> skribis:
> This is intended to help with spotting duplication in the object cache, so
> where many keys, for example package records map to the same derivation. This
> represents an opportunity for improved performance if you can reduce this
> duplication in the cache, and better take advantage of the already present
> cache entries.
Another way to detect this is by looking at ‘add-data-to-store-cache’
stats:
--8<---------------cut here---------------start------------->8---
$ GUIX_PROFILING="add-data-to-store-cache object-cache" guix build --no-grafts
greetd wlgreet du-dust circtools --target=aarch64-linux-gnu -d
/gnu/store/mivzv83wryv9gp5bjncg5m1831dx2xwr-circtools-1.0.0.drv
/gnu/store/4xf7kh9mi0vpvs8m1ak4x8w1rpsdpv6z-du-dust-0.8.6.drv
/gnu/store/ayk54gvlbc1qam6irzf9kaig56dhzni0-wlgreet-0.4.1.drv
/gnu/store/r601i40cii9ic5w1k4hy5c2yngfayh64-greetd-0.9.0.drv
Object Cache:
fresh caches: 22
lookups: 40435
hits: 36821 (91.1%)
cache size: 3613 entries
'add-data-to-store' cache:
lookups: 4090
hits: 958 (23.4%)
.drv files: 3062 (74.9%)
Scheme files: 916 (22.4%)
$ GUIX_PROFILING="add-data-to-store-cache object-cache" ./pre-inst-env guix
build --no-grafts greetd wlgreet du-dust circtools --target=aarch64-linux-gnu
-d
/gnu/store/1wsldmvigjb8w2gk418npbnfznlb0ck1-circtools-1.0.0.drv
/gnu/store/b5c73fawjdvkgy431qxz9l6l9y9a9lhz-du-dust-0.8.6.drv
/gnu/store/zwc7qzsbzf62dgbbzy74lki4hsr406bw-wlgreet-0.4.1.drv
/gnu/store/vjdd23hc82701afb132z1ajcqa7hfd74-greetd-0.9.0.drv
Object Cache:
fresh caches: 22
lookups: 37942
hits: 34523 (91.0%)
cache size: 3418 entries
'add-data-to-store' cache:
lookups: 3895
hits: 763 (19.6%)
.drv files: 2957 (75.9%)
Scheme files: 826 (21.2%)
--8<---------------cut here---------------end--------------->8---
Ideally, the hit rate there would be 0% and we could remove it.
If there’s a positive hit rate, it means we keep adding the same .drv
and/or *-builder files to the store, meaning that the object cache was
ineffective.
> +(define* (report-object-cache-duplication store #:key (threshold 10)
> + (port (current-error-port)))
Do you have an example output of this?
How helpful does it look to you in practice?
I guess it we’d need to add a ‘register-profiling-hook!’ call somewhere?
If we were to add this, I’d suggest adding a docstring and following the
same style as the rest of the code (no long identifiers for local
variables and no car/cdr, for example).
Thanks,
Ludo’.
- [bug#68266] [PATCH 7/7] packages: rust: Memoize make-rust-sysroot results., (continued)
[bug#68266] [PATCH 6/7] gnu: Memozise cross-libc results., Christopher Baines, 2024/01/05
[bug#68266] [PATCH 1/7] gnu: Memozise make-ld-wrapper results., Ludovic Courtès, 2024/01/08
[bug#68266] [PATCH v2] guix: store: Add report-object-cache-duplication., Christopher Baines, 2024/01/10
- [bug#68266] [PATCH v2] guix: store: Add report-object-cache-duplication.,
Ludovic Courtès <=