Re: qemu CI & ccache: cache size is too small

[Michael Tokarev]

27.05.2024 14:19, Thomas Huth wrote:
> On 27/05/2024 12.49, Michael Tokarev wrote:
> > Hi!
> >
> > Noticed today that a rebuild of basically the same tree (a few commits apart)
> > in CI result in just 11% hit rate of ccache:
> >
> > https://gitlab.com/mjt0k/qemu/-/jobs/6947445337#L5054
>
> For me, the results look better:
>
>   https://gitlab.com/thuth/qemu/-/jobs/6918599017#L4954

Yeah, it's a bit better, but still not good enough.
I dunno how much changes the source had between the two runs.
It still had 11 cleanups, and the cache size is at the same level.
(It is an older ccache, too).

> > while it should be near 100%.  What's interesting in there is:
> >
> > 1) cache size is close to max cache size,
> > and more important,
> > 2) cleanups performed 78
> >
> > so it has to remove old entries before it finished the build.
>
> Did you maybe switch between master and stable branches before that run? ... I guess that could have invalidated most of the cached files since we 
> switched from CentOS 8 to 9 recently...?

Nope, nothing else ran between the two and it was just a few
source-level commits (stable-8.2 pick ups), without changing
giltab/containers/etc configuration.

I increased cache size to 900M and did another test run, here are
the results: https://gitlab.com/mjt0k/qemu/-/jobs/6947894974#L5054

cache directory                     /builds/mjt0k/qemu/ccache
primary config                      /builds/mjt0k/qemu/ccache/ccache.conf
secondary config      (readonly)    /etc/ccache.conf
stats updated                       Mon May 27 11:17:44 2024
stats zeroed                        Mon May 27 11:10:22 2024
cache hit (direct)                  1862
cache hit (preprocessed)             274
cache miss                          1219
cache hit rate                     63.67 %
called for link                      285
called for preprocessing              71
compiler produced empty output         5
preprocessor error                     2
no input file                          6
cleanups performed                     0
files in cache                      9948
cache size                         654.6 MB
max cache size                     900.0 MB

This is having in mind that the previous run was with CCACHE_SIZE=500M
and had multiple cleanups, so 63% is actually more than I'd expect already.

Thanks,

/mjt

--
GPG Key transition (from rsa2048 to rsa4096) since 2024-04-24.
New key: rsa4096/61AD3D98ECDF2C8E  9D8B E14E 3F2A 9DD7 9199  28F1 61AD 3D98 
ECDF 2C8E
Old key: rsa2048/457CE0A0804465C5  6EE1 95D1 886E 8FFB 810D  4324 457C E0A0 
8044 65C5
Transition statement: http://www.corpit.ru/mjt/gpg-transition-2024.txt