Re: Gitlab CI caching is not working

[Daniel P . Berrangé]

On Thu, Sep 12, 2024 at 07:38:37AM +0200, Thomas Huth wrote:
> 
>  Hi!
> 
> While looking at some recent CI jobs, I noticed that the caching of the
> Gitlab-CI jobs is not working at all anymore. In the build jobs, the ccache
> saving is not working and causing a complete cache miss of each compile:
> 
>  https://gitlab.com/qemu-project/qemu/-/jobs/7802183187#L5328
> 
> And, maybe more important, in the avocado/functional jobs we don't cache the
> assets anymore, causing a re-download of multiple gigabytes each time:
> 
>  https://gitlab.com/qemu-project/qemu/-/jobs/7802183251#L29
> 
> (the du -chs in line 35 is not executed, thus the cache is nonexistent)
> 
> The problem is not new, it's been there for some weeks already, e.g. here's
> a run from the last freeze period (when the job was only running avocado
> tests):
> 
>  https://gitlab.com/qemu-project/qemu/-/jobs/7753544153#L86
> 
> There is a suspicious message at the beginning of the logs:
> 
>  "No URL provided, cache will not be downloaded from shared cache server.
> Instead a local version of cache will be extracted."
> 
> ... but since we use throw-away containers for building, I guess there is no
> local version of the cache?
> 
> Anyway, the problem only exists for the k8s runners, in my private clone of
> the repository that uses shared runners from gitlab, the caching is working
> right.

Right, by default caches are local to the runner's execution environment.
If you want a distributed cache, that has to be turned on in the runner
configuration:

  
https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|