--- Begin Message ---
|
Subject: |
[PATCH 0/1] 'guix publish --cache' can publish items not yet cached |
|
Date: |
Sat, 24 Oct 2020 16:49:29 +0200 |
Hello!
The ‘--cache’ mode of ‘guix publish’ is nice for many reasons. Until
now though, it would only return 200 to narinfo and nar requests
corresponding to items already in cache—already “baked”. Thus, the
first narinfo request for an item would always return 404; one would
have to wait until the item is baked to get 200 and download the
substitute.
If you’re fetching substitutes for popular packages on a popular
instance, the extra delay is most likely invisible. However, if you’re
using an instance with few users, or if you’re interesting in
substitutes for an package that’s not popular, the behavior described
above is a showstopper. (Many people here have criticized it in the
past. :-))
This patch changes the behavior of ‘--cache’: if a store item is not
yet in cache, and if it’s “small enough”, then narinfo/nar requests for
it immediately return 200, as in the no-cache mode. You’re trading
possibly increased server resource usage for reduced publication delay.
To put an upper bound on the extra resource usage, the
‘--cache-bypass-threshold’ allows users to control the maximum size
of a store item that can receive this treatment.
An interesting use case that can benefit from this change is .drv
substitutes: .drv themselves can be substitutes, so you can run:
guix build /gnu/store/….drv
and the daemon will fetch the .drv and its closure and start
building it (since commit 9c9982dc0c8c38ce3821b154b7e92509c1564317).
It’s not something we use much so far, but maybe we could put it
to good use in the future (I know Chris has been playing with it
in the context of the Data Service).
Thoughts?
Ludo’.
Ludovic Courtès (1):
publish: Add '--cache-bypass-threshold'.
doc/guix.texi | 24 +++++++++++-
guix/scripts/publish.scm | 85 ++++++++++++++++++++++++++++++++--------
tests/publish.scm | 43 ++++++++++++++++++--
3 files changed, 130 insertions(+), 22 deletions(-)
--
2.28.0
--- End Message ---
--- Begin Message ---
|
Subject: |
Re: [bug#44193] [PATCH 1/1] publish: Add '--cache-bypass-threshold'. |
|
Date: |
Wed, 28 Oct 2020 16:26:48 +0100 |
|
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) |
Ludovic Courtès <ludo@gnu.org> skribis:
> * guix/scripts/publish.scm (show-help, %options): Add
> '--cache-bypass-threshold'.
> (low-compression): New procedure.
> (cache-bypass-threshold): New parameter.
> (bypass-cache?): New procedure.
> (render-narinfo/cached): Call 'render-narinfo' when 'bypass-cache?'
> returns true.
> (render-nar/cached): Call 'render-nar' when 'bypass-cache?' returns
> true.
> (guix-publish): Parameterize 'cache-bypass-threshold'.
> * tests/publish.scm ("with cache", "with cache, lzip + gzip")
> ("with cache, uncompressed"): Pass '--cache-bypass-threshold=0'.
> ("with cache, vanishing item"): Expect 200 for RESPONSE.
> ("with cache, cache bypass"): New test.
> ---
> doc/guix.texi | 24 +++++++++++-
> guix/scripts/publish.scm | 85 ++++++++++++++++++++++++++++++++--------
> tests/publish.scm | 43 ++++++++++++++++++--
> 3 files changed, 130 insertions(+), 22 deletions(-)
Pushed as ecaa102a58ad3ab0b42e04a3d10d7c761c05ec98.
Ludo’.
--- End Message ---