Re: Speed up grafts by storing reference offset in index

[Ludovic Courtès]

Hi!

Ricardo Wurmus <rekado@elephly.net> skribis:

> Since it is December and I'm in a silly mood here is a silly idea: would
> it make sense to shift parts of the grafting work to an offloadable
> build?  Here's what I imagine:
>
> - on the build farms build an additional derivation for a references
>   file.  The references file is an S-expression containing a list of
>   tuples of the form (FILE-NAME OFFSET).  Each of these tuples
>   identifies the location of a single reference at the recorded byte
>   OFFSET in FILE-NAME.
>
> - when computing grafts, don't search the local files sequentially for
>   references but look them up in the references file.  Instead of
>   computing the reference file substitute it from a build server.

This sounds quite ambitious and it’s unclear that this would be
beneficial (it would be beneficial *if* scanning for references is
substantially more expensive than just copying the part of the file that
would be scanned, and it’s far from obvious that this holds.)

I have another, more down-to-earth proposal: ungraft more often!  That’s
the spirit of the auto-ungraft manifest and jobset:
<https://issues.guix.gnu.org/74654>… but it doesn’t quite work as
expected because of ‘rust-ring’ shenanigans:
<https://lists.gnu.org/archive/html/guix-devel/2024-12/msg00113.html>.

(Making grafting faster would still be welcome, but I’d rather look for
a “local” optimization in the code itself.)

Ludo’.