[bug#50960] [PATCH 10/10] shell: Maintain a profile cache.

[Ludovic Courtès]

Maxime Devos <maximedevos@telenet.be> skribis:

> Ludovic Courtès schreef op za 02-10-2021 om 16:12 [+0200]:
> [reordered]
>> > So I think the cache should also check if these dependencies have been 
>> > modified.
>> > To keep track of the dependencies, something like the ‘compile-all,compile:
>> > Keep track of dependencies of compiled modules.’ patch from
>> > <https://issues.guix.gnu.org/50384> could be used, though the 'include', 
>> > 'load',
>> > 'include-from-path' and maybe 'use-modules' (if something like "guix shell 
>> > -Lextra-modules-directory ..."
>> > is done) macros would need to be replaced.
>> 
>> Problem is that any attempt to keep track of dependencies is always an
>> approximation because macros can do anything—I can have my own macro
>> that reads files at expansion time.  So I’m inclined to not even try.
>
> I expect most people use 'include', 'load', 'include-from-path' or 
> 'load-from-path'
> instead of writing their own macro reading files at expansion time, and if 
> they do
> write their own macro, I expect it would be implemented in terms of the 
> former anyway,
> so I expect replacing these macros and 'use-modules' to be sufficient, 
> especially
> if the flag proposed below is available as ‘escape hatch’ for when the 
> dependency
> tracking is insufficient.
>
> This is not merely an expansion-time problem, files loaded at 'load' or 
> 'eval' time
> are important as well.  E.g., consider the case where the manifest is 
> something like
>
>   (list
>     (package
>       (inherit stuff)
>       (source (local-file "stuff" #:recursive? #t)))
>     other-packages ...)

In a ‘guix.scm’ file, ‘source’ doesn’t matter.  It does matter in a
manifest, but that sounds a bit less common.

> then all files inside 'stuff' are important as well.  I don't see how 
> dependencies
> could be tracked here without an excessive amount of 'stat' calls, maybe guix 
> should
> ignore these dependencies (possibly with a warning, and a reference to the 
> manual
> documenting which dependencies are tracked and which are not?). 

[...]

> A documented flag to always consider the cache stale seems good, though I 
> think
> at least the dependencies made with the common macros and procedures 
> 'include',
> 'load', 'include-from-path', 'load-from-path', 'use-modules' and non-recursive
> 'local-file' could be tracked, though this could be left as a TODO for later
> I suppose.

Tracking those uses reliably is impossible: there could be same-named
bindings that do other things, there could be custom macros, there could
be “dynamic arguments” (whose value is not known statically), etc.  You
have to expand + evaluate the code to get better results, and even then,
there might be different paths in the code so you can’t be sure you got
it right.

We could get an approximation for common uses by recognizing special
forms as you suggest.  But it’s just that, an approximation.

In such situations, I err on the side of not even trying.  The added
complexity for a flaky result doesn’t pay off to me.  I prefer to be
upfront, document limitations, and let users handle them as they see
fit.

WDYT?

A similar problem occurs with system provenance tracking, which saves
‘configuration.scm’ but leaves it up to the user to preserve additional
files if needed (info "(guix) Service Reference").

Thanks,
Ludo’.