Speeding up “guix pull”: splitting modules

[Ricardo Wurmus]

Hi Guix,

thanks to Ludo’s excellent work on “guix pull”, updating Guix has become
a lot faster than in the old days.  With build farm support one usually
only needs to wait a little while to be able to download all parts of
an up-to-date Guix.

Unfortunately, changes to the repository invalidate expensive
derivations regularly, and waiting for the build farm to get around to
building these derivations gets old quickly.  Running “guix pull”
repeatedly is a drag because the chances of getting a fully remotely
built Guix are pretty slim.

Part of the problem is that there’s just one derivation for the
compilation of all package modules.  This one derivation’s output is an
input to another expensive derivation: the one for the Guix System
(services, etc).

We should aim to split up the guix-packages derivation, so that the
guix-system derivation will not have to be rebuilt when its packages
aren’t affected.  Unfortunately, it’s difficult to build modules
independently from one another because of the many inter-module
dependency cycles.

Gábor once suggested an iterative approach of identifying the most
important nodes in the package graph that should be moved to their own
modules, so that we would end up with a package graph that looks less
like a hair ball.  I think it would useful to get a better view on the
relationships between modules.

On the other hand: this would need to be an ongoing effort.  Newly
introduced packages or even new features might create complex module
cycles.  It sounds tedious to keep track of this and to enforce
boundaries.

Is there an alternative?  Could we, for example, make all lookups of
cross-module package references lazy?  Or could we improve compilation
times by disabling optimizations?  Or by marking the modules as
declarative and thus unlock more optimizations (with Guile 3)?

--
Ricardo