Re: Prototype tool for building derivations

[Ludovic Courtès]

Hello!

Christopher Baines <address@hidden> skribis:

> In terms of getting to a point where Guix packages build reliably and
> reproducibly, I think more testing is what's going to help. By taking
> packages and building them more, on a wide variety of hardware and
> software configurations, we'll get data on what works, what doesn't, and
> where improvements and fixes can be made.
>
> It's very much a prototype, but I've pushed some code up here [1] now,
> the README.org file [2] contains usage instructions as well as a
> description of the architecture.
>
> 1: https://git.cbaines.net/guix/build-coordinator/
> 2: https://git.cbaines.net/guix/build-coordinator/about/
>
> So far, I've mostly done the boring stuff, but I'm excited about what
> this could support.

Neat!  I like the architecture and the fact that it’s easy to track down
what was built and where.  (guix scripts offload) currently doesn’t save
that info.

A note about the usage as explained in the README: be sure to register
GC roots for derivations before passing them around processes.  :-)

Having an HTTP interface is really nice (I recently had someone ask
whether one could coordinate offloading over HTTP rather than SSH, this
is helpful in some contexts.)

That also makes me wonder whether we could implement some of the store
RPCs over HTTP (I think Nix does something along these lines for
substitutes nowadays).  Because the coordinator interface is in fact
close to a subset of the daemon’s RPC interface.

> Because the allocation/scheduling of builds is controlled, this offers
> the possibility of doing some builds before others. If you were using
> this for providing substitutes for example, it could be valuable to try
> and prioritise building things that are requested more often, or those
> that are more expensive (in time or space) to build.

Yup, this highlights another shortcoming of ‘guix offload’ as well as
Cuirass actually, whereby there’s no way to observe scheduling decisions
nor to influence them dynamically.

> Often there are concurrency issues with builds, I want to add a way of
> specifying where builds should run. This would make it easy to test
> building the same derivation in different setups, then capture where it
> succeeds, fails, and how the output differs (if at all) across the
> different environments.

Yup.

> I think it would be good to get point where there are many different
> individuals and groups providing independent sources of Guix packages,
> such that users can have a high level of confidence that the substitutes
> they're getting correspond to the source code. Getting there will be
> easier if substitute servers are easy to operate, and part of that I
> think comes down to how easy it is to see what's going on. With the
> current daemon implementation, I'm not sure how to get much data out
> (this could be possible, I haven't looked very closely). This approach
> however where the scheduling is done outside the daemon makes the
> information more accessible.

That’s a worthy goal!  I’m not sure the coordinator is necessarily
helping directly there, because it’s another component (or two!) to set
up, in addition to ‘guix publish’ and something like Cuirass that
monitors a channel and builds it.

However I think it could be a way to restructure (guix scripts offload)
and/or Cuirass: they could talk to the coordinator instead of doing
their own thing.

In fact, I think it would be quite easy to reimplement (guix scripts
offload) using the coordinator (see ‘process-request’ for the protocol
with the daemon), and it would be interesting to see how that works.

> I think some of the design decisions here are quite short sighted. I
> think it would be better if some of this functionality could be handled
> by the guix-daemon, especially things like providing information on
> builds that are going to happen, but haven't happened yet. Once there's
> a Guile implementation of the guix-daemon, hopefully some of this
> technical debt can be repaid.

Yup!

Lots of food for thought!  :-)

Thanks,
Ludo’.