[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: substitute derivation: also substitute grafts?
From: |
Maxime Devos |
Subject: |
Re: substitute derivation: also substitute grafts? |
Date: |
Thu, 15 Sep 2022 17:06:31 +0200 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.12.0 |
On 15-09-2022 16:46, Csepp wrote:
Ricardo Wurmus <rekado@elephly.net> writes:
[...]
Did I say *all items*? Well, … grafts are not included, because graft
derivations are marked as not substitutable.
Can we change that conditionally? I would really like to avoid having
to build grafts on B when they have already been built on A.
I would love this too, because IO can be incredibly slow on HDDs and
large packages. My netbook would be thankful.
There are some some opportunities for optimizations in the grafting code
before substituting more -- for example, to avoid seek times, it would
be possible to rewrite multiple files concurrently (maybe using
'par-for-each' to process each file in a directory in parallel).
This is already done in (guix build grafts), except that:
* 'find-files' itself is not parallelised, even though parallelising it
could potentially reduce the time spent seeking (*)
* it uses (parallel-job-count), which is (IIUC) 1 by default because
--cores=1 by default. As grafts are more IO intensive than CPU
intensive, maybe it would be reasonable to impose a _minimum_ amount
of parallelism, I don't, know, 2 or 4 or so?
(I'm assuming the main problem here is seek times.)
Also combinable with the proposal of having substitutes for grafts.
Greetings,
Maxime.
(*) IIUC, the time for N parallel seeks should, in theory, ≃ 1 seek, for
small values of N, because of elevator algorithms.
OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key
OpenPGP_signature
Description: OpenPGP digital signature