bug#70663: nss@3.99 is really hard to build

[Maxim Cournoyer]

Hi,

Christopher Baines <mail@cbaines.net> writes:

[...]

>> I think there's two issues here, was this spotted before merging, and
>> what if anything can be done about this now. Where there's not a
>> substitute available for nss@3.99, this will affect guix pull/guix
>> time-machine, e.g.
>>
>>   → guix time-machine --commit=72308f262c910977e40c2c9f350dc563c0a8437a -- 
>> describe
>>   Updating channel 'guix' from Git repository at 
>> 'https://git.savannah.gnu.org/git/guix.git'...
>>   substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 
>> 100.0%
>>   substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 
>> 100.0%
>>   substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 
>> 100.0%
>>    nss-3.99.tar.xz  55.2MiB                                                  
>>                                            13.7MiB/s 00:04 
>> ▕██████████████████▏ 100.0%
>>   building /gnu/store/8379qa0y6s7ssjr8gplm5fyw9r5pnxhn-nss-3.99.0.drv...
>
> So with the changes in #70693 merged, this issue should be fixed going
> forward, but the revisions with the broken nss are going to be affected
> forever and thus the impact is going to drag on for a while. For
> example, data.guix.gnu.org is going to be struggling to process the
> revisions with the broken nss for a long while to come.
>
> Before closing this bug, it would be good to understand more about how
> this happened and from that try to think if anything can be done to
> prevent similar issues in the future?
>
> At least from what I can see on the issues, the problem was introduced
> with the update to 3.98.0 [3] and then continued with the update to 3.99
> [4]. Given the changes in 70662 were sent to guix-patches and then
> merged less than 24 hours later, I'd imagine that wasn't sufficient time
> for data.qa.guix.gnu.org to fail attempting to build nss.

I think in [3] you meant https://issues.guix.gnu.org/70569, not #70662.

Since this was security sensitive, I built it on x86_64, tested it there
to ensure that IceCat worked as expected, had others confirmed it worked
for them on #guix then pushed.

In the past, I've had more patience waiting for QA to build things, but
since this is not guaranteed (it sometimes never happened), it seems
reasonable to me to promptly push security fixes that were manually
built & tested and adjust for any breakage later, if there is any.

> 3: https://issues.guix.gnu.org/70662
> 4: https://issues.guix.gnu.org/70618
>
> Had the changes waited for longer, then these failures should have been
> spotted by QA, I would guess that the revision might have failed to be
> processed, and if it was processed successfully, the nss failures should
> have shown up, so maybe we should start requiring [5] that not only are
> changes sent to guix-patches@gnu.org, but that QA processes them (to
> some extent) before merging?

I have some apprehensions about that; given the QA build farm is
somewhat under-resourced for builds, I fear security changes could be
gated for longer periods of time than is reasonable to wait.  If we go
that route, I think we should dedicate more hardware first.

-- 
Thanks,
Maxim