[bug#47336] Disarchive as a fallback for downloads

[Ludovic Courtès]

Hi!

Timothy Sample <samplet@ngyro.com> skribis:

> * guix/download.scm (%disarchive-mirrors): New variable.
> (%disarchive-mirror-file): New variable.
> (built-in-download): Add 'disarchive-mirrors' keyword argument and
> pass its value along to the 'builtin:download' derivation.
> (url-fetch): Pass '%disarchive-mirror-file' to 'built-in-download'.
> * guix/scripts/perform-download.scm (perform-download): Read
> Disarchive mirrors from the environment and pass them to
> 'url-fetch'.
> * guix/build/download.scm (disarchive-fetch/any): New procedure.
> (url-fetch): Add 'disarchive-mirrors' keyword argument, use it to
> make a list of URIs, and use the new procedure to fetch the file if
> all other methods fail.

[...]

> +  #:use-module (guix base16)
>    #:use-module (guix base64)
>    #:use-module (guix ftp-client)
>    #:use-module (guix build utils)
>    #:use-module (guix progress)
> +  #:use-module (guix swh)

Maybe #:autoload them.

> +(define* (disarchive-fetch/any uris file
> +                               #:key (timeout 10))
> +  "Fetch a Disarchive specification from any of URIS, assemble it,
> +and write the output to FILE."
> +  (define (fetch-specification uris)
> +    (any (lambda (uri)
> +           (false-if-exception*
> +            (let-values (((port size) (http-fetch uri
> +                                                  #:verify-certificate? #t
> +                                                  #:timeout timeout)))

Perhaps add #:key (verify-certificate? #t) and have the caller pass it?
Currently (guix scripts perform-download) sets it to #f, which is a good
idea IMO.

> +  (match (and=> (resolve-module '(disarchive) #:ensure #f)
> +                (lambda (disarchive)
> +                  (cons (module-ref disarchive '%disarchive-log-port)
> +                        (module-ref disarchive 'disarchive-assemble))))
> +    (#f #f)
> +    ((%disarchive-log-port . disarchive-assemble)
> +     (format #t "Trying to use Disarchive to assemble ~a~%" file)
> +     (match (fetch-specification uris)
> +       (#f #f)
> +       (spec (parameterize ((%disarchive-log-port (current-output-port)))
> +               (disarchive-assemble spec file #:resolver resolve)))))))

So we would normally arrange so that the ‘guix’ package depends on
Disarchive, such that the above ‘resolve-module’ call works when done
via ‘guix perform-download’, right?

In the #f case, perhaps we should print something like “Disarchive not
found, bailing out”?

That’s all I have to say; it looks great to me!

That’s quite a milestone, it’d be great to have that in the upcoming
release.  Next we can discuss how to populate the Disarchive database
and where to do that (or your hosting fees could easily skyrocket :-)).
I suppose we could run that in Berlin and/or we could make an argument
about using SWH or Inria resources for that.

Thanks,
Ludo’.