bug#42162: Recovering source tarballs

[Ludovic Courtès]

Hello!

Timothy Sample <samplet@ngyro.com> skribis:

>> If we expose the database over HTTP (like over cgit), we can arrange so
>> that (guix download) simply GETs db.example.org/sha256/xyz.  No need to
>> fetch the whole database.
>>
>> It might be more reasonable to have a real database and a real service
>> around it, I’m sure Chris Baines would agree ;-), but we can choose URLs
>> that could easily be implemented by a “real” service instead of cgit in
>> the future.
>
> I got it working over cgit shortly after sending my last message.  :)  So
> far, I am very much on team “good enough for now”.

Wonderful.  :-)

>> Timothy Sample <samplet@ngyro.com> skribis:
>>
>>> I was imagining an escape hatch beyond this, where one could look up a
>>> provenance record from when Disarchive ingested and verified a source
>>> code archive.  The provenance record would tell you which version of
>>> Guix was used when saving the archive, so you could try your luck with
>>> using “guix time-machine” to reproduce Disarchive’s original
>>> computation.  If we perform database migrations, you would need to
>>> travel back in time in the database, too.  The idea is that you could
>>> work around breakages in Disarchive automatically using the Power of
>>> Guix™.  Just a stray thought, really.
>>
>> Seems to me it Shouldn’t Be Necessary?  :-)
>>
>> I mean, as long as the format is extensible and “future-proof”, we’ll
>> always be able to rebuild tarballs and then re-disassemble them if we
>> need to compute new hashes or whatever.
>
> If Disarchive relies on external compressors, there’s an outside chance
> that those compressors could change under our feet.  In that case, one
> would want to be able to track down exactly which version of XZ was used
> when Disarchive verified that it could reassemble a given source
> archive.

Oh, true.  Gzip and bzip2 are more-or-less “set in stone”, but xz, lzip,
or zstd could change.  Recording the exact version of the implementation
would be a good stopgap.

> Maybe I’m being paranoid, but if the database entries are being
> computed by the CI infrastructure it would be pretty easy to note the
> Guix commit just in case.

Yeah, that makes sense.  At least we could have “notes” in the file
format to store that kind of info.  Using CI is also a good idea.

>> I was thinking that it might be best to not use Guix for computations.
>> For example, have “disarchive save” not build derivations and instead do
>> everything “here and now”.  That would make it easier for others to
>> adopt.  Wait, looking at the Git history, it looks like you already
>> addressed that point, neat.  :-)
>
> Since my last message I managed to remove Guix as dependency completely.
> Right now it loads ‘(guix swh)’ opportunistically, but I might just copy
> the code in.  Directory references now support multiple “addresses” so
> that you could have Nix-style, SWH-style, IPFS-style, etc.  Hopefully my
> next message will have a WIP patch enabling Guix to use Disarchive!

Neat, looking forward to it!

Thank you,
Ludo’.