--- Begin Message ---
|
Subject: |
whishlist: time-machine --channel falls back to SWH |
|
Date: |
Sat, 24 Oct 2020 00:17:51 +0200 |
Dear,
Let’s describe the use case. Consider that:
guix time-machine -C channels -- install foo
is provided in some documentation, say scientific paper. Where the
channels.scm file is completly described:
--8<---------------cut here---------------start------------->8---
(list (channel
(name 'kikoo)
(url "https://example.org/that-great.git";)
(commit
"353bdae32f72b720c7ddd706576ccc40e2b43f95")))
--8<---------------cut here---------------end--------------->8---
In the future, if https://example.org/that-great.git disappears, then
build/install the package ’foo’ is becoming difficult, nor impossible.
However, let’s consider that the repo ’that-great’ had been saved in SWH
(say manually); since it is a regular Git repo. Guix should be able to
fallback to it transparently.
Obviously, another whislist is to have something to ease the save
request of the channel on SWH. Maybe this latter could be part of the
several-times discussed “guix channel” subcommand. :-)
All the best,
simon
--- End Message ---
--- Begin Message ---
|
Subject: |
Re: bug#44187: Channel clones lack SWH fallback |
|
Date: |
Sat, 18 Sep 2021 23:10:12 +0200 |
|
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) |
Hello!
zimoun <zimon.toutoune@gmail.com> skribis:
> The original URL of the channel was:
> <https://github.com/zimoun/channel-example.git>. And this channel
> defines a package where the upstream has also disappeared
> <https://github.com/zimoun/hello-example.git>. Note the URL in the
> package definition is not bogus… but using one was already working. :-)
>
> All is saved on SWH, so now all is transparent! From my point of view,
> this is a killer feature for scientific folks. :-)
Yay! Great that you came up with a nice example to test it on!
>> First, fallback is implemented only for fresh clones, not for updates.
>> Thus, if I rerun the first example, having now the clone in
>> ~/.cache/guix/checkouts, with a different commit, I get:
>
> SWH is not a forge but an archive. :-) Therefore, this update case does
> not make sense to me. I mean,
>
> $ git -C
> ~/.cache/guix/checkouts/6k7wvrcpbdsw3pje5b4squybw3jfn3viyrj7gcl7fipa5yjflaza
> fetch
> fatal: dépôt 'http://example.org/sdf/' non trouvé
Right, that’s a reasonable limitation.
> Well, maybe this cache could be removed if the commit is not found
> inside this cache and retry to fetch it from SWH. Obviously, the
> downdate case works.
It’s still useful to keep it cached around in case the user is going to
use it several times in a row.
> Note that on fresh clone, the error message could be improved:
>
> $ ./pre-inst-env guix build guix --with-git-url=guix=https://example.org
> --with-commit=guix=ff613c2b68aac539262822490448e637d8f315ba -n
> updating checkout of 'https://example.org'...
> guix build: error: Git failure while fetching https://example.org: unexpected
> http status code: 404
>
>
> where https://example.org is bogus and
> ff613c2b68aac539262822490448e637d8f315ba is not yet archived on SWH. It
> could be nice to warn in addition to the 404 that it is not found in
> SWH. WDYT?
Agreed; I’ve made this change (actually ‘swh-download’ prints something
upon failure since commit 60b42bec8413aa9844e625fb1903257f1bc1e55c, but
it looks more like a debugging message.)
> $ guix build guix --with-git-url=guix=https://example.org
> --with-commit=guix=c75b30d58f0becb0a5cd6a8bfe69d1063b0d1ada -n
> updating checkout of 'https://example.org'...
> SWH: found revision c75b30d58f0becb0a5cd6a8bfe69d1063b0d1ada with directory
> at
> 'https://archive.softwareheritage.org/api/1/directory/ca2e8a7222b4850c7bea935dff86b9c2a905efd6/'
> SWH vault: requested bundle cooking, waiting for completion...
> SWH vault: Processing...
> [...]
>
>
> then after several hours, I get this:
>
> SWH vault: failure: Internal Server Error. This incident will be reported.
> SWH vault: retrying...
> SWH vault: requested bundle cooking, waiting for completion...
> SWH vault: Processing...
>
> and after more than 12h, the status is still: «SWH vault: Processing...»
> and nothing is complete.
Did it eventually succeed? We obviously have no guarantee as to how
long it might take to cook a bundle.
> About this ’keyring’ branch, somehow it could be as a separated repo, so
> why not effectively do it. :-) I mean, get the branch as it is and
> mirror this branch in another Git repo saved on SWH; fallback to it if
> ’keyring’ branch is not there. I do not know… Or simply wait that SWH
> improves their things. :-)
Yeah, they’re planning to support it eventually.
>> *Third, and this answers the asterisk above, we must keep in mind that
>> this is content-addressibility *with SHA1*. Generating a chosen-prefix
>> collision is becoming affordable³, so users absolutely need an additional
>> mechanism to authenticate code they fetched.
[...]
> How a chosen-prefix attack could work here? I understand why the second
> preimage attack is an issue. But I miss how the SHA-1 chosen-prefix attack
> could be exploited here to compromise the user, because this hash is provided
> by this very same user.
I think you’re right, it’s rather second-preimage attacks that would be
a serious problem. My point is: as time passes, assuming that a SHA1
resolves to a single revision on SWH is becoming more and more
questionable.
>> swh: Support downloads of bare Git repositories.
>> git: 'update-cached-checkout' can fall back to SWH when cloning.
>> git: 'reference-available?' recognizes 'tag-or-commit'.
I’ve pushed this after adding the warning as you suggested:
dce2cf311b * git: 'reference-available?' recognizes 'tag-or-commit'.
05f44c2d85 * git: 'update-cached-checkout' can fall back to SWH when cloning.
6ec81c31c0 * swh: Support downloads of bare Git repositories.
Thanks a lot for reviewing and testing on real-world examples!
Ludo’.
--- End Message ---