--- Begin Message ---
|
Subject: |
[PATCH 00/12] Better source code recovery from SWH |
|
Date: |
Fri, 23 Feb 2024 15:22:51 +0100 |
Hello Guix!
This patch series improves source code recovery from SWH, as a followup
to <https://issues.guix.gnu.org/68741>.
It does several things:
• ‘guix lint -c archival’ now emits save requests for VCSes other
than Git.
• Fix <https://issues.guix.gnu.org/69070>.
• Allow content-addressed recovery of Mercurial and Subversion
checkouts.
• Allow Bazaar recovery using ‘download-nar’ (I didn’t bother with SWH).
• Have all these things honor the ‘GUIX_DOWNLOAD_SEQUENCE’ environment
variable.
You can try the various methods like this:
GUIX_DOWNLOAD_SEQUENCE=nar ./pre-inst-env guix build -S apl --check
GUIX_DOWNLOAD_SEQUENCE=swh ./pre-inst-env guix build -S guile-wisp --check
GUIX_DOWNLOAD_SEQUENCE=swh ./pre-inst-env guix build -S guile-gcrypt --check
In the last case, note that you must be running guix-daemon for the checkout
since that uses “builtin:git-download”, which is implemented on the server
side.
There’s a few caveats:
• Mercurial SWH fallback almost works, but not quite, due to this SWH bug:
<https://gitlab.softwareheritage.org/swh/infra/sysadm-environment/-/issues/5256>.
• Right now, no Subversion checkout has the nar-sha256 ExtID at SWH for
unclear reasons, so retrieving the source of ‘apl’ (say) from SWH
doesn’t work yet.
• Multi-directory Subversion downloads (‘svn-multi-fetch’) is not supported
yet. For that we’ll need to arrange with our SWH friends so they
compute nar-sha256 ExtIDs for combined directories (and we’ll have to
include that info in ‘sources.json’).
Feedback welcome!
Ludo’.
Ludovic Courtès (12):
lint: Switch to SRFI-71.
lint: archival: Fix crash in non-Git case.
lint: archival: Trigger “Save Code Now” for VCSes other than Git.
swh: Add ‘type’ field to <visit>.
swh: ‘origin-visits’ takes an optional ‘max’ parameter.
swh: ‘lookup-origin-revision’ handles branches pointing to
directories.
hg-download: Use ‘swh-download-directory-by-nar-hash’.
svn-download: Use ‘swh-download-directory-by-nar-hash’.
bzr-download: Implement nar fallback.
download-nar: Distinguish ‘output’ and ‘item’ parameter.
perform-download: Allow use of ‘download-nar’ for ‘--check’ builds.
download: Honor ‘GUIX_DOWNLOAD_SEQUENCE’ environment variable.
guix/build/bzr.scm | 3 +-
guix/build/download-nar.scm | 12 +--
guix/build/download.scm | 50 +++++++---
guix/build/git.scm | 27 ++++--
guix/bzr-download.scm | 57 ++++++++---
guix/cvs-download.scm | 24 +++--
guix/download.scm | 53 ++++-------
guix/git-download.scm | 20 ++--
guix/hg-download.scm | 36 ++++---
guix/lint.scm | 151 +++++++++++++++++++-----------
guix/scripts/perform-download.scm | 65 +++++++------
guix/svn-download.scm | 84 +++++++++++------
guix/swh.scm | 71 ++++++++------
tests/lint.scm | 20 ++++
tests/swh.scm | 74 +++++++++++++++
15 files changed, 501 insertions(+), 246 deletions(-)
base-commit: ffcce77ec488e3c89401ad77fafa65fcd9e9f5be
--
2.41.0
--- End Message ---
--- Begin Message ---
|
Subject: |
Re: [bug#69328] [PATCH v2 00/12] Better source code recovery from SWH |
|
Date: |
Sat, 09 Mar 2024 19:51:19 +0100 |
|
User-agent: |
Gnus/5.13 (Gnus v5.13) |
Hello,
Simon Tournier <zimon.toutoune@gmail.com> skribis:
> On mar., 05 mars 2024 at 12:06, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> Ludovic Courtès (12):
>> lint: Switch to SRFI-71.
>> lint: archival: Fix crash in non-Git case.
>> lint: archival: Trigger “Save Code Now” for VCSes other than Git.
>> swh: Add ‘type’ field to <visit>.
>> swh: ‘origin-visits’ takes an optional ‘max’ parameter.
>> swh: ‘lookup-origin-revision’ handles branches pointing to
>> directories.
>> hg-download: Use ‘swh-download-directory-by-nar-hash’.
>> svn-download: Use ‘swh-download-directory-by-nar-hash’.
>> bzr-download: Implement nar fallback.
>> download-nar: Distinguish ‘output’ and ‘item’ parameter.
>> perform-download: Allow use of ‘download-nar’ for ‘--check’ builds.
>> download: Honor ‘GUIX_DOWNLOAD_METHODS’ environment variable.
>
> LGTM.
Pushed as 2f441fc738976175d438f7942211b1894e2eb416, thank you & Timothy
for taking a look!
I’ll update the ‘guix’ package in the coming days so we can benefit from
all of this.
> Unrelated things for later. :-)
>
> 1. About CVS, IIRC, there is only one package: gnu-standards. And it
> changes barely. Why not fetch from FTP or else instead of CVS?
Good idea (or maybe someday someone will finally migrate it to some
other VCS?).
> 2. About the lookup, currently it is done item per item when it could be
> done several at once – Timothy does that with PoG. This helps for the
> rate limit. For instance if one uses “guix lint -c archival -m
> manifest.scm”.
>
> 3. The option ’-m’ for “guix lint” seems missing.
>
> These #2 and #3 would help third-party channels, IMHO.
All good ideas.
> Although, I am slowly working on some “guix swh” extension… but I have
> been distraction by another extension “guix try-out”, then distracted by
> another one “guix cite”. Well, I need to finish all my homeworks. ;-)
Heh, sounds exciting!
Ludo’.
--- End Message ---