[bug#44178] Add a Go Module Importer

[Ludovic Courtès]

Hi Katherine,

Katherine Cox-Buday <cox.katherine.e@gmail.com> skribis:

>>From cc92cbcf5ae89891f478f319e955419800bdfcf9 Mon Sep 17 00:00:00 2001
> From: Katherine Cox-Buday <cox.katherine.e@gmail.com>
> Date: Thu, 22 Oct 2020 19:40:17 -0500
> Subject: [PATCH] * guix/import/go.scm: Created Go Importer *
>  guix/scripts/import.scm: Created Go Importer Subcommand * guix/import/go.scm
>  (importers): Added Go Importer Subcommand

Nice!  I think that can make a lot of people happy.  :-)

Here’s a quick review.  I won’t promise I can reply to followups in the
coming days because with the release preparation going on, I’d rather
focus on that.  So perhaps this patch will have to wait until after this
release, but certainly before the next one!

> +(define (escape-capital-letters s)
> +  "To avoid ambiguity when serving from case-insensitive file systems, the
> +$module and $version elements are case-encoded by replacing every uppercase
> +letter with an exclamation mark followed by the corresponding lower-case
> +letter."
> +  (let ((escaped-string (string)))
> +    (string-for-each-index
> +     (lambda (i)
> +       (let ((c (string-ref s i)))
> +         (set! escaped-string
> +           (string-concatenate
> +            (list escaped-string
> +                  (if (char-upper-case? c) "!" "")
> +                  (string (char-downcase c)))))))
> +     s)
> +    escaped-string))

As a general comment, the coding style in Guix is functional “by
default” (info "(guix) Coding Style").  That means we almost never use
‘set!’ and procedures that modify their arguments.

We also avoid idioms like car/cdr and ‘do’, which are more commonly used
in other Lisps, as you know very well.  ;-)

In the case above, I’d probably use ‘string-fold’.  The resulting code
should be easier to reason about and likely more efficient.

> +(define (fetch-latest-version goproxy-url module-path)
> +  "Fetches the version number of the latest version for MODULE-PATH from the
> +given GOPROXY-URL server."
> +  (assoc-ref
> +   (json-fetch (format #f "~a/~a/@latest" goproxy-url
> +                       (escape-capital-letters module-path)))
> +   "Version"))

I’d suggest using ‘define-json-mapping’ from (json) like in the other
importers.

> +(define (infer-module-root module-path)
> +  "Go modules can be defined at any level of a repository's tree, but 
> querying
> +for the meta tag usually can only be done at the webpage at the root of the
> +repository. Therefore, it is sometimes necessary to try and derive a module's
> +root path from its path. For a set of well-known forges, the pattern of what
> +consists of a module's root page is known before hand."
> +  ;; See the following URL for the official Go equivalent:
> +  ;; 
> https://github.com/golang/go/blob/846dce9d05f19a1f53465e62a304dea21b99f910/src/cmd/go/internal/vcs/vcs.go#L1026-L1087
> +  (define-record-type <scs>
> +    (make-scs url-prefix root-regex type)
> +    scs?
> +    (url-prefix      scs-url-prefix)
> +    (root-regex scs-root-regex)
> +    (type    scs-type))

Maybe VCS as “version control system”?  (It took me a while to guess
what “SCS” meant.)

> +(define (fetch-module-meta-data module-path)
> +  "Fetches module meta-data from a module's landing page. This is necessary
> +because goproxy servers don't currently provide all the information needed to
> +build a package."
> +  (let* ((port (http-fetch (string->uri (format #f "https://~a?go-get=1"; 
> module-path))))
> +         (module-metadata #f)
> +         (meta-tag-prefix "<meta name=\"go-import\" content=\"")
> +         (meta-tag-prefix-length (string-length meta-tag-prefix)))
> +    (do ((line (read-line port) (read-line port)))
> +        ((or (eof-object? line)
> +             module-metadata))
> +      (let ((meta-tag-index (string-contains line meta-tag-prefix)))
> +        (when meta-tag-index
> +          (let* ((start (+ meta-tag-index meta-tag-prefix-length))
> +                 (end (string-index line #\" start)))
> +            (set! module-metadata
> +              (string-split (substring/shared line start end) #\space))))))

I’d suggest a named ‘let’ or ‘fold’ here.

Likewise, instead of concatenating XML strings (which could lead to
malformed XML), I recommend using SXML: you would create an sexp like

  (meta (@ (name "go-import") (content …)))

and at the end pass it to ‘sxml->sxml’ (info "(guile) Reading and
Writing XML").

> +    (else
> +     (raise-exception (format #f "unsupported scs type: ~a" scs-type)))))

‘raise-exception’ takes an error condition.  In this case, we should use
(srfi srfi-34) for ‘raise’ write something like:

  (raise (condition (formatted-message (G_ "…" …))))

> +(define* (go-module->guix-package module-path #:key (goproxy-url 
> "https://proxy.golang.org";))
> +  (call-with-temporary-output-file
> +   (lambda (temp port)
> +     (let* ((latest-version (fetch-latest-version goproxy-url module-path))
> +            (go.mod-path (fetch-go.mod goproxy-url module-path latest-version
> +                                       temp))

It seems that ‘go.mod-path’ isn’t used, and thus ‘fetch-go.mod’ &
co. aren’t used either, or am I overlooking something?

> +            (dependencies (map car (parse-go.mod temp)))

Please use ‘match’ instead, or perhaps define a record type for the
abstraction at hand.

> +            (guix-name (to-guix-package-name module-path))
> +            (root-module-path (infer-module-root module-path))
> +            ;; SCS type and URL are not included in goproxy information. For
> +            ;; this we need to fetch it from the official module page.
> +            (meta-data (fetch-module-meta-data root-module-path))
> +            (scs-type (module-meta-data-scs meta-data))
> +            (scs-repo-url (module-meta-data-repo-url meta-data goproxy-url)))
> +       (values
> +        `(package
> +           (name ,guix-name)
> +           ;; Elide the "v" prefix Go uses
> +           (version ,(string-trim latest-version #\v))
> +           (source
> +            ,(source-uri scs-type scs-repo-url temp))
> +           (build-system go-build-system)
> +           ,@(maybe-inputs (map to-guix-package-name dependencies))
> +           ;; TODO(katco): It would be nice to make an effort to fetch this
> +           ;; from known forges, e.g. GitHub
> +           (home-page ,(format #f "https://~a"; root-module-path))
> +           (synopsis "A Go package")
> +           (description ,(format #f "~a is a Go package." guix-name))

Maybe something like “fill it out!” so we don’t get patch submissions
with the default synopsis/description.  :-)

> +           (license #f))

Likewise.

Two more things: could you (1) and an entry under “Invoking guix import”
in doc/guix.texi, and (2) add tests, taking inspiration from the
existing importer tests?

Thank you!

Ludo’.