Re: packaging a golang package

[Katherine Cox-Buday]

Hello again, François! I've redirected this thread to guix-devel, and
the bug since we've begun discussing implementation details.

JOULAUD François <Francois.JOULAUD@radiofrance.com> writes:

> First is to use vendored dependencies (when upstream provides them). This
> one has the merits of simplicity (as we just have to download the
> source and launch `go build` or whatever is needed) and less risks of
> bugs. The downsides are known: difficult to audit Licences, duplication
> of code source, difficulty to follow security bugs, etc.

-1 to this approach (explanation below).

> Second is to re-vendor package from go.mod (coming directly from code
> source or synthetized from source) by creating a source derivation
> including all dependencies. This is the strategy followed by Nixpkgs
> as explained in [1]. (It seems to me this is the route followed in
> the patches to  from Helio in [2] but I did not take the time to read
> them.) With this approach we still have some of the downsides of using
> vendored upstream but it is at least easier to verify the existence
> of upstream.

I don't fully understand this approach, so I don't understand how this
is different to approach three? Does this mean we create a pseudo,
non-public package which states all the dependencies as inputs? If so,
why wouldn't we just go with option three?

> Third is to package every dependencies. This is the most transparent way
> and the one that leads to less code duplication but the downside is the
> difficulty to scale with the number of packages (even if the objective of
> patch at [3] is to automate it) and the need to have only one reference
> version for each package in the distribution which have its own share of
> problems (one of them being that we don't use those tested by upstream
> as stated in [4]).

I think this is the eternal conflict between distributions and
code-bases. As a software engineer, I am a fan of vendoring. It removes
entire classes of issues:

- Dependency on remote servers to be up to fetch dependencies and
  perform builds
- Requiring some kind of system to pin versions
- Needing to stay on top of releases even if that doesn't meet your
  schedule or needs

As a packager for a distribution, I dislike vendoring because of the
reasons you outlined above, _but_ I also dislike building upstream
software with versions of dependencies that weren't approved, tested,
and verified, upstream. It seems to me like that's a recipe for
unstable, maybe even insecure, software.

Normally, distributions are forced to do this because their packaging
and build systems are only set up to support one version at a time. Guix
can theoretically support all versions, but doesn't want to take on the
dependency graph explosion this approach would cause.

If we go on historical precedent, and how Guix handles the other
language ecosystems, we should only have one version of each dependency,
and build all software which relies on that dependency on the version we
have packaged. Guix has already begun taking on the difficult challenges
with this approach, including patching upstream to work with versions of
dependencies upstream was not built for.

I dislike it, but I also don't think we should try to solve the broader
class of issues while trying to implement an importer. It should be a
larger discussion within the Guix community across _all_ language
packages about how we might achieve upstream parity while still
maintaining our goals as a distribution, all while not crippling our
infrastructure and people :)

That's my viewpoint, but I think we should all defer to some of the
longer-term maintainers who have helped guide Guix.

Thanks for writing up these options.

--
Katherine