Re: Core ELPA was: Testing fontification, indentation, and buffer manipulation

[Stefan Monnier]

>>> I'm unconvinced. I think a build should be repeatable. Imagine bisecting
>>> a bug that has come up in an ELPA package when you are not sure whether
>>> it's the package or Emacs core that has broken things.
>>
>> That's already a problem with Elisp packages (whether from GNU ELPA or
>> elsewhere), so the fact that we bundle some of them doesn't make much of
>> a difference in this respect.
>
> It is, but, now that ELPA package will be in the tarball download; I
> think when someone says "bug X, affecting package Y, in Emacs-26.Z" that
> should give you enough information to try and reproduce the error.

For the actual release, I'd expect the branch/tag to be very precise,
indeed (maybe even an SHA at that point).

> Without a repeatable build, you will also need to know what verison
> package Y is, if and only if it is a "core ELPA" package.

We could also consider such bug reports exactly like we do now: treat
the bundled GNU ELPA packages as if they weren't bundled, and ask the
user what is the version of the GNU ELPA package affected.

>>> That might mean multiple branches of master which would produce a very
>>> cluttered namespace. The problem is that ELPA currently uses a different
>>> (non git) mechanism to identify the current version of every package; so
>>> you can't identify this from git metadata (except for SHA!).
>> I don't know what you mean.  A branch/tag is just a name for an SHA, so
>> I can't see why a branch/tag wouldn't work where an SHA does.
> This is how a tag is implemented not what it is. A tag is a meaningful
> name, living in a single namespace for an entire repository, which can
> refer to different versions over time. A SHA key is not meaningful,
> which while it lives in a single namespace has been written to attempt
> to be world-unique, and cannot refer to different versions over time.

I understand this difference but I don't see how it relates to what you
said above.

> I have extensive experience with naming schemes and "simply" is rarely a
> term I would apply; especially, in this case, where many packages
> inhabit a single namespace. This rules out the obvious scheme of just
> using the Emacs version number.
> My naming scheme would be to use a stable, meaningless identifier.

I was thinking of something like "<scopingprefix>/<pkgname>/<emacsbranch>"

E.g. bundled/company/emacs-27

[ I had suggested that in addition to `emacs-27` we also create a branch
  `emacs-27.1` (created when we get into the final phase where we only
  allow commits that are explicitly allowed by the maintainer), so we
  could have bundled/company/emacs-27.1 as the branch pointing to the
  "final" version of company bundled with the Emacs-27.1 release.  ]

In order to reduce the number of such branches, we'd likely want some
fallback scheme where we have "<scopingprefix>/<emacsbranch>"
when "<scopingprefix>/<pkgname>/<emacsbranch>" doesn't exist, and
finally use just `master` when "<scopingprefix>/<emacsbranch>" doesn't
exist either.

> Regardless, the code works either way, because as you say an tag is
> implemented as an SHA.

Indeed.  All this discussion is really about not needing to pull from
the repository as part of a normal `make`, but moving this operation to
a separate invocation explicitly requesting it (call it `make
update-elpa` or something).

> Currently, to achieve this on a clone of Emacs, you'd have to
> reconfigure. With "./configure", my build would not install ELPA
> packages and would (when I've written the code!) only produce a single
> source tarball. With "./configure --enable-elpa", it would install ELPA
> and produce two source tarballs, but would fail to build without ELPA.
>
> Happy to put another mechanism in, but not sure what that would be.

Not sure what it should look like at the code level, but I think we
could live without "./configure --enable-elpa".  Instead, we'd have:

    git clone; ./configure; make

build Emacs normally but including warnings about GNU ELPA packages not
being found.

    git clone; ./configure; make update-elpa; make

build Emacs with the all the bundled GNU ELPA packages.  And subsequent

    git pull; ./configure; make

will build with those same bundled GNU ELPA packages, potentially
emitting some warnings about *some* GNU ELPA packages not being found.

>> Maybe in the future, we'll want to allow some bundled-GNU-ELPA packages
>> to be more important (e.g. be *necessary* for example because they're
>> used by some core Elisp packages), but there's no need to cross that
>> bridge now.
> Ironically, the package that stimulated this discussion was
> "assess.el" which is a testing package. From a user perspective, of
> course, it makes no difference but for the developer it's exactly the
> sort of package that you would want to make a dependency to many other
> packages. Likewise, seq.el (which is why Nicolas wants to move it from
> ELPA to core!).

I know: I don't mean to rule out having GNU ELPA packages that are
indispensable, but I think we should take it one step at a time,
otherwise the whole thing might just be rejected.

FWIW, seq.el *is* (and always has been, IIRC) in core.

As for assess.el, having it in GNU ELPA is OK as long as it is not
needed when *building* Emacs's core packages.  Basically, it should be
OK if we can do (like we do with ERT): `make lisp; make elpa; make
tests` (tho we'll probably have to work extra to try and make sure that
`make tests` nicely skips those tests that require assess.el when
assess.el is missing).


        Stefan