Re: Request to add Package to GNU ELPA

[Philip Kaludercic]

Jonas Bernoulli <jonas@bernoul.li> writes:

> Hello,

Hi, hope I can add a few useful comments here.

> I would like to request that "package.el" be added to GNU Elpa as a
> "core" package.  This would make new features available to users who are
> stuck on older Emacs releases and it would even bring us one step closer
> to being able to make backward incompatible changes in the future.
>
> Some examples for why that would be desirable:
>
> - A recent addition that could be useful is the new "package-vc.el"
>   (which I think should be distributed as part of the `package' package,
>   but it could also be distributed as a separate core package).

This will be difficult, since package-vc depends on the new "vc-clone"
function, which would also have to be provided on ELPA to work.  This
seems to me to be a general issue with proposing to create new core
packages.  When working in emacs.git it is easy and usually
uncontroversial to make use of whatever is the current state-of-the-art
when it comes to Elisp.  The Compat library on ELPA can help here, as it
already does in a few cases (ERC and Python, and it has been discussed
for project.el).

All of this touches on the general topic of how Emacs should be
developed: Small and to be extended, therefore of little use OOTB or big
and feature-full, but slow to be updated.  IIUC a compromise could be
achieved by having two versions of Emacs that would respectively attempt
to satisfy the two aspirations.  There was some work in that direction a
few years ago[0], but I don't know if anything has happened since.
Perhaps pushing for this more general solution would help solve this
particular problem?

[0] https://git.savannah.gnu.org/cgit/emacs.git/log/?h=elparized-core

>   That isn't just useful for users who like to live on the edge.  For
>   example, if a package drops support for an old Emacs release, then
>   a user who is stuck on that release, may wish to keep using the last
>   release of the package that still supports that Emacs release, and
>   package-vc would allow them to do just that.
>
> - I don't use Package myself, so I am not fully aware of all the quality
>   of live enhancements that surely have accumulated over the years.  But
>   I am aware of some small missing features, that would be beneficial to
>   users of older Emacs releases.
>
> - For example, I think it would be nice if the columns displayed by
>   `list-packages' were customizable.  A user might want to increase the
>   width of the "Version" column, so that not every GNU-devel ELPA version
>   is truncated, 

A more general solution to this problem would be if tabulated-list-mode
could dynamically resize a column, or at least shrink it possible.

Also on another topic, I am still uncertain as to the status of the
-devel archives.  I see them primarily as a means for developers to
check the status of a package pre-release or to make sure that the ELPA
build-system is working the way they want it to (e.g. when building
manuals, news or README files).  They are not advertised anywhere, and I
don't think they are made for general consumption -- or rather I don't
think they should be, especially with package-vc there is a much better
way to track development and make it possible to contribute changes
upstream.

>                 or they might want to add an "Author" column.  We could
>   even allow third-party packages to add columns such as "Downloads" or
>   "Stars".

> - I should also mention that my main motivation for pushing for this
>   now, is that I would like to improve version handling.  That is a
>   whole other can of worms, so I do not wish to discuss it just yet,
>   to avoid distracting from the topic at hand, but I thought I should
>   at least mention it.  You might very well end up being against my
>   suggestions regarding versions strings, once I present them, but that
>   should not be cause to oppose the change requested here as well.  Even
>   if my suggestions regarding version strings are ultimately rejected, I
>   still think it would be a good idea to add `package' to GNU ELPA, for
>   the other reasons presented here.

This implies that packages might themselves depend on newer versions of
package.el -- which I think one should put some thought into before
anything is done.  How can you e.g. change the way versions are handled
in a way that people with older versions of package.el could still
handle the change without confusion?  (This will be an issue for at
least a decade to come, knowing the pace at which users get access to
newer versions of Emacs).  You mentioned updating the ELPA protocol a
few months back, and I believe a few points of interest were collected
in that thread.  It seems to me these issues are all intertwined.

> - I should also make an example for a backward incompatible change.
>   Yesterday, in https://debbugs.gnu.org/cgi/bugreport.cgi?bug=62524 I
>   suggested a backward *compatible* change, but doing it in a backward
>   compatible fashion is a bit ugly, and I would like to *eventually*
>   replace it with a backward *incompatible* change.
>
>   In short, some packages have multiple maintainers and
>   `describe-package' should list them all.  Unfortunately `:maintainer'
>   (unlike `:authors') in "archive-contents" can only be a single
>   maintainer.  If we listed multiple maintainers, then current versions
>   of Package would signal an error.  To address that I proposed that we
>   *add* a `:maintainers' property.  Older Package don't know about that
>   property and would simply ignore it, and newer Package versions could
>   start doing all the maintainers justice.
>
>   It isn't exactly horrible to have both `:maintainer' and
>   `:maintainers' for a while, but it is a wart that should be remove
>   eventually.  We should avoid accumulating more and more such warts
>   over time.  One way of doing that is to simply avoid adding features
>   that have to be implemented in an ugly fashion to avoid breaking
>   backward compatibility.  I feel that approach has been taken too many
>   times in the past.  I myself have certainly done it on several
>   occasions.
>
> I mentioned that distributing Package on GNU ELPA would bring us one
> step closer to being able to make backward incompatible changes, but
> unfortunately it is not enough.  Just because `package' is available
> on GNU ELPA, that doesn't mean that it gets installed automatically.
>
> There are two additional steps required.  (1) We have to somehow get
> users to install `package' from GNU ELPA a first time, and (2) once
> users have done that and a new `package' version becomes available,
> they have to be informed about the update and be prompted to install
> it.
>
> To address (2), `package-refresh-contents' could be taught to check if
> a new `package' version is available, and if that is the case, then it
> should prompt the user to update that immediately, before any other
> packages.

With Emacs 29, `package-refresh-contents' now calls a Hook called
`package-refresh-contents-hook'.  This was initially introduced so that
package-vc can download the "elpa-packages.eld" file, but it could also
be of use here as well -- but it wouldn't help anyone with Emacs 28 or
older...  I could imagine a more hacky approach based on carefully
placed advice, but only as a last resort.

> Addressing (1) is harder, and I don't think it possible to do so in a
> way that guarantees that not a single user would end up seeing an error
> due to an incompatible change.  On the other hand "archive-contents"
> comes with a version field, and if we bump that, we at least get a
> meaningful warning: "Package archive version 2 is higher than 1".  This
> doesn't say that the solution is to install `package' from GNU ELPA, but
> it should be enough to feed into a search engine to get to that answer.

The upgrading procedure for archive-contents has never appeared to me to
be very robust.  Perhaps it would be better to introduce a second file
(archive-contents.eld) with a more flexible format?

> Additionally, we could get many users to install `package' from GNU
> ELPA, by making a few popular packages explicitly depend on a `package'
> version that is available from GNU ELPA for a few months.

Again, a few months wouldn't be enough to solve the issue.  But again as
well, Compat could help by adding package as a noop dependency.

>      Thanks for considering this proposal,
>      Jonas

-- 
Philip Kaludercic