bug#62720: 29.0.60; Not easy at all to upgrade :core packages like Eglot

[Dmitry Gutov]

On 22/04/2023 11:26, Eli Zaretskii wrote:

> > The above scenarios (package-install or (use-package eglot :ensure t))
> > when ~/.emacs.d/elpa is empty, result in the very latest Eglot being
> > installed when using Emacs 28. And will result in something different
> > with Emacs 29 (not the latest version). That's not nothing: the CI
> > scripts will have to be updated, and the user instructions for reporting
> > problems will have to be made more complicated as well. Some
> > possibilities will simply be gone (the user won't be able to upgrade
> > Eglot to the very latest by deleting it from ~/.emacs.d/elpa and calling
> > package-install, for example). This *is* a non-backward compatible
> > change from the perspective of Eglot's maintainer.
>
> We were talking about users installing and updating packages.  The CI
> scenario doesn't belong here.  It is also much less important one
> (test suites are always required to chase the changes in development).
>
> Let's not complicate an already complicated set of issues by bringing
> up unimportant secondary use cases.

You asked. I relayed what's already been said on the matter by Joao.

> > I, personally, don't really buy this kind of argument, but I figured you
> > might. After all, it's rather in line with reasoning we've seen voiced
> > around these parts many times ("X has worked this way for Y years, let's
> > never change it from now on"). Classic https://xkcd.com/1172/.
>
> If you allude to my reasoning, then it is never that simplistic, and
> always considers each case separately, not "by analogy".

Of course that's a simplification. Hence the "might" anyway.

> > Either way would be fine, IMO, as long as the behavior is logical and
> > matches documentation. But having a separate command to upgrade now lets
> > us fix it separately without worry of breaking something more globally.
>
> Except that, based on what we have (see below) ,we don't really have
> an "upgrade" operation, we only have "install" and "delete"
> (i.e. "uninstall").  So maybe we should preserve that, to minimize
> problems and user surprise/confusion.

We do. We have commands for upgrading, both in "list-packages", and used 
interactively. Which do the thing of installing the new version and 
removing the old one.

Which is what upgrading means in various tools, e.g. 'apt'.

> > > In interactive invocation, package-upgrade calls completing-read with
> > > its 4th argument non-nil, so you cannot select a package which is not
> > > in the collection returned by package--updateable-packages.  What I
> > > meant above is to allow that collection to include built-in packages
> > > as optional behavior.  I just tried invoking package-update for ElDoc,
> > > and I get "No match" after typing "eldoc" to its prompt, although
> > > eldoc version 1.14.0 is in the list presented by list-packages as
> > > "available".
> >
> > That's what I imagined: adding a new optional argument to
> > package--updateable-packages which would include builtins in the result.
> >
> > And only pass it when called from package-upgrade.
> >
> > Hopefully that's the kind of optional that you meant.
>
> Yes, something like that.  Presumably activated by the same new option
> introduced for package-installed.

Okay, then it's something different, and you didn't answer my question.

> > > > > > 'package-update' is an
> > > > > > interactive function which itself it called from only one place:
> > > > > > 'package-update-all', and since the plan is to improve both, we can make
> > > > > > sure they only do what we ask of them: package-update will upgrade
> > > > > > builtins when invoked directly, and package-update-all will upgrade them
> > > > > > only when the builtin has been upgraded before (making it not a builtin
> > > > > > anymore), or a new user option is set.
> > > > >
> > > > > This is one possibility, and it might make sense to some users.  But I
> > > > > don't think we can be sure it will make sense to an overwhelming
> > > > > majority of the users.
> > > >
> > > > Hence the user option?
> > >
> > > Which one?  Are you suggesting to add a new one?  If so, why not use
> > > the one we already added, package-install-upgrade-built-in?
> >
> > The user option I was thinking of would probably be called a little
> > shorter: package-upgrade-built-in. And it would only affect the upgrader
> > commands.
>
> We could rename the existing option, if the name is the problem.
> Otherwise, I don't see why we would need two separate options: they do
> the same job and have the same meaning from the user's POV.

The name is a problem, yes. What could also be a problem is a user that 
customizes this option to have package-update update builtin packages (a 
reasonable behavior that should be on by default anyway), will also 
automatically have change the behavior of package-install to be more 
surprising (install an already installed package).

Further, if we have a user option affect package-update, we'll have to 
alter package-update-all and package-manu-mark-for-update in the same 
patch (otherwise we'll have more nonsense on our hands). Whereas the 
first version I sent is more minimal.

> > All this is to say, the first step (upgrading Eglot to the version from
> > ELPA) will be less user-friendly compared to the other UIs we have. But
> > it's probably manageable, especially if documented well.
>
> I don't see why it would be less user-friendly.

The same reason we do have commands with "upgrade" in their names, 
rather than force everybody to use the "install" and "delete" ones.

> > My point here, however, is that commit 580d8278c5f48 improved the
> > situation to a lesser degree that some people here might have expected.
>
> One again: commit 580d8278c5f48 solved precisely the problem which
> opened this bug report, nothing more, nothing less.

It doesn't seem like the originator of the report agrees with that.

> > But we can conclude that we do have two different installation actions:
> >
> >    - package-install which will install the latest version of a package,
> > but only if it's not installed;
> >    - package-menu-mark-install, which will mark a specific version for
> > installation, disregarding whether some earlier version is already
> > installed; the previous version will remain installed still.
>
> Which is again a breaking behavior change, AFAIU.  Is this a good idea
> so late in the development of Emacs 29?

The above is not a breaking change, it's how things work already. And 
have been working for quite a while.

> > But even if we decide that we want to eliminate that split, doing *that*
> > would really be a breaking change. I don't have a reasonable plan to
> > present for doing that in Emacs 29, so far.
>
> There's no "split".  What I wanted to point out is that we don't seem
> to have a clear vision of these two commands, since they are confusing
> intertwined.  In fact, one could argue that package-upgrade in its
> current form is simply a convenience shortcut for "delete old and
> install new".

What should an upgrade command do, in your opinion, if not "delete old 
and install new"?