Re: Stability of core packages (was: Not easy at all to upgrade :core packages like Eglot)

[Dmitry Gutov]

On 19/04/2023 23:57, João Távora wrote:
> On Wed, Apr 19, 2023 at 9:50 PM Dmitry Gutov<dmitry@gutov.dev>  wrote:
>
> > So... are you sure that (use-package 'eglot :ensure t) upgrades/upgraded
> > the package?
> >
> >   From what I'm reading in use-package-ensure-elpa, all 'package-install'
> > calls are wrapped in
> >
> >     (unless (package-installed-p package)
> >       ...
> >
> > That seems to mean that, as long as the package is installed already
> > (whether by being bundled, or because of a previous manual
> > installation), it shouldn't get upgraded when this form is executed.
> That's true.  Exactly the same as package-install. It will not
> reinstall over another one.
OK then, I think have to re-evaluate my position on this. Previously, I 
guess, I made some hasty conclusions from how the discussion went on 
without refreshing the exact details about package.el and use-package 
(the latter I never knew to begin with). Apologies.
Eli, let me know if we should take this back to the bug tracker instead.

> But that doesn't change anything.  In Emacs 26/27/28 from scratch
> package-install and use-package_will_  rev up Eglot to whatever is the
> newest version.  In Emacs 29 it won't.
I think one of the conclusions to be made here is that even if 
(package-install 'eglot) now installs the newest version of Eglot in 
Emacs 29,
  (use-package 'eglot :ensure t)

still won't do that in Emacs 29 because (package-installed-p 'eglot) 
returns t still. So the commit 580d8278c5f48 doesn't help with your 
"most common upgrade method" cited below, if they rely on use-package 
instead of calling 'package-install' directly.
The patch I +1'd here https://debbugs.gnu.org/62720#467 wouldn't help 
with (use-package 'eglot :ensure t) either, IIUC.
But it would help in case the user put (package-install 'eglot) inside 
their init script, or if the CI script contains that. But the CI scripts 
could be fixed anyway (there must be a lot fewer of them out there than 
there are user installations).
Next: 'package-install' isn't supposed to install a new version when 
some version is already installed. The docstring says so, and I just 
tried this out: had older 'vertico' installed (with a newer one 
available in the repo), evaluated (package-install 'vertico), got back 
the message "already installed".
Do we want to change the semantics of 'package-install' just so the 
(minor) fraction of users who call this function directly will get Eglot 
upgraded when starting over from an empty config with Emacs 29? And/or 
change (package-installed-p 'eglot) to return nil when the appropriate 
user option is set? I'm not sure that's wise, certainly not the latter.
> In fact deleting Eglot and restarting Emacs the config run again is what
> I suppose the most common upgrade method (since there is no
> package-update in Emacs 28).
I've never considered this particular scenario, but as explained above, 
it's still not "fixed" by either of the proposed approaches. Not 
entirely, at least.
The most common upgrade method, though, is hopefully 
'package-menu-mark-upgrades'. Or:
  M-x list-packages
  U
  x

And that one also doesn't work for Eglot and other builtin packages. 
Unlike 'package-install', this behavior is not obviously correct, 
although it's "always been like this" so we're probably not going to 
change what it does by default at this point in time.
Another function which is supposed to upgrade packages in 'M-x 
package-update' (why aren't people commenting on bug#62750? the naming's 
gonna drive me crazy). This one is obviously incorrect because upgrades 
exist for Eglot, and yet it doesn't suggest any.
So I would suggest to focus on functions that don't work as intended. 
Namely:
- Add a user option for the list of builtin packages which would be 
upgraded automatically by 'package-menu-mark-upgrades' and 'M-x 
package-upgrade-all' (nee package-update-all). Maybe make it nil by 
default, or maybe add 'eglot' to it. I don't have a strong opinion.
- Fix 'M-x package-upgrade' (nee package-update) to suggest Eglot as one 
of the options and actually perform the upgrade. That shouldn't require 
changes to 'package-install' because, as we already know, the user can 
already install a newer version of Eglot using the 'list-packages' menu 
(and picking the exact version manually). That execution path is going 
to go through 'package-install' as well, so it must be suitable already.
- Revert 580d8278c5f48 because it creates odd semantics (upgrading 
certain packages that are already installed but not others) and it 
doesn't solve the issue with (use-package 'eglot :ensure t) anyway. We 
could keep it, but seems like a half-measure that didn't make anyone 
happy anyway. OTOH, it could minimize the rewrites of CI scripts.