bug#62720: 29.0.60; Not easy at all to upgrade :core packages like Eglot

[Eli Zaretskii]

> Date: Thu, 20 Apr 2023 01:06:10 +0300
> Cc: Eli Zaretskii <eliz@gnu.org>, arne_bab@web.de, jporterbugs@gmail.com,
>  emacs-devel <emacs-devel@gnu.org>
> From: Dmitry Gutov <dmitry@gutov.dev>
> 
> OK then, I think have to re-evaluate my position on this. Previously, I 
> guess, I made some hasty conclusions from how the discussion went on 
> without refreshing the exact details about package.el and use-package 
> (the latter I never knew to begin with). Apologies.
> 
> Eli, let me know if we should take this back to the bug tracker instead.

I've moved this back to the bug-tracker.  Please post all further
replies about this particular issue, i.e. updating of built-in
packages with package.el, to this bug and not to emacs-devel.

> So I would suggest to focus on functions that don't work as intended. 
> Namely:
> 
> - Add a user option for the list of builtin packages which would be 
> upgraded automatically by 'package-menu-mark-upgrades' and 'M-x 
> package-upgrade-all' (nee package-update-all). Maybe make it nil by 
> default, or maybe add 'eglot' to it. I don't have a strong opinion.
> 
> - Fix 'M-x package-upgrade' (nee package-update) to suggest Eglot as one 
> of the options and actually perform the upgrade. That shouldn't require 
> changes to 'package-install' because, as we already know, the user can 
> already install a newer version of Eglot using the 'list-packages' menu 
> (and picking the exact version manually). That execution path is going 
> to go through 'package-install' as well, so it must be suitable already.
> 
> - Revert 580d8278c5f48 because it creates odd semantics (upgrading 
> certain packages that are already installed but not others) and it 
> doesn't solve the issue with (use-package 'eglot :ensure t) anyway. We 
> could keep it, but seems like a half-measure that didn't make anyone 
> happy anyway. OTOH, it could minimize the rewrites of CI scripts.

I don't think we are ready to make any new decisions in this matter.
I think we don't even have a comprehensive and detailed picture of the
problems with updating/upgrading built-in packages in Emacs 29.
People are still discovering facts and subtleties of various
package.el commands and features, and are still arguing what exactly
happens in this or that scenario.

So before we discuss solutions, we need a full and detailed
description of the problems to solve.  If someone can do the footwork
of collecting this information and posting it here, please do, and
TIA.

I will say up front that, given what I already read here and in the
related thread on emacs-devel, there seem to be too many
inconsistencies and dark corners in this, in particular when built-in
packages are involved.  We will probably be unable to solve them in
time for Emacs 29.1, certainly not all of them.  So don't raise your
expectations too high.