bug#69657: Missing imenu entries with eglot

[Sebastian Poeplau]

> The problems are in Imenu.
>
> You should rather lobby for a better imenu structure (which will be
> hard to  do fully  backward compatibly will all existing UIs -- except if
> you use the string properties trick that Eglot/breadcrumb use as
> I explained)  You could also lobby for a brand new imenu replacement
> in Emacs.

I've been thinking about this. A "brand new imenu replacement" sounds
like something that would be hard to get adoption for. So, assuming that
we wanted to change the imenu structure to something that works better
for representing ASTs, I think there are two somewhat related problems:
(1) how to represent the AST internally, and (2) how to show it to the
user.

1. The current representation in `imenu--index-alist', as you said, has
   the problem that it doesn't allow position information on inner
   nodes; also, additional metadata like the category has to be tagged
   on with text properties or other tricks.

   One could extend the existing structure. For example, leaves could
   become lists (name beg end category) with everything but NAME and BEG
   being optional, or something like that; inner nodes could optionally
   have the same format instead of being just strings. I don't see how
   to much such a change in a way that wouldn't break existing UIs, but
   maybe one could find a way that at least makes it easy for them to
   support the new structure (e.g., by providing helper functions in
   imenu.el, which could also facilitate future changes in
   `imenu--index-alist').

   I guess while changing the structure one could try to optimize it for
   the use cases of existing UIs. Breadcrumb, for example, probably
   needs an efficient means of computing the most specific node
   corresponding to a buffer position, as well as the path down from the
   root to that node, and the list of its sibling nodes.

2. A menu structure like the one offered by M-x imenu or
   `imenu-add-menubar-index' makes it difficult to jump to inner nodes.
   The best I can think of is an entry labeled "(top)" or something like
   that underneath each leave node with a position; in my earlier
   example, the "Foo" menu entry would then expand to a list of two
   sub-entries, "(top)" and "bar". Alternative UIs probably have much
   better ways to display the information (such as the entry in
   consult-imenu that I was originally looking for).

Even if this sounds like a nice improvement, I wonder about the
compatibility issue. Do you think such a patch would even be considered?