bug#11035: 24.0.94; icomplete with multiline candidates and standalone minibuffer

[Drew Adams]

1. Download these two files from Emacs wiki:
http://www.emacswiki.org/emacs/download/hexrgb.el
http://www.emacswiki.org/emacs/download/oneonone.el
 
2. Start Emacs like this:
runemacs.exe -Q --debug-init -l "hexrgb.el" -l "oneonone.el" -f "1on1-emacs"
 
That gives you a standalone minibuffer frame that is two lines high.
 
3. Turn on icomplete-mode.
 
4. Evaluate this:
 
(completing-read
 "This is the prompt: "
 '(("aaaaa candidate\naaaaa second line\n")
   ("bbbbb candidate\nbbbbb second line\n")
   ("ccccc candidate\nccccc second line\n")
   ("ddddd candidate\nddddd second line\n")
   ("eeeee candidate\neeeee second line\n")
   ("fffff candidate\nfffff second line\n")
   ("ggggg candidate\nggggg second line\n")
   ("hhhhh candidate\nhhhhh second line\n")))
 
5. Type `b'.
 
Symptom: The minibuffer prompt and user input are moved up one line,
out of the frame.  The cursor appears at bol, just before this
icomplete overlay text:
 
(bbbb candidate
bbbbb second line
 
The rest of the overlay text is off the bottom of the frame (that's
normal, since it is only two lines high - but see below, near the end).
 
If you use the mouse to click the title bar of another frame at this
point, selecting it, you can see that the minibuffer now shows the first
line instead:
 
This is the prompt: b
 
with the cursor after the `b'.  Why this change in display, I don't know.
 
If you click the mouse on the minibuffer frame title bar, the display
returns to showing (bbbb candidate etc., with the cursor shown at bol.
 
(In my own setup, I see the same problem even without the final \n at
the end of each completion candidate, but the recipe above is the
simplest I can offer to repro the problem.)
 
This buggy behavior started with Emacs 23.  There is no such problem
with Emacs 22.3 or prior.  In Emacs 23, the icomplete.el code switched
from inserting text to using an overlay (which I agree should be a
better approach, in principle, though I don't know if that change was
just made gratuitously or was in response to some reported bug).
 
IOW, there is no problem in Emacs 22.3 or earlier.
 
In Emacs 22.3, the display appears like this:
 
This is the prompt: b(bbbb candidate
bbbbb second line
)  [Matched]
 
with the cursor on the left paren, after the first `b'.  That is normal
(and useful).  The user can see the prompt and the input, in addition to
the start of the first candidate (in fact all of the first candidate, in
this case).
 
The other problem I have with the Emacs 23+ icomplete code is the
following.  Although I recognize that using an overlay should make
sense, it messes up my code, which automatically increases the
minibufferframe height when the minibuffer content grows additional
lines.  I do that using fit-frame.el (my library), which measures the
buffer content to determine the needed frame height (respecting user-set
limits).
 
Since the overlay is not really buffer content, it doesn't get measured,
so the frame is not refit to accommodate the text of the overlay.  This
is why the last line shown for Emacs 22.3 above does not appear in the
frame in Emacs 23+ at all.
 
This is a drawback, IMO, with using an overlay for icomplete: the
overlay is not "in" the minibuffer, so operations on the minibuffer
content do not take it into account.  (That is of course also the
strength of using an overlay: it is not part of the minibuffer content.)
 
Perhaps Emacs Dev has a similar problem for icomplete wrt resizing the
minibuffer window when it is not standalone - dunno.  If someone has a
suggestion in this regard, I'm interested.
 
In sum, my code works fine in Emacs 20-22, but is broken in Emacs 23 and
later because of the changed implementation of icomplete.
 

In GNU Emacs 24.0.94.1 (i386-mingw-nt5.1.2600)
 of 2012-02-26 on MARVIN
Windowing system distributor `Microsoft Corp.', version 5.1.2600
Configured using:
 `configure --with-gcc (4.6) --no-opt --enable-checking --cflags
 -ID:/devel/emacs/libs/libXpm-3.5.8/include
 -ID:/devel/emacs/libs/libXpm-3.5.8/src
 -ID:/devel/emacs/libs/libpng-dev_1.4.3-1/include
 -ID:/devel/emacs/libs/zlib-dev_1.2.5-2/include
 -ID:/devel/emacs/libs/giflib-4.1.4-1/include
 -ID:/devel/emacs/libs/jpeg-6b-4/include
 -ID:/devel/emacs/libs/tiff-3.8.2-1/include
 -ID:/devel/emacs/libs/gnutls-3.0.9/include'