bug#69785: 29.2; Image mode can loop due to invalid images

[Eli Zaretskii]

> Date: Wed, 13 Mar 2024 22:13:55 +0100
> From:  Rahguzar via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> 
> Dear Emacs developers,
> 
> Starting with emacs -Q, evaluate
> 
> (with-current-buffer (get-buffer-create "*image-mode-test*")
>   (insert (propertize "abc" 'display (list 'image "Definitely not an 
> image.")))
>   (image-mode))
> 
> twice.
> 
> The second time results in an error and pop ups the debugger, with the
> following text,
> 
> Debugger entered--Lisp error: (excessive-lisp-nesting 1601)
>   display-images-p()
>   image-mode()
>   major-mode-restore((image-mode image-mode-as-text))
>   image-mode-to-text()
>   image-mode-as-text()
>   image-mode--display()
>   image-mode()
>   major-mode-restore((image-mode image-mode-as-text))
>   image-mode-to-text()
>   image-mode-as-text()
>   image-mode--display()
>   image-mode()
>   major-mode-restore((image-mode image-mode-as-text))
>   image-mode-to-text()
>   image-mode-as-text()
>   image-mode--display()
>   image-mode()
>   ...
> 
> It is unclear to me why this doesn't happen the first time but I think
> sequence causing it the second time is as follows:
> 
> 1. 'image-mode' calls 'major-mode-suspend' when 'major' mode is
> 'fundamental-mode'. So 'major-mode--suspended' is not set.
> 
> 2. 'image-mode--display' catches error and call 'image-mode-as-text'
> which calls 'image-mode-to-text'.
> 
> 3. 'image-mode-as-text' calls 'major-mode-restore' with '(image-mode
> image-mode-as-text) as mode to avoid.
> 
> 4. Since major-mode--suspended was not set in step 1
> 'major-mode-restore' falls back to 'normal-mode' which calls
> 'image-mode' causing the loop.
> 
> I think the simplest way of fixing this is to make sure that
> 'image-mode' doesn't call 'major-mode-suspend' with 'fundamental-mode'
> instead calling 'text-mode' or 'special-mode' before calling
> 'major-mode-suspend' if it starts in 'fundamental-mode'.

How about the patch below instead?

diff --git a/lisp/image-mode.el b/lisp/image-mode.el
index 355685e..fa64f1a 100644
--- a/lisp/image-mode.el
+++ b/lisp/image-mode.el
@@ -654,8 +654,9 @@ image-mode
   (unless (display-images-p)
     (error "Display does not support images"))
 
-  (major-mode-suspend)
-  (setq major-mode 'image-mode)
+  (unless (eq major-mode 'image-mode)
+    (major-mode-suspend)
+    (setq major-mode 'image-mode))
   (setq image-transform-resize image-auto-resize)
 
   ;; Bail out early if we have no image data.