bug#66218: 29.1.50; `beginning-of-defun' jumps to wrong position in `emacs-lisp-mode'

[Jens Schmidt]

Function `beginning-of-defun-raw' uses `syntax-pss' at [1] to
verify a potential BOF in this snippet:

------------------------- snip -------------------------
    (and (let (found)
           (while
               (and (setq found
                          (re-search-backward
                           (if defun-prompt-regexp
                               (concat (if open-paren-in-column-0-is-defun-start
                                           "^\\s(\\|" "")
                                       "\\(?:" defun-prompt-regexp "\\)\\s(")
                             "^\\s(")
                                              nil 'move arg))
                    (nth 8 (syntax-ppss))))
                           ^^^^^^^^^^^^^ [1]
           found)
         (progn (goto-char (1- (match-end 0)))
                               ^^^^^^^^^^^^^ [2]
                t)))
------------------------- snip -------------------------

However, if `syntax-pss' calls a `syntax-propertize-function', the latter
can overwrite match data, which is referenced at [2] above.

Here is an "emacs -Q" reproducer that shows the problem (CHANGES
REQUIRED):

------------------------- snip -------------------------
(progn
  ;; CHANGE FILE NAME AS NEEDED.  It is important that the file
  ;; is found first in this test case, or otherwise syntax
  ;; properties may already have been populated.
  (ignore-errors (kill-buffer "shr.el"))
  (with-current-buffer
      (find-file-noselect "~/work/emacs-29/lisp/net/shr.el")
    ;; CHANGE POSITION AS NEEDED, should be line 2044, at BOL
    ;; before the first `cond' in `shr-table-body'
    (goto-char 70719)
    (beginning-of-defun)
    ;; point should be at BOF of `shr-table-body', but is in fact
    ;; somewhere near a Unicode character name
    (pop-to-buffer (current-buffer))))
------------------------- snip -------------------------

Patch attached.  Please let me know if you think that ERTs or
anything else is requried.

In GNU Emacs 29.1.50 (build 4, x86_64-pc-linux-gnu, GTK+ Version
 3.24.24, cairo version 1.16.0) of 2023-09-25 built on sappc2
Repository revision: ca5b48fd76db2ab7c154c2db6a7d4a9fb7857f6c
Repository branch: beginning-of-defun-save-match-data
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: Debian GNU/Linux 11 (bullseye)

Configured using:
 'configure -C --with-native-compilation'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES
NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF
TOOLKIT_SCROLL_BARS X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB

Important settings:
  value of $LC_COLLATE: POSIX
  value of $LC_TIME: POSIX
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils comp comp-cstr
warnings icons subr-x rx cl-seq cl-macs gv cl-extra help-mode
cl-loaddefs cl-lib bytecomp byte-compile rmc iso-transl tooltip cconv
eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type
elisp-mode mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd
fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow
isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo
move-toolbar gtk x-toolkit xinput2 x multi-tty make-network-process
native-compile emacs)

Memory information:
((conses 16 93935 9790)
 (symbols 48 7139 0)
 (strings 32 26103 2634)
 (string-bytes 1 831368)
 (vectors 16 20039)
 (vector-slots 8 365879 14596)
 (floats 8 41 21)
 (intervals 56 237 0)
 (buffers 976 12))

0001-Fix-beginning-of-defun-not-jumping-to-BOF.patch
Description: Text Data