bug#69536: 30.0.50; Runtime error happens with Tamago input method after recent change with obarry

[Yasuhiro Kimura]

Hello,

I use Emacs on FreeBSD. To install Emacs I use FreeBSD Ports
Collection. In FreeBSD Ports Collection there are two versions of
Emacs. One is "editors/emacs", latest release version, and the other
is "editors/emacs-devel", shapshot from master branch of Emacs git
repository. And I use the latter.

To input non-ascii characters I also use Tamago input method. In
FreeBSD Ports Collection source code used by Tamago port
(editors/tamago) is maintained by the maintainer of the port in
following repository.

https://github.com/hrs-allbsd/tamago/tree/freebsd

Last Saturday "editors/emacs-devel" was updated from commit
4b89fb08bdd (commited at Fri Feb 16 22:17:57 2024 +0800) to commit
8b96503b6e8 (commited at Fri Mar 1 18:58:44 2024 +0100). And after
that runtime error happens as following when Tamago is invoked by
typing 'C-\'.

----------------------------------------------------------------------
Debugger entered--Lisp error: (wrong-type-argument obarrayp [nil nil nil nil 
nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil 
nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil 
nil nil nil nil nil nil ...])
  intern("　" [nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil 
nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil 
nil nil nil nil nil nil nil nil nil nil nil nil nil nil ...])
  
byte-code("\306\307\310\311\"\311\211\30\31\32\33\13\203H\0\13@@\21\13@A\20\13A\23\10\203C\0\312\313\10@@!\f\"\10@AL\210\312\314\11!\313\10@A!P\15\"\10@@L\210\10A\211\20\204\36\0\n\210\202\13\0,\311\207"
 [pair lang hash table its-full-half-table its-half-full-table ((Japanese 
(12288 . 32) (65292 . 44) (65294 . 46) (12289 . 44) (12290 . 46) (65306 . 58) 
(65307 . 59) (65311 . 63) (65281 . 33) (180 . 39) (65344 . 96) (65342 . 94) 
(65343 . 95) (65507 . 126) (12540 . 45) (8212 . 45) (8208 . 45) (65295 . 47) 
(65340 . 92) (12316 . 126) (65372 . 124) (8216 . 96) (8217 . 39) (8220 . 34) 
(8221 . 34) (65288 . 40) (65289 . 41) (65339 . 91) (65341 . 93) (65371 . 123) 
(65373 . 125) (12296 . 60) (12297 . 62) (65291 . 43) (8722 . 45) (65309 . 61) 
(65308 . 60) (65310 . 62) (8242 . 39) (8243 . 34) (65509 . 92) (65284 . 36) 
(65285 . 37) (65283 . 35) (65286 . 38) (65290 . 42) (65312 . 64) (65296 . 48) 
(65297 . 49) ...) (Chinese-GB (12288 . 32) (65292 . 44) (65294 . 46) (12289 . 
44) (12290 . 46) (65306 . 58) (65307 . 59) (65311 . 63) (65281 . 33) (65344 . 
96) (65342 . 94) (65343 . 95) (65507 . 126) (8213 . 45) (65295 . 47) (65340 . 
92) (65374 . 126) (65372 . 124) (8216 . 96) (8217 . 39) (8220 . 34) (8221 . 34) 
(65288 . 40) (65289 . 41) (65339 . 91) (65341 . 93) (65371 . 123) (65373 . 125) 
(65291 . 43) (65293 . 45) (65309 . 61) (65308 . 60) (65310 . 62) (65287 . 39) 
(65282 . 34) (65509 . 36) (65285 . 37) (65283 . 35) (65286 . 38) (65290 . 42) 
(65312 . 64) (65296 . 48) (65297 . 49) (65298 . 50) (65299 . 51) (65300 . 52) 
(65301 . 53) (65302 . 54) (65303 . 55) ...) (Chinese-CNS (12288 . 32) (65292 . 
44) (65294 . 46) (12289 . 44) (12290 . 46) (65306 . 58) (65307 . 59) (65311 . 
63) (65281 . 33) (8245 . 39) (8242 . 96) (65087 . 94) (65343 . 95) (8254 . 126) 
(65123 . 45) (65295 . 47) (65340 . 92) (8725 . 47) (65128 . 92) (8764 . 126) 
(65372 . 124) (8216 . 96) (8217 . 39) (12317 . 34) (12318 . 34) (8220 . 34) 
(8221 . 34) (65288 . 40) (65289 . 41) (12308 . 91) (12309 . 93) (65117 . 91) 
(65118 . 93) (65371 . 123) (65373 . 125) (65115 . 123) (65116 . 125) (12296 . 
60) (12297 . 62) (65291 . 43) (65293 . 45) (65309 . 61) (65308 . 60) (65310 . 
62) (65284 . 36) (65285 . 37) (65283 . 35) (65286 . 38) (65290 . 42) ...) 
(Korean (12288 . 32) (65292 . 44) (65294 . 46) (65306 . 58) (65307 . 59) (65311 
. 63) (65281 . 33) (8217 . 39) (8216 . 96) (65342 . 94) (65343 . 95) (65507 . 
126) (8213 . 45) (173 . 45) (65295 . 47) (65340 . 92) (8764 . 126) (65372 . 
124) (8216 . 96) (8217 . 39) (8220 . 34) (8221 . 34) (65288 . 40) (65289 . 41) 
(65339 . 91) (65341 . 93) (65371 . 123) (65373 . 125) (12296 . 60) (12297 . 62) 
(65291 . 43) (65293 . 45) (65309 . 61) (65308 . 60) (65310 . 62) (65287 . 39) 
(65282 . 34) (65510 . 92) (65284 . 36) (65285 . 37) (65283 . 35) (65286 . 38) 
(65290 . 42) (65312 . 64) (65296 . 48) (65297 . 49) (65298 . 50) (65299 . 51) 
(65300 . 52) ...)) make-vector 100 nil intern char-to-string symbol-name] 5)
  require(its)
  
byte-code("\300\301!\210\300\302!\210\300\303!\210\300\304!\210\300\305!\210\306\307\310\"\207"
 [require its menudiag egg-mlh egg-cnv egg-com add-hook kill-emacs-hook 
egg-kill-emacs-function] 3)
  require(egg)
  
byte-code("\300\301!\210\300\302!\210\303\304\305\306\307\301%\210\310\311\305\312\307\304\313\314&\7\210\310\315\316\317\307\304\313\320&\7\210\310\321\305\322\307\304\313\323&\7\210\324\325\326\"\207"
 [require egg egg-edep custom-declare-group anthy-egg nil "Anthy interface for 
Tamago 4." :group custom-declare-variable anthy-egg-use-utf8 
("/usr/local/share/emacs/30.0.50/site-lisp/egg/egg/anthy.elc" . 86) :type 
(choice (const :tag "Use UTF8" t) (const :tag "No use UTF8" nil)) 
anthy-egg-input-method-list '("japanese-egg-anthy") 
("/usr/local/share/emacs/30.0.50/site-lisp/egg/egg/anthy.elc" . 138) (repeat 
(string :format "Input method: %v\n" :size 0)) 
anthy-egg-use-chinese-korean-server 
("/usr/local/share/emacs/30.0.50/site-lisp/egg/egg/anthy.elc" . 183) (choice 
(const :tag "Use Anthy with Chinese and/or Korean servers" t) (const :tag "No 
use" nil)) setplist anthy-egg-conversion-backend (egg-start-conversion 
anthy-egg-convert egg-get-bunsetsu-source anthy-egg-get-bunsetsu-source 
egg-get-bunsetsu-converted anthy-egg-get-bunsetsu-converted egg-list-candidates 
anthy-egg-get-candidates egg-decide-candidate anthy-egg-select-candidate 
egg-change-bunsetsu-length anthy-egg-resize-segment egg-end-conversion 
anthy-egg-commit egg-special-candidate anthy-egg-special-candidate 
egg-word-registration anthy-egg-word-registration egg-get-source-language 
anthy-egg-get-source-language egg-get-converted-language 
anthy-egg-get-converted-language)] 8)
  egg-activate-anthy("japanese-egg-anthy" its-select-hiragana)
  activate-input-method("japanese-egg-anthy")
  toggle-input-method(nil 1)
  funcall-interactively(toggle-input-method nil 1)
  command-execute(toggle-input-method)
----------------------------------------------------------------------

I tried `git bisect` with Emacs git repository. And according to the
result of it runtime error start either of following successive two
commits. (`git bisect` can't determine which is the exact start of the
problem as build of commit 462d8ba813e fails with segfault of pdumper
process.)

----------------------------------------------------------------------
commit 3ea77c735de
Author:     Mattias Engdegård <mattiase@acm.org>
AuthorDate: Sun Feb 11 15:11:21 2024 +0100
Commit:     Mattias Engdegård <mattiase@acm.org>
CommitDate: Fri Feb 23 13:02:27 2024 +0100

    Use the new obarray type for the initial obarray
    
    This can improve performance a lot, especially after the obarray has
    been fed many symbols.
    
    * src/lread.c (OBARRAY_SIZE): Remove.
    (load_path_check): Create an obarray object instead of a vector.

commit 462d8ba813e
Author:     Mattias Engdegård <mattiase@acm.org>
AuthorDate: Sat Feb 10 21:14:09 2024 +0100
Commit:     Mattias Engdegård <mattiase@acm.org>
CommitDate: Fri Feb 23 13:02:27 2024 +0100

    Add a proper type for obarrays
    
    The new opaque type replaces the previous use of vectors for obarrays.
    `obarray-make` now returns objects of this type.  Functions that take
    obarrays continue to accept vectors for compatibility, now just using
    their first slot to store an actual obarray object.
    
    obarray-size and obarray-default-size now obsolete.
    
    * lisp/obarray.el (obarray-default-size, obarray-size):
    Declare obsolete.
    (obarray-make, obarrayp, obarray-clear): Remove from here.
    * src/fns.c (reduce_emacs_uint_to_hash_hash): Remove from here.
    * src/lisp.h (struct Lisp_Obarray, OBARRAYP, XOBARRAY, CHECK_OBARRAY)
    (make_lisp_obarray, obarray_size, check_obarray)
    (obarray_iter_t, make_obarray_iter, obarray_iter_at_end)
    (obarray_iter_step, obarray_iter_symbol, DOOBARRAY, knuth_hash): New.
    (reduce_emacs_uint_to_hash_hash): Moved here.
    * src/lread.c (check_obarray): Renamed and reworked as...
    (checked_obarray_slow): ...this.
    (intern_sym, Funintern, oblookup, map_obarray)
    (Finternal__obarray_buckets): Adapt to new type.
    (obarray_index, allocate_obarray, make_obarray, grow_obarray)
    (obarray_default_bits, Fobarray_make, Fobarrayp, Fobarray_clear): New.
    * etc/emacs_lldb.py (Lisp_Object):
    * lisp/emacs-lisp/cl-macs.el (`(,type . ,pred)):
    * lisp/emacs-lisp/cl-preloaded.el (cl--typeof-types):
    * lisp/emacs-lisp/comp-common.el (comp-known-type-specifiers):
    * lisp/emacs-lisp/comp.el (comp-known-predicates):
    * src/alloc.c (cleanup_vector, process_mark_stack):
    * src/data.c (Ftype_of, syms_of_data):
    * src/minibuf.c (Ftry_completion, Fall_completions, Ftest_completion):
    * src/pdumper.c (dump_obarray_buckets, dump_obarray, dump_vectorlike):
    * src/print.c (print_vectorlike_unreadable):
    * test/lisp/abbrev-tests.el (abbrev-make-abbrev-table-test):
    * test/lisp/obarray-tests.el (obarrayp-test)
    (obarrayp-unchecked-content-test, obarray-make-default-test)
    (obarray-make-with-size-test):
    Adapt to new type.
----------------------------------------------------------------------

As far as I read the commit messages they seems to be related with
internal change of obarray type.

To be honest I don't know about internal of Tamago almost at all. But
as far as I search the source tree with grep, Tamago doesn't seem to
use obarray (at least directry).

So I guess that change of obarray causes incompatible change of
function (or somthing else) that uses it and it results in the runtime
error of Tamago.

In GNU Emacs 30.0.50 (build 1, amd64-portbld-freebsd14.0)
Repository revision: 8b96503b6e8514f1f9f92895a0707c78b1bbd1fd
Repository branch: master
System Description: 14.0-RELEASE-p5

Configured using:
 'configure --disable-build-details --localstatedir=/var
 --without-gconf --without-libsystemd --without-selinux
 --with-sound=no --with-x-toolkit=no --without-cairo --without-dbus
 --without-gconf --without-gif --without-gsettings
 --without-imagemagick --without-jpeg --without-lcms2 --without-libotf
 --without-m17n-flt --without-pgtk --without-png --without-rsvg
 --without-tiff --without-toolkit-scroll-bars --without-webp
 --without-x --without-xim --without-xpm --without-xwidgets
 --enable-acl --without-cairo --without-dbus --without-gif
 --with-gnutls --without-gsettings --without-harfbuzz --without-jpeg
 --with-json --with-file-notification=kqueue --without-lcms2
 --without-m17n-flt --without-imagemagick --without-mailutils
 --with-modules --with-native-compilation=aot --without-libotf
 --without-pgtk --without-png --without-toolkit-scroll-bars
 --with-sqlite3 --without-rsvg --with-threads --without-tiff
 --with-tree-sitter --without-webp --without-xft --without-xim
 --with-xml2 --without-xpm --without-xwidgets --with-x-toolkit=no
 --prefix=/usr/local --mandir=/usr/local/share/man
 --disable-silent-rules --infodir=/usr/local/share/emacs/info/
 --build=amd64-portbld-freebsd14.0 'CFLAGS=-O2 -pipe
 -fstack-protector-strong -Wl,-rpath=/usr/local/lib/gcc13 -isystem
 /usr/local/include -fno-strict-aliasing ' 'CPPFLAGS=-isystem
 /usr/local/include' 'LDFLAGS= -fstack-protector-strong
 -Wl,-rpath=/usr/local/lib/gcc13 -L/usr/local/lib/gcc13
 -L/usr/local/lib ''

Configured features:
ACL GMP GNUTLS JSON LIBXML2 MODULES NATIVE_COMP NOTIFY KQUEUE PDUMPER
SQLITE3 THREADS TREE_SITTER ZLIB

Important settings:
  value of $LANG: ja_JP.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Text

Minor modes in effect:
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  line-number-mode: t
  auto-fill-function: do-auto-fill
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/usr/local/share/emacs/30.0.50/site-lisp/transient hides 
/usr/local/share/emacs/30.0.50/lisp/transient
/home/yasu/share/emacs/lisp/ruby-mode hides 
/usr/local/share/emacs/30.0.50/lisp/progmodes/ruby-mode
/usr/local/share/emacs/30.0.50/site-lisp/flim/sasl hides 
/usr/local/share/emacs/30.0.50/lisp/net/sasl
/usr/local/share/emacs/30.0.50/site-lisp/compat/compat hides 
/usr/local/share/emacs/30.0.50/lisp/emacs-lisp/compat

Features:
(shadow pp mew-varsx mew-unix mew-w3m w3m doc-view jka-compr
image-mode exif timezone w3m-hist bookmark-w3m w3m-ems wid-edit
w3m-favicon w3m-image w3m-fb tab-line w3m-proc w3m-util mew-auth
mew-config mew-imap2 mew-imap mew-nntp2 mew-nntp mew-pop mew-smtp
mew-ssl mew-ssh mew-net mew-highlight mew-sort mew-fib mew-ext
mew-refile mew-demo mew-attach mew-draft mew-message mew-thread
mew-virtual mew-summary4 mew-summary3 mew-summary2 mew-summary
mew-search mew-pick mew-passwd mew-scan mew-syntax mew-bq mew-smime
mew-pgp mew-header mew-exec mew-mark mew-mime mew-edit mew-decode
mew-encode mew-cache mew-minibuf mew-complete mew-addrbook mew-local
mew-vars3 mew-vars2 mew-vars mew-env mew-lang-jp mew-mule3 mew-mule
mew-gemacs mew-key mew-func mew-blvs mew-const mew emacsbug compile
term/tmux term/xterm xterm magit-version yaml-mode tmux rng-nxml
rng-valid rng-loc rng-uri rng-parse nxml-parse rng-match rng-dt
rng-util rng-pttrn nxml-ns nxml-mode nxml-outln nxml-rap sgml-mode
facemenu dom nxml-util nxml-enc xmltok forge-list hl-line
forge-commands forge-semi forge-bitbucket buck forge-gogs gogs
forge-gitea gtea forge-gitlab glab forge-github ghub-graphql treepy
gsexp ghub url-http url-gw nsm url-auth let-alist gnutls forge-notify
forge-revnote forge-pullreq forge-issue forge-topic yaml parse-time
iso8601 bug-reference forge-post markdown-mode color noutline outline
forge-repo forge forge-core forge-db closql emacsql-sqlite-common
emacsql emacsql-compiler eieio-base pcase magit-submodule magit-blame
magit-stash magit-reflog magit-bisect magit-push magit-pull
magit-fetch magit-clone magit-remote magit-commit magit-sequence
magit-notes magit-worktree magit-tag magit-merge magit-branch
magit-reset magit-files magit-refs magit-status magit package
browse-url url url-proxy url-privacy url-expand url-methods
url-history url-cookie generate-lisp-file url-domsuf url-util
url-handlers url-parse auth-source json map url-vars magit-repos
magit-apply magit-wip magit-log which-func imenu magit-diff
smerge-mode diff image diff-mode easy-mmode git-commit log-edit
message sendmail mailcap yank-media puny dired dnd dired-loaddefs
rfc822 mml mml-sec password-cache epa derived epg rfc6068 epg-config
gnus-util text-property-search time-date mm-decode mm-bodies mm-encode
mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr
mailabbrev mail-utils gmm-utils mailheader pcvs-util add-log
magit-core magit-autorevert autorevert filenotify magit-margin
magit-transient magit-process with-editor comp comp-cstr warnings
icons comp-run comp-common shell pcomplete comint regexp-opt ansi-osc
ring server ansi-color magit-mode transient cl-extra edmacro kmacro
help-mode tool-bar magit-git magit-base magit-section cl-seq
format-spec cursor-sensor crm subr-x eieio byte-opt bytecomp
byte-compile eieio-core cl-macs gv dash cl-loaddefs cl-lib
gitignore-mode gitconfig-mode rx conf-mode gitattributes-mode
thingatpt compat cp5022x japan-util rmc iso-transl tooltip cconv eldoc
paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode
tabulated-list replace newcomment text-mode lisp-mode prog-mode
register page tab-bar menu-bar rfn-eshadow isearch easymenu timer
select mouse jit-lock font-lock syntax font-core term/tty-colors frame
minibuffer nadvice seq simple cl-generic indonesian philippine cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj
charscript charprop case-table epa-hook jka-cmpr-hook help abbrev
obarray oclosure cl-preloaded button loaddefs theme-loaddefs faces
cus-face macroexp files window text-properties overlay sha1 md5 base64
format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads kqueue multi-tty
make-network-process native-compile emacs)

Memory information:
((conses 16 341556 22831) (symbols 48 27534 35)
 (strings 32 95697 5580) (string-bytes 1 2847054) (vectors 16 49388)
 (vector-slots 8 668642 16172) (floats 8 294 9985)
 (intervals 56 421 0) (buffers 984 14))