bug#11073: 24.0.94; BIDI-related crash in redisplay with certain byte sequences

[Eli Zaretskii]

The person who reported this to me in private email won't go public,
for whatever reasons, so I'm reporting this for them.

The recipe:

 emacs -Q
 C-x C-f bidicrash.txt RET

where the file bidicrash.txt was created with this shell command:

 echo -e "\0365\0205\0264\0225"

(On Windows, use the port of GNU `echo' rather than the built-in shell
command.)

Emacs crashes; the backtrace is below.

I'm working on fixing this.

Breakpoint 1, w32_abort () at w32fns.c:7196
7196      button = MessageBox (NULL,
(gdb) bt
#0  w32_abort () at w32fns.c:7196
#1  0x012f2e49 in bidi_get_type (ch=4195533, override=NEUTRAL_DIR)
    at bidi.c:108
#2  0x012f4120 in bidi_resolve_explicit_1 (bidi_it=0x82cff8) at bidi.c:1400
#3  0x012f44a8 in bidi_resolve_explicit (bidi_it=0x82cff8) at bidi.c:1529
#4  0x012f4a2f in bidi_resolve_weak (bidi_it=0x82cff8) at bidi.c:1614
#5  0x012f5110 in bidi_resolve_neutral (bidi_it=0x82cff8) at bidi.c:1850
#6  0x012f5a49 in bidi_type_of_next_char (bidi_it=0x82cff8) at bidi.c:2020
#7  0x012f5d6f in bidi_level_of_next_char (bidi_it=0x82cff8) at bidi.c:2133
#8  0x012f630e in bidi_move_to_visually_next (bidi_it=0x82cff8) at bidi.c:2342
#9  0x0116aded in set_iterator_to_next (it=0x82ca40, reseat_p=1)
    at xdisp.c:6898
#10 0x011941c1 in display_line (it=0x82ca40) at xdisp.c:19341
#11 0x0118917a in try_window (window=55991301, pos=..., flags=1)
    at xdisp.c:15977
#12 0x01186a32 in redisplay_window (window=55991301, just_this_one_p=0)
    at xdisp.c:15502
#13 0x011800b8 in redisplay_window_0 (window=55991301) at xdisp.c:13625
#14 0x01033d1b in internal_condition_case_1 (
    bfun=0x1180086 <redisplay_window_0>, arg=55991301, handlers=53234414,
    hfun=0x1180065 <redisplay_window_error>) at eval.c:1553
#15 0x01180055 in redisplay_windows (window=55991301) at xdisp.c:13605
#16 0x0117dff8 in redisplay_internal () at xdisp.c:13182
#17 0x0117b2ea in redisplay () at xdisp.c:12405
#18 0x010087fb in read_char (commandflag=1, nmaps=2, maps=0x82fa30,
    prev_event=53250074, used_mouse_menu=0x82fb5c, end_time=0x0)
    at keyboard.c:2446
#19 0x0101c246 in read_key_sequence (keybuf=0x82fc60, bufsize=30,
    prompt=53250074, dont_downcase_last=0, can_return_switch_frame=1,
    fix_current_buffer=1) at keyboard.c:9326
#20 0x01005aa8 in command_loop_1 () at keyboard.c:1448
#21 0x01033c0b in internal_condition_case (bfun=0x10054b6 <command_loop_1>,
    handlers=53307802, hfun=0x1004ce0 <cmd_error>) at eval.c:1515
#22 0x0100511c in command_loop_2 (ignore=53250074) at keyboard.c:1159
#23 0x010335cb in internal_catch (tag=53305826,
    func=0x10050f9 <command_loop_2>, arg=53250074) at eval.c:1272
#24 0x010050d4 in command_loop () at keyboard.c:1138
#25 0x0100469e in recursive_edit_1 () at keyboard.c:758
#26 0x010049c0 in Frecursive_edit () at keyboard.c:822
#27 0x010027c8 in main (argc=2, argv=0xa32880) at emacs.c:1715
(gdb) up
#1  0x012f2e49 in bidi_get_type (ch=4195533, override=NEUTRAL_DIR)
    at bidi.c:108
108         abort ();
(gdb) up
#2  0x012f4120 in bidi_resolve_explicit_1 (bidi_it=0x82cff8) at bidi.c:1400
1400      type = bidi_get_type (curchar, NEUTRAL_DIR);
(gdb) p bidi_it->charpos
$1 = 2
(gdb) p bidi_it->bytepos
$2 = 4
(gdb) p bidi_it->ch_len
$3 = 2
(gdb) p bidi_it->ch
$4 = 4195533
(gdb) p/x bidi_it->ch
$5 = 0x4004cd
(gdb)

This is on Windows.  On GNU/Linux, or if you change the EOL format of
the file to be Unix-style LF, the last command prints 0x4004ca
instead.  Evidently, Emacs is trying to produce a Unicode codepoint
from bytes that include the newline sequence.


In GNU Emacs 24.0.94.1 (i386-mingw-nt5.1.2600)
 of 2012-02-27 on HOME-C4E4A596F7
Windowing system distributor `Microsoft Corp.', version 5.1.2600
Configured using:
 `configure --with-gcc (3.4)'

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: ENU
  value of $XMODIFIERS: nil
  locale-coding-system: cp1255
  default enable-multibyte-characters: t

Major mode: Mail

Minor modes in effect:
  diff-auto-refine-mode: t
  flyspell-mode: t
  desktop-save-mode: t
  show-paren-mode: t
  display-time-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  temp-buffer-resize-mode: t
  line-number-mode: t
  abbrev-mode: t

Recent input:
a l SPC D E F A U L T _ F A C E _ I D S-SPC i n t a 
c t . ) <down> <return> <return> S o SPC t h i s SPC 
b u g SPC h a s SPC r a h e <backspace> <backspace> 
t h e r SPC l o w SPC p r i o r i t y SPC a t SPC t 
h i s SPC t i m e , SPC a s SPC i t ' s SPC n o t SPC 
a SPC r e g r e s s i o n SPC w r t SPC E m a c s SPC 
2 3 . SPC SPC N e v e r t h e l e s s , <M-backspace> 
I S-SPC w i l l SPC a t SPC t h e SPC v e r y SPC l 
e a s t SPC t r y SPC t o SPC f i g u r e SPC o u t 
SPC w h a t SPC c h a n g e s SPC a r e SPC n e e d 
e d SPC t o SPC m a k e SPC t h i s SPC w o r k SPC 
a s SPC e x p e c t e d . <return> <up> <up> <C-right> 
<C-right> <C-right> <C-right> <C-right> <C-left> T 
i m e SPC p e r m i t t i n g , SPC M-q <down> <down> 
<down> <up> <up> <up> <up> <M-left> <C-home> C-c C-s 
<switch-frame> n n n n p p <switch-frame> M-x e m a 
c s - r e <M-backspace> <M-backspace> r e p o r t <tab> 
<return>

Recent messages:
Mark set [4 times]
Auto-saving...done
byte-code: End of buffer
Auto-saving...done
Mark set
Sending...
Added to d:/usr/eli/rmail/SENT.MAIL
Sending email 
Sending email done
Sending...done

Load-path shadows:
None found.

Features:
(shadow emacsbug etags cc-awk network-stream starttls tls smtpmail
auth-source eieio assoc gnus-util password-cache mailalias sendmail
multi-isearch find-func help-mode view rmailout dabbrev ld-script
dired-x dired tcl nxml-uchnm rng-xsd xsd-regexp rng-cmpct rng-nxml
rng-valid rng-loc rng-uri rng-parse nxml-parse rng-match rng-dt
rng-util rng-pttrn nxml-ns nxml-mode nxml-outln nxml-rap nxml-util
nxml-glyph nxml-enc xmltok sgml-mode org-wl org-w3m org-vm org-rmail
org-mhe org-mew org-irc org-jsinfo org-infojs org-html org-exp ob-exp
org-exp-blocks org-agenda org-info org-gnus org-docview org-bibtex
bibtex org-bbdb org byte-opt warnings bytecomp byte-compile cconv
macroexp advice help-fns advice-preload ob-emacs-lisp ob-tangle ob-ref
ob-lob ob-table org-footnote org-src ob-comint ob-keys ob ob-eval
org-pcomplete pcomplete org-list org-faces org-compat org-entities
org-macs cal-menu calendar cal-loaddefs noutline outline arc-mode
archive-mode diff-mode conf-mode newcomment parse-time sh-script
executable gud easy-mmode comint ansi-color ring generic jka-compr
make-mode flyspell ispell vc-cvs autorevert info vc-bzr cc-mode
cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine
cc-vars cc-defs regexp-opt qp rmailsum rmailmm message format-spec
rfc822 mml mml-sec mm-decode mm-bodies mm-encode mailabbrev gmm-utils
mailheader mail-parse rfc2231 rmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils desktop server filecache mairix cus-edit
easymenu cus-start cus-load wid-edit saveplace midnight generic-x
paren battery time time-date tooltip ediff-hook vc-hooks
lisp-float-type mwheel dos-w32 disp-table ls-lisp w32-win w32-vars
tool-bar dnd fontset image fringe lisp-mode register page menu-bar
rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax
facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak
czech european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer loaddefs button faces
cus-face files text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget hashtable-print-readable backquote
make-network-process multi-tty emacs)