bug#72165: 31.0.50; Intermittent crashing with recent emacs build

[Eli Zaretskii]

> From: Dima Kogan <dima@secretsauce.net>
> Cc: 72165@debbugs.gnu.org
> Date: Mon, 29 Jul 2024 05:49:54 -0700
> 
> - I'm at the top of redisplay_window() that has a crashing state, right
>   after a redisplay_window() cycle with a non-crashing state. The
>   crashing state has z_byte-z = 8 but pt_byte-pt = 12. Both are at the
>   end of the buffer, and 8 is the correct value.
> 
> - I run backwards until the truncation in message_dolog():
> 
>         if (FIXNATP (Vmessage_log_max))
>           {
>             scan_newline (Z, Z_BYTE, BEG, BEG_BYTE,
>                           -XFIXNAT (Vmessage_log_max) - 1, false);
>             del_range_both (BEG, BEG_BYTE, PT, PT_BYTE, false);
>           }
> 
>   Right after this both z_byte-z=8 (correct) and the point is at the
>   beginning. This is correct, and the truncation itself didn't break
>   anything. Note: this was (eventually) called from redisplay_window(),
>   in the display_mode_lines() call in xdisp.c:20941
> 
> - Then we restore the oldpoint; what I suggested was the problem
>   previously:
> 
>       if (point_at_end)
>       TEMP_SET_PT_BOTH (Z, Z_BYTE);
>       else
>       /* We can't do Fgoto_char (oldpoint) because it will run some
>            Lisp code.  */
>       TEMP_SET_PT_BOTH (marker_position (oldpoint),
>                         marker_byte_position (oldpoint));
> 
>   After that, both z_byte-z and pt_byte-pt are 8. This is correct.
> 
> - Then we do this, in redisplay_window() in xdisp.c:21908
> 
>   /* Restore current_buffer and value of point in it.  The window
>      update may have changed the buffer, so first make sure `opoint'
>      is still valid (Bug#6177).  */
>   if (CHARPOS (opoint) < BEGV)
>     TEMP_SET_PT_BOTH (BEGV, BEGV_BYTE);
>   else if (CHARPOS (opoint) > ZV)
>     TEMP_SET_PT_BOTH (Z, Z_BYTE);
>   else
>     TEMP_SET_PT_BOTH (CHARPOS (opoint), BYTEPOS (opoint));
> 
>   We pick the "else" branch. The end result is the exact failing state
>   we hit on the next pass through redisplay_window().
> 
> 
> This sounds like a different flavor of what I described yesterday. So in
> summary: in a redisplay_window() call:
> 
> - We set the opoint on line 20064. This is a text_pos, NOT a marker. So
>   it wouldn't be updated due to the buffer changing.
> 
> - On line 20941 we call display_mode_lines() which eventually deletes
>   lines at the start of *Messages* and moves the point.
> 
> - On line 21908 we restore the opoint, saved prior to the lines being
>   deleted, and no longer valid

So let me see if I understand you correctly regarding what happens:

  . The *Messages* buffer is displayed in a window, which is
    redisplayed, and the display engine calls redisplay_window for it.
  . redisplay_window records the original position of point in the
    *Messages* buffer, then calls display_mode_lines, as it does for
    any window whose mode line needs to be redrawn for some reason
  . somewhere inside display_mode_lines, we call message_dolog, most
    probably because the mode-line format calls :eval, which signals
    an error
  . message_dolog adds some text to *Messages* and removes some other
    text from it, which invalidates the position of point recorded at
    the beginning of redisplay_window
  . redisplay_window then uses invalid value of point (including its
    byte position, which no longer corresponds to the character
    position) to set point, and that opens the gates of hell

Is that correct?  If so, this puzzle has the following pieces:

  . *Messages* is displayed and includes non-ASCII text
  . mode-line-format that signals an error when the window showing
    *Messages* is redisplayed
  . the size of *Messages* buffer and its contents are such that
    moving point to the value recorded at entry to redisplay_window
    produces a mismatch between PT and PT_BYTE

If all of the above happen, we are toast.  Right?

Can you verify that the above theory is true?  For example does
CHARS_MODIFF value of the buffer after display_mode_lines returns
differ from its value before the call?