bug#65451: 30.0.50; `after-change-functions' are not triggered in the same order the changes are made

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: casouri@gmail.com, 65451@debbugs.gnu.org
> Date: Wed, 23 Aug 2023 08:52:30 +0000
> 
> To update the old AST we need to know which regions in the old AST (and
> old buffer text) were edited and their change in length. Then, we (1)
> remove elements in AST that are structurally affected by the edit;
> (2) shift elements after the edit that are not structurally affected but
> whose boundaries must be adjusted to accommodate for the buffer length
> being changed; (3) re-parse buffer text (after the edit) and fill the
> gap in the AST.
> 
> For step (1), it is critical to have information about changed regions
> in the same order the edits happen. And, in theory, we might utilize
> before-change-functions to obtain this information (assuming that
> before-change-functions are always called in the same order the edits
> happen). But, unfortunately, before-change-functions are too noisy for
> this purpose because, for example, simply altering text properties also
> triggers before-change-functions, while no actual change in buffer text
> occurs in such scenario and re-parsing AST is unnecessary. So, we
> instead have to use after-change-functions and work out the original
> changed region boundaties using
> beg_changed end_changed pre-change_lentgh ->
> beg_before_change = beg_changed;
> end_before_change = end_changed - pre-change_length
> This calculation of the original changed region becomes incorrect when
> the order of after-change-functions is not the same as the editing order
> - this bug report.

If these measures still don't help you enough, perhaps the conclusion
is that it isn't feasible to implement text parsers in Lisp, at least
as long as you want all those micro-optimizations of knowing exactly
which parts of the buffer text were modified (as opposed to only know
how many characters at the beginning and at the end of the buffer
remain unchanged, and reparse the rest).  Maybe it must be done in C,
if we want Emacs to remain a relatively safe environment.

> Indeed. As I tried to explain, the problem for me is not the granularity
> of changes, but their ordering.

I still don't understand why the ordering matters, but I do know that
you cannot rely on it, and I hope I explained why.