bug#70077: An easier way to track buffer changes

[Stefan Monnier]

>> The driving design was:
>>
>> - Try to provide enough info such that it is possible and easy to
>>   maintain a copy of the buffer simply by applying the reported changes.
>>   E.g. for uses such as `eglot.el` or `crdt.el`.
>> - Make the API less synchronous: take care of combining small changes
>>   into larger ones, and let the clients decide when they react to changes.
>
> Before we discuss the API, may you allow me to raise one critical
> concern: bug#65451.

Thanks, I wasn't aware of it.

Note that bug#65451 would affect `track-changes.el` but not its clients
nor its API.

Well, I guess it couldn't really insulate its clients from such bugs,
but it would be `track-changes.el`s responsibility to detect such
problems and pass down to its clients something saying "oops we have
a problem, you have to do a full-resync".

> If my reading of the patch is correct, your code is relying upon the
> buffer changes arriving in the same order the changes are being made.

Indeed.

> I am skeptical that you can achieve the desired patch goals purely
> relying upon before/after-change-functions, without reaching down to C
> internals.

There's a FIXME for that:

    ;; FIXME: Try and do some sanity-checks (e.g. looking at `buffer-size'),
    ;; to detect if/when we somehow missed some changes.

All the current non-trivial users of *-change-functions have such sanity
checks.  They're designed to handle those nasty cases where we have
a bug in the C code.  I don't claim my new API can magically paper over
those bugs.  The intention to deal with those bugs is:

- When they're detected, call the clients with a special value for
  `before` which says that we lost track of some change, so the client
  knows that it may be necessary to do a full resync.
  Luckily for many/most clients a full-resync is only a performance
  problem (e.g. having to reparse the whole file), tho for some (like
  lentic) I suspect it can result in an actual loss of information.

- Try and fix them, of course.  Alan has done a great job in the past
  fixing many of them (tho apparently still not all).
  [ And also a great job of convincing me that we *can* and thus
    *should* fix them.  ]

IOW, no magic bullet: the clients would still have to somehow handle such
a "full-resync"s.

The main advantage would be that the job of sanity-checking would be
taken care of by `track-changes.el` and the clients would have to check
only `before` for a special value to indicate a problem.


        Stefan