Re: How to add pseudo vector types

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Clément Pit-Claudel <cpitclaudel@gmail.com>
> Date: Tue, 20 Jul 2021 16:36:42 -0400
> 
> Thanks for the detailed reply.
> 
> On 7/17/21 3:16 AM, Eli Zaretskii wrote:
> > When Emacs moves or enlarges/shrinks the gap, that affects the entire
> > buffer text after the gap, regardless of where the gap is.  So it will
> > affect the TS reader if it reads stuff after the gap.
> 
> Doesn't enlarging the gap require allocating a new buffer and copying data to 
> it?

Not necessarily.  First, gap could be enlarged for reasons other than
growing buffer text as a whole.  And even if we must grow buffer text,
a good memory-allocation system will many times resize the existing
memory block before it allocates another..

> If so it wouldn't affect the TS reader.

Not true, in general.  When a new block is allocated by the OS/libc,
the old one is generally invalid and cannot be accessed.  In many
cases, the old block could be unmapped from the program's address
space, in which case accessing it will segfault.

> Moving is indeed trickier, that's what I referred to as "limited contention".

We move the gap quite a lot.

> >> You do need to be careful to not read the garbage data from the gap, but 
> >> otherwise seeing stale or even inconsistent data from the parser thread 
> >> shouldn't be an issue, since tree-sitter is supposed to be robust to bad 
> >> parses.
> > 
> > What would be the purpose of calling the parser if we know in advance
> > it will fail when it gets to the "garbage" caused by async access to
> > the buffer text?
> 
> It won't fail, will it?

"Fail" in the sense that it will be able to process only a small
portion of buffer text before it gets to garbage.

> > And what do you want the code which requested parsing do while the
> > parse thread runs?  The requesting code is in the main thread, so if
> > it just waits, you don't gain anything.
> 
> You'd have the parser running continuously in the background, every time 
> there is a change.  When a piece of code requests a parse it blocks and 
> waits, but presumably for not too long because a very recent previous parse 
> means that the blocking parse is fast.

Well, you cannot safely/usefully parse the buffer "continuously in the
background", for the reasons explained above, because Lisp programs
change buffer text quite a lot.

> > I don't understand what could recording the gap solve.  The stuff in
> > the gap is generally garbage, and can easily include invalid multibyte
> > sequences.  I don't think it's a good idea to pass that to TS.  Also,
> > recording the gap changes in the main thread and accessing that
> > information from a concurrent thread again opens a window for races,
> > and requires synchronization.
> 
> This list of gap changes wouldn't be accessed concurrently: you would 
> (message-)pass a copy of it to the parser thread every time it starts a new 
> parse.

I still don't see the point.  Can you describe in more detail what
would you suggest doing with the list of gap changes?  Just take a
specific example of a small set of gap changes and tell how to use
that.