Re: New package emacs-parser-generator

[Christian Johansson]

Ok thanks for the link, I will investigate and see if I can figure it out. 

Read a bit of the research and it seems to to use a state-independent 
lex-analyzer and do like multiple alternative / nondeterministic error-tolerant 
parses to make a probalistic best parse which is then merged into the previous 
AST or something like that..

Regards
Christian

> 1 dec. 2021 kl. 20:25 skrev Yuan Fu <casouri@gmail.com>:
> 
> 
>> 
>>> 
>>> Tree-sitter parses incrementally, I modified primitive insert/delete 
>>> functions in insdel.c to incrementally parse changed content. There is no 
>>> need for threads as incremental parsing is extremely fast. We don’t make 
>>> copies of buffer string, instead, we pass tree-sitter library a function 
>>> that reads directly from the buffer.
>> 
>> Ok where can I read about this function? How does treesitter handle 
>> incremental parses which are state-dependent? For example PHPs lex-analyzer 
>> works differently in different states of the grammar, like do you signal to 
>> treesitter a point in the buffer so it can backtrack the parsers states in 
>> order to correctly perform a incremental parse on the new content?
> 
> I assume you mean the read function? It can be found here:
> 
> https://github.com/casouri/emacs/blob/106d050ad5d02f673f8a089e1f10c1eacfedd124/src/tree-sitter.c#L372
> 
> Tree-sitter only requires to be informed of every change to the buffer, it 
> will read the buffer for itself and update the AST. I have no idea if it 
> backtracks behind the scenes. Presumably tree-sitter can figure out from 
> where to backtrack from the change information we give it.
> 
> Yuan