Re: Tree-sitter maturity

[Björn Bidar]

Yuan Fu <casouri@gmail.com> writes:

>> On Dec 29, 2024, at 3:29 PM, Björn Bidar <bjorn.bidar@thaodan.de> wrote:
>> 
>> Daniel Colascione <dancol@dancol.org> writes:
>> 
>>> Lynn Winebarger <owinebar@gmail.com> writes:
>>> 
>>>> On Fri, Dec 27, 2024, 9:25 AM Daniel Colascione <dancol@dancol.org> wrote:
>>>> 
>>>>> 
>>>>> 
>>>>> It's a shame there's no way to write TS grammars in plain elisp. I figure
>>>>> vendoring both the source and the generated code would be best, as it'd
>>>>> allow building Emacs anywhere but still make it convenient on systems with
>>>>> needed tools (JS runtime, Rust, etc.) to update and modify the grammar. As
>>>>> with any scheme involving checking in generated outputs, the source and
>>>>> output can get out of sync, but I think there are build time guardrails we
>>>>> can build to make sure it doesn't happen.
>>>>> 
>>>> 
>>>> I looked into this last year.  The tree-sitter library provides a parsing
>>>> engine that references a fairly standard LR type parsing table in binary
>>>> form.  I got stuck in adding a generic primitive functionality for reading
>>>> and writing arbitrary binary data structures based on a data description
>>>> DSL, since I wouldn't want to tie the interpreter core to the data
>>>> structures of an external, dynamically-loadable library.  But, I wasn't
>>>> sure such an extension would be accepted into emacs, as I am not an expert
>>>> on the possible security implications.
>>>> 
>>>> Other than that, emacs already has the code for calculating (LA)LR parsing
>>>> tables in the semantic packages.  The tree-sitter grammar compiler may have
>>>> additional logic for providing multiple starting symbols, but the parsing
>>>> engine should still function with a classic parsing table.
>>> 
>>> Thanks.  Such an approach would let us treat tree-sitter grammars a lot
>>> more like font-lock-keywords, and I think for some modes, that'd be a
>>> good option.  (Of course, SHTDI.)
>>> 
>>> Tree sitter, as wonderful as it is, strikes me as a bit of a Rube
>>> Goldberg machine architecturally: JS *and* Rust *and* C? Really? :-)
>
>> I was wondering the same. How the hell? There had been some talks to
>> support a more lightweight JavaScript interpreter as an alternative but
>> it hasn't gone anyway. Somehow because compatibility reason. I don't how
>> could node be dependency for these. Grammars are mostly without
>> dependencies except some have dependencies to other grammars on the
>> source level such as the C++ require the C grammar.
>
> I don’t think you need nodejs to build the grammar. You might need it
> to develop the grammar, but compiling grammar.js to parser.c only
> requires the tree-sitter CLI which is written in Rust.

If you want to build from source the you need to compile
grammar.js. That the compiled grammar is bundled is practice for
compatibility reasons.