Re: Tree-sitter maturity

[Daniel Colascione]

On December 29, 2024 10:05:26 AM EST, "Björn Bidar" <bjorn.bidar@thaodan.de> 
wrote:
>Daniel Colascione <dancol@dancol.org> writes:
>
>> On December 27, 2024 9:59:14 AM EST, Eli Zaretskii <eliz@gnu.org> wrote:
>>>> Date: Fri, 27 Dec 2024 08:46:06 -0500
>>>> From: Daniel Colascione <dancol@dancol.org>
>>>> CC: rms@gnu.org, manphiz@gmail.com
>>>> 
>>>> >> It might take a while for that to happen, which is why I still believe
>>>> >> it would be better if tree-sitter major modes would populate
>>>> >> `treesit-language-source-alist' on their own, and point to the specific
>>>> >> checkouts that the major mode developer tested their implementation
>>>> >> against.
>>>> >
>>>> >We could have done that, but there's no way we could keep the value of
>>>> >treesit-language-source-alist up-to-date, because the grammar
>>>> >libraries put out new versions much more frequently than Emacs
>>>> >releases, especially if you consider libraries that have no official
>>>> >versions at all (in which case we can only point to some revision in
>>>> >their repository).
>>>> >
>>>> >The question that bothers me is how useful is it to have
>>>> >treesit-language-source-alist that is outdated?  What do we expect the
>>>> >users to do with such an outdated value?
>>>> >
>>>> 
>>>> Why not just vendor all the grammars with the Emacs modes that use them?
>>>
>>>We'd need to ask their developers to agree to this. 
>>
>> Why? They're free software. For copyright assignment? Seems like an 
>> exception would make sense here.
>>
>>> Other than that,
>>>I don't see how is that different from pointing to a specific version
>>>of each grammar: both will be outdated a short time after we point to
>>>the version or release Emacs with that version.
>>>
>>>So why do you think this is better?
>>
>> Vendoring enables building a full featured Emacs without a network 
>> connection and guarantees build reproducibility in perpetuity.
>
>Did you think of the long term consequences?
>
>The embedded dependencies would have to be maintained first by Emacs and
>later by packagers.
>
>All the infrastructure around syncing of grammars is time spend that
>could spend on more long term efforts such as stabilizing the
>tree-sitter based modes to not break as easy on grammar changes or to
>improve tree-sitter it self.

I've vendored plenty of things. Works fine in practice. Big programs like 
Firefox vendor the world too, and they work fine. It's really not that much 
work. It eliminates entire classes of problem. It's going to take more time to 
deal with the problems of taking a dependency and the headaches of not having a 
stable interface than it would to set up a few git subtrees or submodules and 
invoke their build system from that of Emacs.

It's not even the precise mechanics: pulling down a grammar by hash is 
tantamount to just checking in the grammar, but with more moving parts. You 
still pair one to one the grammar and the Lisp code meant to use it so you 
don't end up chasing down weird compatibility issues. IMHO, since they're 
tightly coupled anyway, we might as well distribute them together.

As for changing TS grammars not to break: why do you think that would be 
feasible? So far TS grammar authors haven't felt particularly obligated to 
maintain compatibility.