Re: Update on tree-sitter structure navigation

[Dmitry Gutov]

On 07/09/2023 06:18, Danny Freeman wrote:
> Dmitry Gutov <dgutov@yandex.ru> writes:
>
> > > clojure-ts-mode keeps a URL for the parser, but doesn't do anything
> > > about the git revision. It easily could but I don't feel the need (yet)
> > > since I am also a maintainer of the clojure grammar and know when we're
> > > about to break grammar consumers.
> >
> > Sure, that's easy enough to do when the package is only in ELPA: upgrade the 
> > grammar, upgrade the
> > package, all in lockstep.
>
> Yeah, soon after I sent that email I realized there is no reason for me
> not to specify a version for the grammar so I pushed a change doing just
> that.

Nice.

> > Grammars distributed from distros are more of a problem, because it's not 
> > always a good idea to
> > abort with "wrong version". But perhaps we could do that and recommend 
> > installing from Git in such
> > cases anyway?
>
> In some cases, distros might place the grammars in a strange location
> made accessible on `treesit-extra-load-path`, which takes precedence
> over the grammars that are installed from git in the user's Emacs
> directory. This is what nix does, but is probably an outlier. I would
> guess more conventional distributions might just make them accessible
> where dynamic libraries are normally located and the grammars installed
> from git would take precedence.

Perhaps the user's Emacs directory should take precendence over 
treesit-extra-load-path. Or treesit-install-language-grammar should pick 
a higher-priority place instead. It just makes sense that the 
user-installed grammar would be loaded first.

> > Another problem is that grammars don't have good versioning, and even if they 
> > did, we'd have to
> > sometimes update the "upper bound" (we'd need coarse ranges, right? rather that 
> > one fixed version
> > requirement) more frequently than Emacs is released. Less of a problem for 
> > modes in ELPA, though.
>
> Yeah I think ranges would be right. It would be good to say, we tested
> this with versions N through M, anything else might not work. There
> would still need to be some checks and patches like what exists in
> js-ts-mode now. But that seems unavoidable, but could be cleaner if we
> had a good way to ID grammars. Not sure about how we'd keep up with
> grammars. Maybe we just can't and would need to have users install older
> versions. That seems okay?

Basically, yes: if the current available grammar is outside of the 
compatibility range (and/or we get query errors, I'm not sure where to 
put the balance: I suppose sometimes the query will succeed but it 
wouldn't match some elements which it matched before), we issue a 
warning to the user that they're recommended to use 
treesit-install-language-grammar - installing the last-known good hash, 
which might as well be older than the current installed grammar.

> > > I'm not so sure we can have a great way to do this without a change to
> > > the tree-sitter libraries. I would love to see some kind of increasing
> > > version number generated in the grammar's C source that we could then
> > > access. It could be used to make decisions about what queries to use, or
> > > to warn the user they need to use a different grammar (maybe offering to
> > > install a compatible version).
> >
> > Yes, that would be an improvement, worth being up on the issue tracker maybe.
>
> Yeah, I think this is a good move. I opened up one here
> https://github.com/tree-sitter/tree-sitter/issues/2611
> Of course, anyone feel free to chime in.

Thanks! I left a note too.