Re: Tree-sitter maturity

[Eli Zaretskii]

> Date: Sat, 04 Jan 2025 16:18:21 -0500
> From: Daniel Colascione <dancol@dancol.org>
> CC: owinebar@gmail.com, bjorn.bidar@thaodan.de, philipk@posteo.net,
>  emacs-devel@gnu.org, rms@gnu.org, manphiz@gmail.com
> 
> >> >It's _our_ win.  The problem is shifted to the distros, where I think
> >> >it belongs.
> >> 
> >> No, it isn't, because distros *can't* do a good job of this.
> >
> >Then they need to get their act together and improve.
> 
> They cannot and will not do major software engineering to work around 
> compatibility problems that should not exist. Do you expect a Fedora packager 
> to notice or care when the distro version of some grammar is subtly 
> incompatible with foo-ts-mode.el in Emacs? And patch the grammar or Emacs or 
> both? And for the Debian and Arch packagers and so to do that too? Probably 
> with subtly different bugs? We ought to have learned after the openssl 
> security debacle not to let distros make nontrivial code changes.

Yes, I expect them to do that.  AFAIK, they actually do this for
optional libraries they package and provide.

> >> Besides: plenty of people (I'd guess a majority going by how the industry 
> >> is set up) were using Emacs outside the context of a distro. If we weren't 
> >> allergic to telemetry, we'd know.
> >
> >People who do that (I'm one of them, btw) can build their grammars.  I
> >do.
> 
> Yeah it's totally normal to download a macOS or Windows package and then have 
> to set up a development tool chain to make the program they just installed 
> actually work. Nothing user hostile about that.

People who don't use distros build their own Emacs, yes?  So they
should already have this set up.  Compiling a grammar library requires
a C compiler and Binutils, plus Make, that's all.

> It's really hard to see how this "rely on the distro" model is supposed to 
> work in the real world. Besides, some of the antique platforms you insist we 
> support, like MS-DOS and Windows 98, have nothing resembling a package 
> manager, packages, or any way to automatically get some compatible version of 
> a grammar. Not including the batteries means the toy doesn't work.

I publish my 32-bit Windows builds of more than 70 grammar libraries
on the ezwinports site, so the users of those antique Windows 9X
platforms can download and install them.

> >> You said that we would need to get written permission from grammar 
> >> projects to include their code in Emacs. When I asked where this 
> >> requirement comes from, you said it had always been this way and that RMS 
> >> might have more information. He appears not to.
> >
> >RMS explicitly asked me to do that for every package we admit into
> >Emacs.
> 
> Is RMS a judge? The assertion is that there is a *legal requirement* to get 
> permission to include something in Emacs. I don't think any such requirement 
> exists. RMS may impose that requirement, but he's not the law, and unlike the 
> law, his policy can change.

I don't own this project, so when RMS asks me to do something, I do
that.

> >Again, you are missing the point: we need to find an appropriate
> >version for each branch, and we need to track the versions of the
> >grammar so ass to be able to know, for each of our branches, the
> >compatible version of the grammar.  The maintenance burden is
> >multiplied by the branches.
> 
> The way it should work is that you have a checked in foo-ts-mode.el and right 
> alongside it a foo.js file that describes the tree sitter grammar that 
> foo-ts-mode uses. When you cut a branch, you snapshot both files. No extra 
> overhead.
> 
> Want to update foo.js? Just sync it from upstream and check it in, just like 
> you'd check in any modified foo-ts-mode.el. You only have to do that when you 
> want to update the grammar, and that happens whenever the mode author feels 
> like doing it --- *exactly* the way it works today when you have a standalone 
> foo-mode.el.

This all is work someone has to do, and do that separately for each of
our two branches.

> >Responsibility has nothing to do with engineering or moving parts.
> 
> I have no idea what you're talking about. There's responsibility to make a 
> worse engineering choice and make a custom downloader? What's the nature of 
> this responsibility?

I think this is a clear sign that this discussion should be ended.

I stand by my opinions: it is not our job to distribute grammar
libraries, and bringing them into Emacs is too much trouble for us.