Re: Tree-sitter maturity

[Daniel Colascione]

On January 4, 2025 3:57:48 PM EST, Eli Zaretskii <eliz@gnu.org> wrote:
>> Date: Sat, 04 Jan 2025 15:46:49 -0500
>> From: Daniel Colascione <dancol@dancol.org>
>> CC: owinebar@gmail.com, bjorn.bidar@thaodan.de, philipk@posteo.net,
>>  emacs-devel@gnu.org, rms@gnu.org, manphiz@gmail.com
>> 
>> >> And it creates the need to do code distribution in a bespoke way. How is 
>> >> that a net win?
>> >
>> >It's _our_ win.  The problem is shifted to the distros, where I think
>> >it belongs.
>> 
>> No, it isn't, because distros *can't* do a good job of this.
>
>Then they need to get their act together and improve.

They cannot and will not do major software engineering to work around 
compatibility problems that should not exist. Do you expect a Fedora packager 
to notice or care when the distro version of some grammar is subtly 
incompatible with foo-ts-mode.el in Emacs? And patch the grammar or Emacs or 
both? And for the Debian and Arch packagers and so to do that too? Probably 
with subtly different bugs? We ought to have learned after the openssl security 
debacle not to let distros make nontrivial code changes.

>> Besides: plenty of people (I'd guess a majority going by how the industry is 
>> set up) were using Emacs outside the context of a distro. If we weren't 
>> allergic to telemetry, we'd know.
>
>People who do that (I'm one of them, btw) can build their grammars.  I
>do.

Yeah it's totally normal to download a macOS or Windows package and then have 
to set up a development tool chain to make the program they just installed 
actually work. Nothing user hostile about that.

It's really hard to see how this "rely on the distro" model is supposed to work 
in the real world. Besides, some of the antique platforms you insist we 
support, like MS-DOS and Windows 98, have nothing resembling a package manager, 
packages, or any way to automatically get some compatible version of a grammar. 
Not including the batteries means the toy doesn't work.

>
>> >> Nobody has been able to describe these legal aspects. Grammars are free 
>> >> software. GPL compatible, too.
>> >> That means we can put them in Emacs. That's what software freedom means.
>> >
>> >I did explain that, and don't want to repeat.  You can consider those
>> >reasons unimportant, but I don't (and cannot, really: I don't call the
>> >shots in this particular game).
>> 
>> No, you didn't explain. You asserted.
>> 
>> You said that we would need to get written permission from grammar projects 
>> to include their code in Emacs. When I asked where this requirement comes 
>> from, you said it had always been this way and that RMS might have more 
>> information. He appears not to.
>
>RMS explicitly asked me to do that for every package we admit into
>Emacs.

Is RMS a judge? The assertion is that there is a *legal requirement* to get 
permission to include something in Emacs. I don't think any such requirement 
exists. RMS may impose that requirement, but he's not the law, and unlike the 
law, his policy can change.

>> >You are missing the point.  My point is that several branches means we
>> >need to match a different version of the library to each branch
>> 
>> If you check the grammar into the tree, the version in each branch matches 
>> that branch automatically. That's what a version control system is. It's the 
>> essence of what it does. Same goes for submodules, which are just checked-in 
>> pointers.
>
>Again, you are missing the point: we need to find an appropriate
>version for each branch, and we need to track the versions of the
>grammar so ass to be able to know, for each of our branches, the
>compatible version of the grammar.  The maintenance burden is
>multiplied by the branches.

The way it should work is that you have a checked in foo-ts-mode.el and right 
alongside it a foo.js file that describes the tree sitter grammar that 
foo-ts-mode uses. When you cut a branch, you snapshot both files. No extra 
overhead.

Want to update foo.js? Just sync it from upstream and check it in, just like 
you'd check in any modified foo-ts-mode.el. You only have to do that when you 
want to update the grammar, and that happens whenever the mode author feels 
like doing it --- *exactly* the way it works today when you have a standalone 
foo-mode.el.

>> Can you give me a concrete example of how checking in code burdens branch 
>> maintenance? Your position isn't making any sense to me in context of how 
>> version control systems work.
>
>Finding the matching version and testing whether it matches does.

You don't do that when you cut a branch. You do that when you update the mode 
or its grammar, and no matter how you associate modes with their grammars, you 
have to test the combination.

>> >> >The difference is that the RI changes.  And that's not something to
>> >> >ignore, from where I stand.
>> >> 
>> >> Huh? In what possible way could a bespoke downloader be a better 
>> >> engineering choice than submodules?
>> >
>> >I didn't say anything about engineering, I was talking about the
>> >responsibility.
>> 
>> Good engineering minimizes the number of moving parts and needless failure 
>> points in a system.
>
>Responsibility has nothing to do with engineering or moving parts.

I have no idea what you're talking about. There's responsibility to make a 
worse engineering choice and make a custom downloader? What's the nature of 
this responsibility?