Re: Tree-sitter integration in python.el

[Yuan Fu]

> On Oct 8, 2022, at 1:03 AM, Augusto Stoffel <arstoffel@gmail.com> wrote:
> 
> On Fri,  7 Oct 2022 at 15:10, Yuan Fu wrote:
> 
>>> On Oct 7, 2022, at 3:03 AM, Augusto Stoffel <arstoffel@gmail.com> wrote:
>>> 
>>> On Fri,  7 Oct 2022 at 01:25, Yuan Fu wrote:
>>> 
>>>> Yeah, with tree-sitter, fortifying types is trivial. In fact all types
>>>> should be fortified already. (I tested with some simple examples.)
>>>> Should we provide some variables to toggle fontification for different
>>>> things? Like python-fontify-type/f-string/assignment/built-in/etc.
>>> 
>>> Looking at the screenshots posted a few messages back, which are VERY
>>> busy, I would really appreciate an option to disable a few fontification
>>> rules or, conversely, disable all but a few of them.  Ideally, this
>>> should be done through a generic mechanism that works across major
>>> modes.
>>> 
>>> Have you seen the new `font-lock-ignore' option?  Tree-sitter could
>>> provide something similar (and much better/less hacky).
>> 
>> The complaint for font-lock-maximum-decoration is that it’s obscure
>> and too corse-grained.
> 
> To me, the biggest problem with font-lock-maximum-decoration is that few
> major modes bothered to implement levels.

That’s their choice. If no one complains it’s not a problem ;-)

> 
>> So my idea is for each major mode to provide fined-grained controls
>> like python-fontify-type/f-string/assignment/built-in/etc.  And
>> tree-sitter makes it easy to implement this kind of toggle.
> 
> Given the lack of success of font-lock-maximum-decoration, I don't see
> this being implemented by many major modes.  Also, if the idea does take
> traction, it will lead to a proliferation of user options that is hard
> to use effectively -- if someone doesn't want to fontify built-ins in
> Python, they probably don't want it in other languages either, so they
> need to set a similar option for N languages.

It sounds nice, but (1) such generalization breaks down for 
not-c-like-general-programming-languages, like html, css, texinfo, etc, and (2) 
maybe one wants builtins in C but not in Python.

> 
>> But I guess a global control is also nice, I can make tree-sitter
>> respect font-lock-maximum-decoration, in addition to the fined grained
>> local-control.
>> 
>> Since we are designing a new system, I don’t think we need to resort
>> to the likes of font-lock-ignore.
> 
> It's exactly the opposite: since you are designing a new systems, you
> can create a much nicer customization mechanism on the lines of
> font-lock-ignore.  For instance, one could select fontification rules
> based on the affected node type.
> 
> The “decoration levels” feature can then build up on this, with the
> advantage that it would be consistent across languages and require no
> extra effort from the major mode developer.

It’s a nice idea, but in tree-sitter, different languages tag differently, it 
could be “function_definition” in python but “function_declaration” in C, etc. 
So it is not consistent across language. Also, it is often more complicated 
than a single tag like “function_identifier”, but rather a nested structure 
like (function_definition (identifier)).

Yuan