bug#60894: 30.0.50; [PATCH] Add treesit-forward-sexp

[Mickey Petersen]

Theodor Thornhill <theo@thornhill.no> writes:

> Hi Emacs (also Juri and Mickey as you've expressed some interest for this)
>
> This is an example patch for sexp movement with tree sitter.  I want to
> put it out here to hopefully produce some discussion, sooner rather than
> later.
>
> Three initial questions:
>
> 1. What should a sexp be?
>

`forward-sexp' and friends behave exactly as they should behave. If
there are sexp-like things on the left and/or right-hand side of
point, notwithstanding certain punction in the direction of travel,
then it uses that to determine where to move; where to kill to; and
the boundaries for transposition.

Absent an sexp, it defaults to word-based movement. Exactly as it should.

With that out of the way. That does not mean that we cannot extend its
behaviour in ways that are both predictable but also useful.

It should know about things the default `syntax-ppss' machinery cannot
figure out for itself.

Consider HTML (or anything in SGML club of languages). I would expect
sexp movement to move over a matched pair of tags. It currently does
not; the reason why is understandable when you know how `syntax-ppss'
does (or does not, as it were) work. (You can equally make an argument
that it should simply go to the end of an opening/closing node and not
the pair, but that is personal preference.)

`nxml-mode' handles it properly; Combobulate handles it properly, too.
But Combobulate also falls back to the classic sexp behaviour if it
cannot find a suitable node in the direction of travel.

And that is the key thing here. It augments; it does not replace.


>   Is it basically "everything", or is there a distincition between
>   "word", "sexp" and "sentence"?  For lisp forward-sexp looks like a
>   "jump over words, or a balanced pair of parens".  In other languages
>   that can look a little weird - consider:
>

Yes, there are distinctions, and no, sexp-based movement is not the
only game in town. Certainly not pre-TS; and definitely not after.

Word movement is a subset of sexp-based movement. Sentence and
paragraph-based movement is rarely altered in coding major modes. But M-{
and M-} are still useful for moving through contiguous blocks of code
separated by blank lines. Not its intended use, but absent something
better, it's OK. Room for improvement for sure.


>   ```
>   foo().|bar().baz(); -> foo().bar|().baz(); -> foo().bar()|.baz();
>   ```
>

Subject to the vagaries of the syntax table, yes, that's right. It's
also useful. I can punch `C-M-k' twice and gobble up the caller and
whatever is in the brackets. That is not possible with word-based
killing alone.

>   In a sense it could be considered "better", or at least distinct from
>   forward-word to:
>
>   ```
>   foo().|bar().baz(); -> foo().bar()|.baz(); -> foo().bar().baz()|;
>   ```
>

That is not how word movement works on my machine:

    foo().bar().baz();
          |  |     |

That is the path of travel for `M-f' in C-mode. This is how `C-M-f'
moves:

    foo().bar().baz();
          |  | |   |

> 2. Should this new function be leveraged in transpose-sexps?
>
>   IMO if the forward-sexp gets too close to forward-word, or
>   forward-sentence we lose some nice properties with the current
>   'treesit-transpose-sexps', namely (among others):
>
>   ```
>   f(String foo,| Integer bar) ->  void foo(Integer bar, String foo|)
>   ```
>
>   I know you Mickey have expressed some dissatisfaction with the current
>   implementation - now is a good time to make some worthwhile
>   improvements.

My dissatisfaction was merely the approach. You cannot transpose
sibling nodes and expect that to behave in a manner that is logical to
anyone who is not fluent in the specifics of every TS grammar they
interact with.

Things that appear to be 'siblings' to the human eye may in fact not
be that at all.

And some grammars are rather more generous with their level of
granularity than others: both in how expansive they are, but also in
how deep the trees are. So when you go foraging for nodes to
transpose, you may end up finding sub-nodes to the ones you'd actually
want to transpose because they are smaller and closer units.

>
> 3. What are the "rules"?
>
>   In c-mode, elisp-mode without paredit forward-sexp won't jump out of
>   the current scope, however with paredit enabled it will.
>
>
> If we simply want some code similar to this to live and slowly evolve I
> guess we can install something like this patch after some tweaks and
> iterate when we have more experience.
>

The problem with that patch is -- and caveat, I do not use Java, so I
am not too versed in the details of the note types used -- but most of
the things I see are things that a normal sexp would work with anyway:

- true, false: they are just words, right?
- identifier: presumably a symbol or something that a decent syntax
  table would probably catch.
- parenthesized_expression: delimited by ( ), etc. ? Sexp handles that
- just fine.
- argument_list: presumably uses ( ) also


> Anyway, I hope these questions and thoughs will spark some discussion,
>
> Theo
>
>
> [2. text/x-patch; 0001-Add-treesit-forward-sexp.patch]...