Re: SMIE documentation

[Stefan Monnier]

> First, here's a patch to make smie-next-sexp correctly return (t POS
> TOKEN) when bumping into a closing thingy in the forward direction.

> --- a/lisp/emacs-lisp/smie.el
> +++ b/lisp/emacs-lisp/smie.el
> @@ -653,7 +653,8 @@ Possible return values:
>                  (if (and halfsexp (numberp (funcall op-forw toklevels)))
>                      (push toklevels levels)
>                    (throw 'return
> -                         (prog1 (list (or (car toklevels) t) (point) token)
> +                         (prog1 (list (or (funcall op-forw toklevels) t)
> +                                      (point) token)
>                             (goto-char pos)))))
>                 (t
>                  (let ((lastlevels levels))

I do not understand what case this intends to solve.  When I try to
"bump into a closing thingy in the forward direction" with the current
code, it seems to work correctly.

> There's a problem with the indentation when there's comment at the start
> of a line before a keyword, like

>     (* Comment *) IF ...                      <-- Press TAB

> The virtual indent of the following keyword is computed as its current
> column. The comment indentation function tries to align the comment with
> this column, making the line wander farther and farther to the right for
> every press of the TAB key.

Good catch, thanks.  Don't have a fix yet, sadly.

>> +Calling this function is sufficient to make commands such as
>> address@hidden, @code{backward-sexp}, and @code{transpose-sexps}
>> +be able to properly handle structural elements other than just the paired
>> +parentheses already handled by syntax tables.  E.g. if the provided
>> +grammar is precise enough, @code{transpose-sexps} can correctly
>> +transpose the two arguments of a @code{+} operator, taking into account
>> +the precedence rules of the language.

> This makes C-M-f and friends behave very differently from most other
> major modes, which doesn't really feel right.

I think it doesn't feel right because you're not yet used to it.
What it does is make C-M-[fbt] behave similar to how they behave in
Lisp languages.  It's very intuitive when skipping over begin...end.
It's admittedly less obviously intuitive for infix/mixfix constructs.

>> +To describe the lexing rules of your language to SMIE, you will need
>> +2 functions, one to fetch the next token, and another to fetch the
>> +previous token.  Those functions will usually first skip whitespace and
>> +comments and then look at the next chunk of text to see if it
>> +is a special token, if so it should skip it and return a description of
>> +this token.  Usually this is simply the string extracted from the
>> +buffer, but this is not necessarily the case.
> It would be good if users could hook their own functions into all places
> that extract text from the buffer ("buffer-substring"), and not just
> smie-forward/backward-token-function; e.g. to use interned token strings
> or to handle some kind of banana brackets using the syntax table.

I do not understand what you're referring to.

>> address@hidden:elem}, in which case the function should return either the 
>> offset
>> +to use to indent function arguments (if @var{arg} is the symbol
>> address@hidden)
> Either there's a bug in the code, or this should be the symbol "args"
> and likewise for the doc string.

I meant @code{args}, indeed.

>> address@hidden smie-rule-parent &optional offset
>> +Return the proper offset to align with the parent.
>> +If address@hidden, @var{offset} should be an integer giving an
>> +additional offset to apply.
>> address@hidden defun
> The function returns an absolute column, not an offset.

It does not return an absolute column (which would be a number) but
a cons cell (column . N), i.e. it returns an "offset" in the sense of:

   @var{offset} can be:
   @itemize
   @item
   @code{nil}: use the default indentation rule.
   @item
   @code{(column . @var{column})}: indent to column @var{column}.
   @item
   @var{number}: offset by @var{number}, relative to a base token which is
   the current token for @code{:after} and its parent for @code{:before}.
   @end itemize

>> +    (:before
>> +     (cond
>> +      ((equal token ",") (smie-rule-separator kind))
>> +      ((member token '("begin" "(" "@{"))
>> +       (if (smie-rule-hanging-p) (smie-rule-parent)))

> Does this really work for "("?

It works in those cases where I've used it, yes.

> Most of the time smie-indent--parent seems to returns nil before
> a paren, which breaks smie-rule-parent.

That would be a bug, indeed.  Please post a test case to reproduce it.


        Stefan