Re: Contributing to Emacs

[Stefan Monnier]

>  I was looking at the file etc/TODO in the repo, and some items there
> caught my attention. For example, under 'Important Features'  I found 'Add
> an "indirect goto" byte-code'. Is this still a desired feature or is it not
> required due to recent efforts towards native compilation? (In the negative
> case, please feel free to suggest another item for me to work on)

I think I was the one to add this point to etc/TODO and AFAIK the need
is pretty much still the same, yes.  Part of the idea is to handle more
efficiently code which uses local lambda expressions (like `cl-flet`).

There's a lot of scope here.  Some steps are easy but won't bring any
(or very little) benefit, which others depend on those first steps and
require much more extensive work.

> If it is still desired I would like to work on it and would appreciate
> guidance. After taking a look at lisp/emacs-lisp/bytecomp.el,it seems that
> forms such as
>
>  (let ((foo (lambda (x) bar)))
>      (dosomething
>        (funcall foo toto)
>        (blabla (funcall foo titi))))
>
> are compiled by calling byte-compile-let. Before generating the binding for
> foo, #'(lambda (x) bar) is compiled by calling
> byte-compile-push-binding-init, which in turn compiles the lambda to
> bytecode and pushes it to the stack.
>
> So to make it more concrete for my sake, I wrote the following form
>
> (let ((foo (lambda (x) (- x 1))))
>   (* (funcall foo 2)
>      (- (funcall foo 3) 2)))

Indeed, I think it might be fairly easy to handle this code more
efficiently.  Things become more ... interesting when we have code like:

    (let* ((count 0)
           (foo (lambda (x) (setq count (1+ count)) (+ x count))))
      (* (funcall foo 2)
         (- (funcall foo 3) 2)))

Where `cconv.el` turns the `count` into a one-element list and
replaces the `setq` with a `setcar`, which wouldn't be needed if we
could directly refer to the `count` var from the inner lambda using
those new byte codes.

Even more interesting would be

    (let* ((count 0)
           (foo (lambda (x) (setq count (1+ count)) (+ x count))))
      (let ((bar (funcall foo 2)))
         (- (funcall foo 3) bar)))

where the relative position of `count` on the stack is different in the
two calls to `foo`.

> and with disassemble inspected the generated bytecode by Emacs 28.1
>
> byte code:
>   args: nil
> 0       constant  <compiled-function>
>       args: (x)
>     0       varref    x
>     1       sub1
>     2       return
>
> 1       dup
> 2       varbind   foo
> 3       constant  2
> 4       call      1
> 5       varref    foo
> 6       constant  3
> 7       call      1
> 8       constant  2
> 9       diff
> 10      mult
> 11      unbind    1
> 12      return
>
> As far as I understood,  the idea of the "indirect goto" here is to
> eliminate the calls on lines 4 and 7 and replace them with an indirect
> goto.

The ones at 4 and 7 can stay as "normal" gotos but they need to pass to
the function an additional argument which is the return address, and
then the inner function would be the one which uses an indirect goto
instead of "return".

> stack* in line 0 (constant <compiled-function>). Thus, for the indirect
> goto idea to work, the bytecode of the lambda would need to be part of the
> bytecode of the let's body**. That way, a computed branch can branch into
> the lambdas code and it is possible to branch out of it when returning.

I guess in theory there could be cases where we could use a computed
goto to jump into a function, but that needs probably too many stars to
be aligned to be worth considering.  The indirect goto is needed for
"return", OTOH.

> Aside from guidance on implementing this, I would appreciate any
> pointers on how to adequately test it.

I don't think there's currently many ways to test bytecode level
operation, other than writing normal ELisp code which we expect will use
those bytecodes and make sure they behave as expected.


        Stefan