bug#59140: 29.0.50; iter-yield from lambda

[Stefan Monnier]

>>> IIRC, one of the main limitations of `generator.el` is that it doesn't
>>> handle `lambda` (and neither should you use `#'iter-yield`, IIRC).
>>> 
>>> I don't really know how to go about fixing it.
>>> 
>>> A good first step would be to make sure it emits an error (or a warning)
>>> when you use `#'iter-yield` or when you call `#'iter-yield` from with
>>> a lambda expression.
>
> Is this unfixable in principle or the fix is just hard (non-obvious)?

To do its job, `generator.el` performs a kind of CPS (continuation
passing style) conversion.  Such a conversion can be made to handle "the
whole language" fairly easily, but at the cost of being unable to
interact seamlessly with code which has not gone through the same
transformation.

So, yes, in principle it can be fixed *if* we make every single chunk of
ELisp code go through that transformation (plus adjust accordingly all
the primitives implemented in C).

Since that's not an option, `generator.el` uses a "local CPS conversion"
but these can't handle higher-order functions in general because they
need to be able to determine easily what is "local" and first-class
functions give you way to much power to play with the control flow.

So in practice it's not fixable.

> If iter-yield is lexically scoped within lambda expression, then could
> the lambda possibly be replaced with the iter-lambda?

Yes, if you can determine *all* the places from where this lambda can be
called and if you can then also change all the other lambdas that can be
called from one of those places, plus all the places where those other
lambdas can be called, plus all the other lambdas that can be called
from those other places, etc....  :-(


        Stefan