continuation passing in Emacs vs. JUST-THIS-ONE

[Thomas Koch]

TL;DR: (Why) is there no standard way for continuation passing style[1] ("event 
driven") programming in Emacs?

During the investigation of an Emacs freeze[1] while starting eglot over tramp, 
we made a couple of observations. It was proposed to me to share these 
observations with you. I don't know elips or Emacs internals though, so 
apologies for any errors.

[1] https://debbugs.gnu.org/61350

Bug #61350 happens because Tramp calls accept-process-output with JUST-THIS-ONE 
set to t. Tramp does this since Bug #12145[2]. However it seems, that this bug 
should have been rather fixed in function `find-dired` instead. (See following 
separate email on this.)

[2] https://debbugs.gnu.org/12145

The JUST-THIS-ONE argument was introduced with this entry in etc/NEWS.22, 
emphasis from me:

"""
*** Function 'accept-process-output' has a new optional fourth arg
JUST-THIS-ONE.  If non-nil, only output from the specified process
is handled, suspending output from other processes.  If value is an
integer, also inhibit running timers.  THIS FEATURE IS GENERALLY NOT
RECOMMENDED, but may be necessary for specific applications, such as
speech synthesis.
"""

The argument was discussed here:
https://lists.gnu.org/archive/html/emacs-devel/2004-08/msg00141.html

and introduced in this commit:
https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=107ed38d4bdec03002b2a23619e205722cd5b8d1

I don't even think that the original motivation for introducing JUST-THIS-ONE 
was valid. Unfortunately there was not much discussion about it. It was argued, 
that it would be hard to make a process filter function reentrant. And I think 
that this was an invalid root cause analysis to start with.

First, the emacs manual says[3]: "Note that if any of those functions are 
called by the filter, the filter may be called recursively." - So one should 
make the filter reentrant, If I understand correctly.

[3] 
https://www.gnu.org/software/emacs/manual/html_node/elisp/Filter-Functions.html

Second, the manual further says: "Quitting is normally inhibited within a 
filter function". This indicates to me, that a filter function should be 
(mostly) "side effect free" besides putting its input somewhere (e.g. in a 
buffer or message queue) and trigger an event if there is enough input for 
further processing. This also reduces the risk, that the function could be 
called recursively in a damaging way.

It seems to me, that there is not yet a standard way in Emacs for continuations 
(or event driven programming) although the Emacs Wiki refers to the 
emacs-deferred library: https://www.emacswiki.org/emacs/ConcurrentEmacs

Because there is no such library in Emacs, people either write their own code 
for continuations (eglot?) or do too much work in a process filter function 
(speechd-el in 2004 which led to JUST-THIS-ONE).

While I don't know elisp, I unfortunately had to do JavaScript. Like Emacs, JS 
is single-threaded. While I share the sentiment about JS, there are still 
things to learn from it, e.g. event driven programming.

See also:

- 2011 emacs-dev discussion: 
https://lists.gnu.org/archive/html/emacs-devel/2011-05/msg00575.html
- 2016 Blogpost https://jyp.github.io/posts/elisp-cps.html
- https://stable.melpa.org/#/deferred
- 
https://www.gnu.org/software/emacs/manual///html_node/elisp/Transaction-Queues.html
- Maybe: https://elpa.gnu.org/packages/fsm.html