Re: continuation passing in Emacs vs. JUST-THIS-ONE

[Tomas Hlavaty]

Hi Stefan,

On Wed 15 Mar 2023 at 13:48, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> I seem to recall that Stefan Monnier (CCed) mentioned having some WIP
>> code to make generator.el easier to use for asynchronous code...
> I think my WiP thingy is very similar to emacs-aio.  I haven't had
> time to work on it and I'd welcome help with it (attached).

Interesting.

>From futur.el:

> ;; (futur-let*
> ;;   (exitcode <- (futur-process-make :command cmd :buffer t))
> ;;   (out (buffer-string)) ;; Get the process's output.
> ;;   (cmd2 (build-second-arg-list exitcode out))
> ;;   (otherexit <- (futur-process-make :command cmd :buffer t)))
> ;;  (futur-pure (buffer-string)))

Seems like beautiful lisp code has no futur. :-)

There is something very ugly about this code.
It looks like assembly, 1 dimensional vertical code.
It is hard to see the structure of the code and what it actually does.
I do not think it is practical to write non-trivial code in this style.

Nice lisp code is usually 2 dimensional,
with indentation and top-left to bottom-right direction.
It is usually much clearer to see what is an argument to what
based on the position in the syntax tree.

Is it possible to make the syntax more structured (lispy)?
Meaning tree-like, not list-like?
Something in the spirit of:

(futur-progn
 (futur-process-make
  :command (futur-let ((exitcode (futur-process-make
                                  :command (build-arg-list)
                                  :buffer t)))
             (build-second-arg-list exitcode (buffer-string)))
  :buffer t)
 (buffer-string))

or would it need some fancy syntax rewriting like other async/cps
syntax rewriting libraries?


Second question: I see that futur-wait blocks the whole emacs due to
the while loop.  How can one use futur without blocking emacs?


I usually prefer pull based code as it does not steal control from me.
Lets say I want to do something nontrivial but not block emacs.  I would
split computation into chunks, identify state explicitly and move it to
the heap and suspend the computation without needing to reserve a stack
for it.  I.e. manually write a kind of stream that yields items or nil
as EOF (without syntax rewriting ala generators.el).  I need EAGAIN for
stuff happening asynchronously.  Stuff that blocks simply needs to be
such a small chunk that it does not negatively affect emacs useability.

Unfortunately futur.el does not have executable example so I'll invent
one.

Example (requires lexical bindings): Traverse filesystem, find
*.el files and do something for each one (here I just count the length
of the absolute path for simplicity).  And the whole thing should not
block emacs.

(defun stream-pull-in-background (stream &optional secs repeat)
  (let (timer)
    (setq timer (run-with-timer
                 (or secs 1)
                 (or repeat 1)
                 (lambda ()
                   ;;(message "@@@ polling!")
                   (unless (funcall stream)
                     (cancel-timer timer)))))))

(defun line-stream (buffer)
  ;; yield buffer lines, follow process output if any
  (let (start)
    (lambda ()
      (with-current-buffer buffer
        (save-excursion
          (unless start
            (setq start (point-min)))
          (goto-char start)
          (let ((end (line-beginning-position 2)))
            ;;(message "@@@ %s %s" start end)
            (if (< start end)
                (prog1 (buffer-substring-no-properties
                        start
                        (line-end-position 1))
                  (setq start end))
              (let ((process (get-buffer-process buffer)))
                (if (and process (process-live-p process))
                    'EAGAIN
                  (let ((end (point-max)))
                    (when (< start end)
                      (prog1 (buffer-substring-no-properties start end)
                        (setq start end)))))))))))))

(defun burst-stream (stream &optional secs)
  ;; pull available data during SECS time window
  ;; this is very crude "scheduler" but keeps emacs mostly useable
  (let ((secs (or secs 0.2)))
    (lambda ()
      (when secs
        (let ((z 'EAGAIN)
              (end (+ secs (float-time (current-time)))))
          ;;(message "@@@ burst %s %s:" (float-time (current-time)) end)
          (while (and (< (float-time (current-time)) end)
                      (setq z (funcall stream))
                      (not (eq 'EAGAIN z))))
          (unless z (setq secs nil))
          z)))))

(defun message2-stream (stream)
  (lambda ()
    (let ((x (funcall stream)))
      (when x
        (unless (eq 'EAGAIN x)
          (message "@@@ %s %s" (length x) x))
        x))))

(defun test-buffer (name)
  (let ((b (get-buffer-create name)))
    (with-current-buffer b
      (buffer-disable-undo)
      (erase-buffer))
    b))

(defun test3 (buffer-name command)
  (stream-pull-in-background
   (let ((b (test-buffer buffer-name)))
     (make-process :name buffer-name
                   :command command
                   :buffer b)
     (burst-stream (message2-stream (line-stream b))))))

;;(test3 "test3" '("cat" "/tmp/a.el"))
;;(test3 "test3" '("find" "/home/tomas/mr/" "-type" "f" "-name" "*.el"))
;;(list-processes)
;;(list-timers)


Last question: How would similar functionality be implemented using
futur?

Cheers

Tomas