Re: Announcing 8sync: an asynchronous programming language for Guile

[Christopher Allan Webber]

Amirouche Boubekki writes:

>> Le 2015-12-04 03:47, Amirouche Boubekki a écrit :
>> 
>>>    Le 2015-12-04 03:15, Amirouche Boubekki a écrit :
>>> 
>>>        ```
>>>        (define (read/ sock)
>>>          (abort-to-prompt 'loop (lambda (cc)
>>>                                   (loop-add-reader sock (lambda () (cc 
>>> (read
>>>        sock)))))))
>>>        ```
>>> 
>> 
>>    This is a mistake, it *must* be a macro
>> 
>
> This will only help with debugging but not catching error and
> dealing with them at runtime ie. it doesn't propagate the exception
> but it can be done and requires a form similar to 
> propagate-%async-exceptions.
>
> TBH the code still seems complex maybe complicated. FWIW here is
> what I think right now. Be warned that everything should be taken
> with a grain of salt.
>
> * Agenda
>
> ** The canonical callback based event loop API is not visible enough
>
> It should be obvious coming from outside Guile world what/where
> is the event loop. As such, agenda doesn't seem like a good name.

"Agenda" is not a unique name for this.  That's what SICP uses, that's
what Sly uses.

> ** Agenda has both `schedule` and `queue`
>
> For a proof of concept, queue/schedule is not useful to demonstrate
> the purpose of eightsync as it's an optimization.

It's no mere optimization.  "schedule" is future events that haven't
been queued; it's only for time-delayed events.  The queue is for things
that must be done immediately.

This separation is very intentional.

> ** `(current-agenda-prompt)`
>
> No need to throw, there should always be a current agenda.

That's not true, if no agenda has been started, %current-agenda will be
#f, so there will be no way to retrieve the current agenda's prompt tag.

> * <schedule>
>
> Schedule should be in its own file. Also, in place of:
>
> ```
> (define-record-type <schedule>
>    (make-schedule-intern segments)
>    schedule?
>    (segments schedule-segments set-schedule-segments!))
> ```
>
> I prefer:
>
> ```
> (define-record-type <schedule>
>    (%make-schedule segments)
>    schedule?
>    (segments schedule-segments-ref schedule-segments-set!))
> ```

%make-schedule might be better.

> Use of `*-ref` and `*-set!`.

-ref is extra typing.  Doing just schedule-segments fits within
conventions shown elsewhere in the manual.

I don't really want to encourage mutation; getters are the default.

> Or better: `(define-record-type* <schedule> segments)` (actually I
> think this macro should be in Guile, since it much easier to
> introduce <records> to new devs).
>
> ** `time` procedures have equivalent in Guile

Yeah I might change that.

> *** `tdelta` is not useful.

tdelta is used so the the agenda knows to put it a future time from the
current execution of the agenda.

> * `<run-request>`
>
> ** `run-it`
>
> It's sound like it's a `(loop-call-soon callback #:optional (delay 0))`
> procedure.
>
> Instead it's a `(make-run-request proc #:optional time)`.
>
> It's never used.
>
> I'd rather to use a `delay`, and let the devs compute the correct delay,
> when they wants to run something at an absolute time to limit the
> proliferation of sugar procedure in the library.

You're right, it's never really used at present.  There is a difference
between (run (foo)) and (run-it foo) which isn't just the optional #:when,
which can indeed be handled by using run-at or run-delay instead.

Here's the difference: run uses (wrap) to do a friendly
wrap-in-a-thunk.  So you can do to this:


  (define (my-handler)
    (let ((foo (bar)))
       (run (mork foo))))

and behind the scenes, it constructs (lambda () (mork foo)) there,
preserving lexically scoped varibles.

If you already have a thunk though, you can just pass it in.  In such a
case you'd normally need to give another layer of indirection with
(run (my-thunk)).

It's an optimization that might not be necessary.  I might remove
it... I'm going to wait and see.

> *** `(wrap e ...)` and `(wrap-apply body)`
>
> I'm pretty sure that `wrap` can be written using `wrap-apply`. This
> looks suspicious for several reasons.

There's a difference between them:

 - (run) uses (wrap) to wrap all body contents in a thunk, preserving
   the illusion that everything looks the same as running code inline,
   as described above.  For most code, that's what you want.  (wrap)
   basically makes making inline thunks a little cleaner, and hands that
   off to other macros which make thunks too.

   This is nice because you can even do something like:

     (run (cond (foo bar)
                (baz basil)))

 - However, things that handle ports aren't thunks... they're callbacks
   that take an argument of whatever port it is.  Hence, making a thunk
   doesn't make sense.  But I still want an easy way to provide the
   clean appearance of inline code.  So (wrap-apply) passes along
   whatever arguments.

   You can't pass certain syntactic forms to (apply), for example cond
   above, so it's not as desirable generally.

> First it looks like a delayed call, so why not use force/delay.

It might be true that force/delay could be used.

> Also wrap-apply is never used.

It probably will be.  I have some local code that uses it.

I might be able to get rid of it.  I'll try.

> `wrap` is used with `(make-run-request proc time)` in `run`, `run-at`
> and `run-delay`, e.g.:
>
> ```
> (define-syntax-rule (%run-delay body ... delay-time)
>    (%run-at body ... (tdelta delay-time)))
> ```
>
> Again, `run` and others are building a datastructure not registering
> a callback so the naming is not good.

(run) and others are building a datastructure that contains information
to the agenda on how to register a callback, so they're doing both.

Rather than mutate the agenda on the spot, 8sync supports "passing back"
information about what callbacks should be done.

> * `(make-port-request port #:key read write except)`
>
> In `async.scm` I don't need to create a datastructure to subscribe to 
> select events.

Again, this is telling the agenda what to do for selecting on the next
iteration of the loop.

If you want multiple ports to be registered to be checked for
information, you need to let the agenda know how to handle all of them.

> ** `port-request` defined at L452 is never used

It will be, again, for permitting adding ports in a pseudo-functional
manner.

> * %8sync
>
> This is the main macro, here is it's definition:
>
> ```
> (define-syntax-rule (%8sync async-request)
>    (propagate-%async-exceptions
>     (abort-to-prompt (current-agenda-prompt) async-request)))
> ```
>
> I'm wondering whether (current-agenda-prompt) is useful.
> I think the code will abort to the prompt of the current dynamic
> context. So except if there is multiple agenda in the same thread,
> it's not useful.

You're right that this is the context in which this becomes useful.  I
might have composed agendas at some point.  I'm not sure.

> It's comparable to the way I define blocking procedures in async.scm
>
> ```
> (define-public (read/ sock)
>    (abort-to-prompt 'loop async-request))))
> ```

Right, that means that 'loop is always for "the agenda", and there will
always be one.

> where `async-request` is in `async.scm`:
>
> ```
> (lambda (cc) (loop-add-reader sock (lambda () (cc (read sock)))))
> ```
>
> It's the way `async.scm` does blocking calls (and *only* blocking
> calls).  `read/` and its `async-request` is missing catch around
> `read` and something like `propagate-%async-exceptions`.

8sync doesn't block though.  Its use of select means you only need to
get information once it's available.

The code you have here will block other code that is available to run
while it's waiting for code.

> 8sync has two types of async-request:
>
> ** run-requests, which implements kind of a *coroutine* behavior.
>
> It pause the execution of the current procedure and schedule
> the provided lambda to be run soonish; This doesn't exists in
> async.scm. The only thing useful this can do, is break the callstack
> to allow deeper recursion.

That's not true.  See the problems of "callback hell" people get into in
node.js.  8sync mitigates this.

As for the callback stack, it does get broken that's true... but that's
why 8sync gives you copies of all the stacks that were recursively
called as it walks back through its ~futures upon some exception.

> ** Last note
>
> This doesn't look nice:
>
> ```
>      (define (ports->procs ports port-map)
>        (lambda (initial-procs)
>          (fold
>           (lambda (port prev)
>             (cons (lambda ()
>                     ((hash-ref port-map port) port))
>                   prev))
>           initial-procs
>           ports)))
> ```
>
> then:
>
> ```
>         ((compose (ports->procs
>                    read-ports
>                    (agenda-read-port-map agenda))
>                   (ports->procs
>                    write-ports
>                    (agenda-write-port-map agenda))
>                   (ports->procs
>                    except-ports
>                    (agenda-except-port-map agenda)))
>          '()))))
> ```

This was to compress the code a bit.  It might be able to be done a bit
better.  I think it's not the worst though.

I realize I disagreed with a lot of what you said, but a lot of the
things you challenged are intentional designs in 8sync, not accidents.

I do appreciate the feedback though, it helped me clarify some of the
design decisions in 8sync, which will be useful for docs-writing!