Re: Asynchronous event loop brainstorm at FSF 30

[Chris Vine]

On Sat, 03 Oct 2015 17:29:16 -0500
Christopher Allan Webber <address@hidden> wrote:

> So David Thompson, Mark Weaver, Andrew Engelbrecht and I sat down to
> talk over how we might go about an asynchronous event loop in Guile
> that might be fairly extensible.  Here are some of what we discussed,
> in bullet points:
> 
>  - General idea is to do something coroutine based.
> 
>  - This would be like asyncio or node.js, asynchronous but *not* OS
>    thread based (it's too much work to make much of Guile fit around
>    that for now)
> 
>  - If you really need to maximize your multiple cores, you can do
>    multiple processes with message passing anyway
> 
>  - Initially, this would probably providing a general API for
>    coroutines.  Mark thinks delimited continuations would not be as
>    efficient as he'd like, but we think it's okay because we could
>    provide a nice abstraction where maybe something nicer could be
>    swapped out later, so delimited continuations could at least be a
>    starting point.
> 
>  - So what we really need is a nice API for how to do coroutines,
> write asynchronous code, and work with some event loop with a
> scheduler
> 
>  - On top of this, "fancier" high level systems like an actor model or
>    something like Sly's functional reactive programming system could
> be done.
> 
>  - Probably a good way to start on this would be to use libuv (or is
> it libev?) and prototype this.  It's not clear if that's a good long
>    term approach (eg, I think it doesn't work on the HURD for those
> who care about that, and Guix certainly does)
> 
>  - Binary/custom ports could be a nice means of abstraction for this
> 
> So, that's our thoughts, maybe someone or one of us or maybe even
> *you* will be inspired to start from here?
> 
> To invoke Ludo, WDYT?
>  - Not Ludo

It is certainly the case that mixing threads with coroutines is usually
best avoided, otherwise it becomes very difficult to know what code
ends up running in which particular thread and thread safety becomes a
nightmare.  However, it would be good to allow a worker thread to post
an event to the event loop safely, whereby the handler for the posted
event would run in the event loop thread.  asyncio allows this.

Although not particularly pertinent to this proposal, which looks
great, I use coroutines implemented with guile's delimited
continuations for a minimalist "await" wrapper over glib's event loop
as provided by guile-gnome (the whole thing is about 20 lines of code),
which appears (to the user) to serialize the GUI or other events posted
to the event loop.  When I don't want to use guile-gnome, which is most
of the time, I have my own (also minimalist) thread-safe event loop
using guile's POSIX wrapper for select.

My uses of guile are pretty undemanding so as I say these are
minimalist.  Something like asyncio for guile would be very nice indeed.

Chris