Re: [Chicken-users] Re: An alternative thread system?

[Shawn Rutledge]

I'm inclined to agree with the argument that as long as forking has a
low enough overhead, processes are better than threads for
general-purpose parallelism.  Especially considering that if you are
using memory to share state between threads, you will always be
limited to a shared-memory architecture - you cannot move the program
unmodified onto a cluster.  But there is more data-copying involved
with message-passing, compared to just accessing the same memory.
Safety and portability have their price.

However what about the case where there is a UI event loop and the
same program also has to poll one or more event sources, such as dbus,
or a socket or a pipe, for output from one of those helper processes
that you redesigned to be a process rather than a thread?  In C your
main choices are to use select() to wait for an event from any of
those sources (only as long as your program has nothing else to do
while it's waiting, and as long as you can gather all the file
descriptors into one list to be able to use select), or use
non-blocking I/O and poll each source manually, or use threads to
pretend that you can poll everything in parallel.

In the dbus egg I use a polling thread (a Chicken thread not a native
one).  This hides the complexity nicely: the user of the egg does not
need to care about polling, it appears instead that the appropriate
callback function can be called spontaneously when a DBus event
occurs.  If I wish to preserve that interface, do I really have any
choice other than threads?

Maybe Chicken could have a higher level abstraction for callbacks.  It
would then have the choice of using either select() or threads under
the covers.

A long time ago (when working at my first software job, which happened
to be on OS/2) it struck me that it's too bad events which originate
from hardware often have to be polled in software.  E.g. a UART
generates an interrupt when a character is received; the OS handles it
and puts the character in a buffer, and the application must poll the
buffer to see when there is data available.  Maybe the OS could tell
the interested application each time a character is received (and
still do the buffering, in case the app doesn't want to handle each
character one-at-a-time).  Of course it works well enough, it's just a
philosophical thing.  (And maybe in fact when using select(), maybe
breaking out of this blocking call is in fact triggered directly by
the interrupt?)  So I guess I'm thinking along the same lines about
callbacks in general: "tell me when X happens" should not have to mean
"waste a bunch of cycles polling until X happens and then call me."
But as long as polling threads are the only practical way to do
callbacks, there is no chance to even try to do an optimization like
that.