Re: Blocking calls and threads

[Eli Zaretskii]

> From: Lynn Winebarger <owinebar@gmail.com>
> Date: Thu, 20 Apr 2023 10:19:11 -0400
> Cc: emacs-devel@gnu.org
> 
> > If you yield before issuing the system call, the system call will wait
> > until you re-acquire the lock.  So how will this help?
> 
> You're talking about yielding the system thread, I'm talking about
> yielding the Lisp machine thread.

No, I'm also talking about the Lisp machine thread.  The thread which
calls insert-file-contents and runs the C code of insert-file-contents
and of the subroutines called by insert-file-contents.

> Even though Lisp machine threads are implemented by mapping them to
> the underlying system thread, the Lisp machine execution state is
> kept coherent by the global (interpreter) lock.  Releasing the lock
> is effectively yielding the Lisp machine thread.  The system thread
> will yield implicitly if the read blocks.

What you say here is not relevant to the issue at hand.

> the read operation should either use some temporary buffer and copy
> into the lisp buffer, or wait for data to be ready, then reacquire
> the GIL before invoking the read syscall with a pointer into the
> Lisp buffer object.

Yes, and that's exactly where we will lose: most of the heavy
processing cannot be done in a separate temporary buffer, because it
calls functions from the Lisp machine, and those are written assuming
nothing else is running in the Lisp machine concurrently.  For
example, take the code which decodes the file's contents we have just
read.  I encourage you to take a good look at that code (most of it is
in coding.c) to appreciate the magnitude of the problem.

So the code which can run in parallel with another Lisp thread will be
able to do only very simple jobs, and will also add overhead due to
the need of copying stuff from temporary buffers to Lisp objects.

Of course, we could redesign and reimplement this stuff, but that's a
lot of non-trivial work.  My assumption was that you are considering
relatively lightweight changes on top of the existing code, not a
complete redesign of how these primitives work.