Re: Releasing the thread global_lock from the module API

[sbaugh]

Eli Zaretskii <eliz@gnu.org> writes:
>> From: Spencer Baugh <sbaugh@catern.com>
>> Date: Sat, 02 Mar 2024 20:33:05 +0000 (UTC)
>> Cc: sbaugh@janestreet.com, emacs-devel@gnu.org
>> 
>> >> So, the main difference between the "new thread" model and the "release
>> >> lock" model is that creating a native thread takes a nontrivial amount
>> >> of time; maybe around 0.1 milliseconds.  If some_native_function would
>> >> takes less time than that, the thread creation cost will slow down
>> >> Emacs, especially because the native module creates the native thread
>> >> while holding the Lisp global_lock.
>> >
>> > Why are you worried by 0.1 msec slowdown (if it indeed takes that
>> > long; I think it should be around 10 to 20 usec at most)?  If this
>> > kind of slowdown is important for you, you are using the wrong
>> > software package (and probably the wrong OS as well).
>> 
>> Because a Lisp program that uses a native module might make thousands of
>> module calls.  This is fine when each call takes a microsecond.  If we
>> add, for example, 500 microseconds of overhead to each module call, then
>> 1000 module calls will take half a second.
>> 
>> For example: I have a project.el backend which uses a native module.
>> Looking up the project for the current directory and then getting the
>> name of the project makes around 5 module calls.  I have around 200
>> projects.  That works out to 1000 module calls to get the names of all
>> my projects.  Currently with the native module backend this takes around
>> a millisecond.  With 500 extra microseconds per call, it will take half
>> a second.
>
> A module whose call takes this little time to complete and whose calls
> from Lisp are so massive should not, and need not, use this technique,
> because releasing the global lock for such a short time will not
> benefit anything.  Releasing a global lock is only beneficial if a
> module call does a lot of work, during which other Lisp threads could
> do something useful.  but if a module call takes a significant time
> (say, a few seconds), then adding a 20 usecs to that time is
> insignificant.

Yes, to avoid the thread creation overhead, a native module should only
create a native thread to run some_native_function if the native module
knows that some_native_function will take a long time.

Unfortunately, most of the time a native module can't know how long
some_native_function will take before calling it.  For example, native
functions which make network calls might return immediately when there's
already data available from the network, but will have to send new
network requests sometimes.  Or, native functions which provide an
in-memory database might be fast or slow depending on the database
contents.

Since the native module doesn't know if some_native_function will take a
long time, the native module needs a way to allow some_native_function
to run in parallel which is cheap, so the native module can do it for
all calls to some_native_function.

>> >> The "release lock" model fits this need.
>> >
>> > But it exposes the sensitive internals and runs the risk of more than
>> > one Lisp thread running at the same time, and thus is not acceptable.
>> 
>> Yes.  But of course in practice we would find a design which allows
>> releasing the lock but is hard to misuse.
>
> Sorry, I don't believe this is reliably possible, or safe enough to
> justify exposing the lock to Lisp.  Certainly not to modules, which is
> an open-ended gateway into the bowels of the Emacs Lisp machine.
>
>> How about this:
>> 
>>    native_output = env->call_without_lock(some_native_function, 
>> native_input);
>> 
>> call_without_lock would be a function which releases the global lock,
>> calls a specified function, then acquires the global lock again.
>
> Sorry, I don't agree to adding such interfaces to Emacs.  And if you
> are still not convinced, let's agree to disagree.

That is understandable, but I think you are not yet appreciating that
this can be a very useful way to introduce parallelism with high
performance.  And that env->call_without_lock is no less safe than the
method you suggested using.

Is there anything that would convince you of such things?