Re: Concurrency via isolated process/thread

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

> That is of course not always possible, or even desirable.  For
> example, some prompts are caused by specific aspects of the
> processing, which may or may not happen, depending on the data.
> Prompting for options unrelated to the actual processing would mean
> annoying users unnecessarily.  Etc. etc.

In that case, the thread will have to suspend itself until the main
thread can read a response from the user.

> So basically, what you have in mind is a way of accruing a body of
> special-purpose Lisp programs written specifically for running from
> non-main threads.  Which means reusing existing code or packages not
> written to these specifications will be impossible, and we will in
> effect have two completely separate flavors of Emacs Lisp programs.
> It would mean, in particular, that many functions in simple.el,
> subr.el, and other similar infrastructure packages will need to have
> specialized variants suitable for running in non-main threads.

Yes.

> And all this will be possible if -- and it's a large "if" -- the
> necessary support on the C level will be written and prove reliable.
>
> If this is the plan, it might be possible, at least in principle, but
> is it really what is on people's mind when they dream about "more
> concurrent Emacs"?  I doubt that.

I don't know what other people think, but it's what I would find
desirable, as my gripe with tools like Semantic is that they cannot run
expensive text crunching tasks in the background.  Having shared-memory
multiprocessing would allow these tasks to be implemented efficiently.

> Memory is cheap these days, and "slow IO" is still a gain when it
> allows us to use more than a single CPU execution unit at the same
> time.  So yes, efficiency is desirable, but ease of use is also
> important.  What's more, I don't think anyone really wants to have to
> write Lisp programs in a completely different way when we want them to
> run from threads.

When programmers write such code for other interactive programs, they
are comfortable with the limitations of running code outside of the UI
thread.  Why should writing new, thread-safe Lisp for Emacs be any more
difficult?

> Those mechanisms only work when Emacs is idle, which is bad for
> features like progress reporting.  Doing this right requires to
> redesign how redisplay kicks in, and probably have several different
> kinds of redisplay, not one.

Maybe.  I haven't worked out the details of that yet, but in most other
GUI programs and toolkits, messages from other threads can only be
processed by the UI thread while it is idle.

> I think none of them are side-effect free, if you look closely.  They
> access buffer text, they move point, they temporarily change the
> selected window and the current buffer, the access and many times
> modify the obarray, etc. etc.

`intern' should be interlocked so that only one thread is accessing the
obarray at any given time.  Point and buffer text shouldn't prove a
problem, provided that the buffer being used is not accessed from any
other thread.

But yes, many of these functions will have to be audited and possibly
rewritten for multi-threaded correctness.