bug#61350: Eglot over Tramp freezes with large project

[Michael Albinus]

Thomas Koch <thomas@koch.ro> writes:

Hi Thomas,

> Please bare with me Michael, if I ask a stupid question although
> you're a long time Emacs contributor. I saw now that concurrency in
> Emacs is an age old problem:
> https://www.emacswiki.org/emacs/ConcurrentEmacs

Don't worry. You don't ask stupid questions, often it is useful to think
differently.

> There are emacs-async and emacs-deferred. Couldn't one of them be used
> to write a non-blocking wrapper for file-io? NodeJS made this
> programming model popular when it appeared in 2009. I can't believe
> that nobody discussed this before for Emacs?

emacs-async uses a second Emacs instance, AFAIU. I doubt that this
approach is helpful for Tramp, which needs to read files into buffers
and alike.

emacs-deferred I don't know.

> Meanwhile I could hack eglot to not block so far by replacing
> find-buffer-visiting with get-file-buffer in
> eglot-handle-notification. Of course there will probably be other
> places that cause problems and I could also only use this hack because
> my project files are not opened via symlinks.

Yes, that's similar to what I've done while trying to mitigate the
problem. But then, later, there was still the famous "Forbidden
reentrant call of Tramp", which we must fix anyway.

In fact, the threaded Tramp some years ago wasn't that bad. It worked in
many cases. The major problem with threads in Emacs is user
interaction. When there is a dialogue with a user, for example asking
for a password in the minibuffer, it happened that the user input didn't
return to the correct thread. Something like a UI-thread is needed in
Emacs, I believe. That is, all threads in Emacs delegate user
interaction to that UI-thread, and then wait for the result via a
mutex. Something like this.

For the new attempt to bring threads to Tramp, I plan another
approach. Not a whole operation (like find-file) shall be located into
an own thread. Instead, just the process communication,
process-send-string and accept-process-output, shall be delegated to
another thread. This are the places we have problems. Let's see how far
I could go. But pls don't hold your breath, it will take time until I
could show results.

Best regards, Michael.