bug#34343: [PATCH] Make project--find-regexp-in-files work with remote files

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Hi Michael,

Hi Dmitry,

>
> On 01.01.2020 14:29, Michael Albinus wrote:
>> I've played a little bit with this. Even if there is no remote file name
>> involved, file-local-name is slow due to the file-remote-p call and the
>> file name handler mechanery. See:
>
> This is unfortunate. But there's a bigger problem: file-local-name is
> even slower when the file name actually *is* remote.
>
>> Refactoring the code, and applying `file-local-name' just to the cases
>> where it is needed, shows an elapsed time similar to the initial one.
>
> If you take a look at xref-matches-in-files, you will find that is
> exactly what I did.
>
> But check this out:
>
> ELISP> (benchmark-run-compiled
>     nil
>   (dotimes (i 10000) (file-local-name (format "/tmp/%d" i))))
>
> (0.01541569 0 0.0)
>
> ELISP> (benchmark-run-compiled
>     nil
>   (dotimes (_i 10000) (file-local-name (format "/ssh:abc@def.com:/"))))
> (3.229403787 28 1.9053286330000034)
>
> And that is for just 10'000 files, not for 1'000'000.

I've started with

--8<---------------cut here---------------start------------->8---
(benchmark-run-compiled
    nil
  (dotimes (i 10000) (file-local-name (format "/ssh:abc@def.com:/%d" i))))

(4.222947337 31 1.384718521)
--8<---------------cut here---------------end--------------->8---

Then I have removed everything from tramp-handle-file-remote-p which
isn't necessary for the 'localname case, and I came to

--8<---------------cut here---------------start------------->8---
(benchmark-run-compiled
    nil
  (dotimes (i 10000) (file-local-name (format "/ssh:abc@def.com:/%d" i))))

(3.876818843 28 1.3257780540000006)
--8<---------------cut here---------------end--------------->8---

Better, but not significantly. And tramp-handle-file-remote-p wasn't
usable for other cases anymore. So I fear we cannot do much better as it
is.

> And with longer file names, the process takes even longer (twice as
> long in my real-world example).

I cannot confirm "twice as long". I've tried

--8<---------------cut here---------------start------------->8---

(benchmark-run-compiled
    nil
  (dotimes (i 10000) (file-local-name (format 
"/ssh:abc@def.com:/a/b/c/d/e/f/g/h/i/j/k/l/m%d" i))))

(4.185161814 30 1.425226717000001)
--8<---------------cut here---------------end--------------->8---

The difference is likely because tramp-file-name-regexp has been
applied to a longer string.

If I simplify tramp-file-name-regexp, we get

--8<---------------cut here---------------start------------->8---
(setq tramp-file-name-regexp "\\`/[^/:]+:[^/:]*:")
"\\`/[^/:]+:[^/:]*:"

(benchmark-run-compiled
    nil
  (dotimes (i 10000) (file-local-name (format 
"/ssh:abc@def.com:/a/b/c/d/e/f/g/h/i/j/k/l/m%d" i))))
(3.510464452 30 1.4363330669999996)
--8<---------------cut here---------------end--------------->8---

Better, but with the penalty that the other parts of a remote file name
aren't detected any longer, and tramp-handle-file-remote-p still works
only for the 'localname part. Both not acceptable.

So I fear we must live with the status. Yes, file-local-name is slow for
remote files, and yes, it doesn't matter for single invocations. Bulk
invocation, as you have it, will show an additional performance penalty,
even if there is no remote execution of any command.

Best regards, Michael.