bug#66117: 30.0.50; `find-buffer-visiting' is slow when opening large number of buffers

[Dmitry Gutov]

On 22/09/2023 15:41, Ihor Radchenko wrote:
> Dmitry Gutov <dmitry@gutov.dev> writes:
>
> > > Moreover, `find-buffer-visiting' is called by `find-file-noselect', and
> > > I simply cannot make use of cache there without modifying the code
> > > upstream; or writing my own version of `find-file-noselect' - bad idea
> > > maintenance-wise.
> >
> > ...if most of said calls are done through find-file-noselect, I suppose
> > that solution is a no-go.
>
> Not a no-go, but not a complete solution either.
> I asked the user who provided the profiler report I have attached to
> replace `find-buffer-visiting' with `get-file-buffer' in the relevant Org
> sources and it did lead to ~30% runtime reduction.
>
> However, upon recording the profiler report with that change,
> `find-buffer-visiting' still took a significant fraction of CPU time.
> So, I decided to reach out upstream.

Because the rest of the calls are made from Org?

I see this in the profile, though:

        6247  20%        - editorconfig--advice-find-file-noselect

There seems to be a lot of timers in that profile, and this is in one of 
them (under "Automatic GC" which takes up 53% of the time).

So one was to improve this in the meantime would be to bring it up with 
"editorconfig" authors. I would even question the wisdom of advising 
find-file-noselect.

> Will it be acceptable to implement the cache using variable watchers?

I think variable watchers are mostly for debugging? I was thinking of a 
straight approach where the base primitives that visit a file or kill a 
buffer update said cache.

> > > I think that the best way that will benefit more than Org mode is
> > > arranging internal cache that will link buffer-file-name,
> > > buffer-file-truename, and buffer-file-number with buffers; and maintain
> > > the correctness of the cache if buffer-file-name changes for any reason.
> >
> > I think that is doable.
> >
> > It probably won't regress the performance of any particular scenario
> > either, but we should benchmark opening a bunch of files this way anyway
> > (might actually get faster, due to find-buffer-visiting calls).
>
> The regression might happen when the number of buffers is small -
> when hash tables become slower compared to simple list lookup.
> But in such scenario, we will be talking about very small absolute
> runtimes anyway, so it should probably not matter in practice.

I suppose.