bug#66117: 30.0.50; `find-buffer-visiting' is slow when opening large number of buffers

[Ihor Radchenko]

Dmitry Gutov <dmitry@gutov.dev> writes:

>> Would it be possible to implement some kind of caching mechanism to be
>> used by `find-buffer-visiting'?
>
> I'm guessing you Cc'd me because of an existing comment inside xref.el? 

Actually, mostly because I though that this discussion might be of
interest for you.

> As you can see I decided not to use this function there, but even 
> get-file-buffer wasn't as fast as I would've wanted, so there's a 
> quick-and-dirty caching solution for sequential lookups (which assumes 
> that the same file would be looked up multiple times in a row).

For me, the situation is different - a new not-yet-open file is looked
up multiple times in a row, which is a bit more tricky.

> Putting aside the perspective of maintaining a cache in the core 
> (hopefully this will be discussed later in this feature request), any 
> chance you could initialize that cache yourself first (by iterating 
> through buffers and building file-truename -> buffer map), before you 
> are doing those many lookups? Somewhere in org-agenda-list, perhaps.
>
> That should change the complexity from N*M to N+M, more or less.

Sure. That's my plan to address older Emacs versions. But such iteration
will involve copy-pasting a big part of `find-buffer-visiting' source
code, which is not something I want to maintain in a long term (what if
`find-buffer-visiting' changes in future?).

Moreover, `find-buffer-visiting' is called by `find-file-noselect', and
I simply cannot make use of cache there without modifying the code
upstream; or writing my own version of `find-file-noselect' - bad idea
maintenance-wise.

I think that the best way that will benefit more than Org mode is
arranging internal cache that will link buffer-file-name,
buffer-file-truename, and buffer-file-number with buffers; and maintain
the correctness of the cache if buffer-file-name changes for any reason.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>