Re: [PATCH] Bug: fragile org refile cache

[Maxim Nikulin]

On 29/04/2021 23:08, Ihor Radchenko wrote:
> My experience is exactly opposite. Or maybe I miss something. Can you
> elaborate?

Some additions. org-outline-path-cache is used solely by 
org-refile-get-targets (maybe there are some calls in other packages) 
but it efficiency is questionable. It was not clear for me earlier that 
the cache is reset before each scan through a buffer. So if 
org-refile-use is disabled, org-outline-path-cache from previous run of 
org-refile or org-goto is not used as well. A query to 
org-outline-path-cache requires at least one backward search and hash 
lookup. During sequential scan in org-refile-get-targets it is enough to 
have previous heading path to update it when new heading is found. I 
think, org-outline-path-cache should be deprecated.

> Just cleanup heading text:

I have realized what is wrong with this benchmark. It runs so fast 
because it matches no headings, so it never spent time for cleaning them up.

> (benchmark-run 1
>    (goto-char (point-min))
>    (while (re-search-forward "^\\*+" nil t)
>      (let ((case-fold-search nil))

(beginning-of-line)
should be added before next call and (end-of-line) somewhere later in 
while body.

>        (looking-at org-complex-heading-regexp)
>        (if (not (match-end 4)) ""
>         ;; Remove statistics cookies.
>         (org-trim
>          (org-link-display-format
>           (replace-regexp-in-string
>            "\\[[0-9]+%\\]\\|\\[[0-9]+/[0-9]+\\]" ""
>            (match-string-no-properties 4)))))))) => (0.013364877 0 0.0)

On 29/04/2021 21:12, Ihor Radchenko wrote:
> For the cleaned heading text, I do not think that re-calculating the
> heading text on each change is a good idea. It may degrade typing
> latency. Yet, an acceptable approach could be simply invalidating cache
> for the changed headings. Then, outline paths can be re-calculated on
> changed headings when needed.

I agree that it is enough to invalidate cleaned heading on edit to 
refresh it in org-refile-get-targets. On the other hand, I still prefer 
text properties since they could be fetched even if some lines have been 
added or removed before. Position-based cache is useless in such cases.

Concerning typing latency, it should be postponed and resumed when no 
new edits is performed for certain period of time (~1s). However I am 
unsure if it is possible to accurately track all affected lines since 
later changes can add/remove lines before the line scheduled for 
invalidation.

On 30/04/2021 02:17, Tim Cross wrote:
> I suspect the reason it is not done automatically is that getting that
> right for all use cases is very hard to do without adding adverse impact
> to performance. A cache which is marked as 'dirty' too often results in
> too frequent cache refresh operations, but at the same time, determining
> what changes in an org file actually invalidate the cache can be a
> process intensive operation.

I believed that idea risen in this thread was to regenerate cache 
instead of spitting
"Please regenerate the refile cache with `C-0 C-c C-w'"
leaving more tricky cases for the user.