Bug: performance issue in org-font-lock-add-priority-faces related to org-priority-regexp [9.4 (9.4-7-g3eccc5-elpaplus @ /home/standard/.emacs.d/elpa/26.3/develop/org-plus-contrib-20200921/)]

[Tim Frana]

Hello,

After updating from org-plus-contrib-20200810 to 
org-plus-contrib-20200921, opening org files has become extremely slow 
for me. As a rough estimate, a 400 KB org file of mine used to take 5 
seconds on first load in a session, and after the update needed more 
than 2 minutes. This is the case both in Emacs 27.1 in Windows, and 
Emacs 26.3 in Windows WSL1 Ubuntu 20.04 LTS, using spacemacs develop.

I can narrow it down to this change:

org-font-lock-add-priority-faces used to search for this regexp: "^\\*+ 
.*?\\(\\[#\\(.\\)\\]\\)"
Now it searches for org-priority-regexp, which by default resolves to 
".*?\\(\\[#\\([A-Z0-9]+\\)\\] ?\\)".

This is much slower, and also matches more than I expected. In the 
following example, I expect only [#A], but it will match [#B] and [#C] 
too. Is this intended?

** TODO [#A] Headline [#B]
this is matched despite not being a heading: [#C] more text

It seems plausible that the slowdown comes from the regex engine now 
doing elaborate matching on all lines, instead of fast skipping over 
non-heading lines.

This version seems to resolve my problem, where the beginning "^\\*+ " 
should narrow matches to heading lines only:
(setq org-priority-regexp "^\\*+ .*?\\(\\[#\\([A-Z0-9]+\\)\\] ?\\)")

Profiler excerpt (the other 51% were mostly eaten by spell checking 
during idling, as I got distracted waiting for the loading to finish):

- set-auto-mode                                     5498  49%
 - set-auto-mode-0                                  5498  49%
  - org-mode                                        5498  49%
   - run-mode-hooks                                 5498  49%
    - apply                                         5497  49%
     - run-hooks                                    5497  49%
      - org-superstar-mode                          5493  49%
       - org-superstar--fontify-buffer              5493  49%
        - font-lock-ensure                          5493  49%
         - jit-lock-fontify-now                     5493  49%
          - jit-lock--run-functions                 5493  49%
           - run-hook-wrapped                       5493  49%
            - #<compiled 0x28ec06d>                 5391 48%
             - font-lock-fontify-region             5391  48%
              - font-lock-default-fontify-region    5391  48%
               - font-lock-fontify-keywords-region  5391  48%
                  org-font-lock-add-priority-faces  5391  48%

Best regards,
Tim


------------------------------------------------------------------------



Emacs  : GNU Emacs 26.3 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.22.30)
 of 2019-09-16
Package: Org mode version 9.4 (9.4-7-g3eccc5-elpaplus @ 
/home/standard/.emacs.d/elpa/26.3/develop/org-plus-contrib-20200921/)

current state:
==============
(setq
 org-src-mode-hook '(org-src-babel-configure-edit-buffer 
org-src-mode-configure-edit-buffer)
 org-after-todo-state-change-hook '(org-expiry-insert-created)
 org-link-shell-confirm-function 'yes-or-no-p
 org-blank-before-new-entry '((heading . t) (plain-list-item))
 org-babel-after-execute-hook '(spacemacs/ob-fix-inline-images)
 org-metadown-hook '(org-babel-pop-to-session-maybe)
 org-agenda-tags-todo-honor-ignore-options t
 org-export-with-sub-superscripts '{}
 org-clock-out-hook '(org-clock-remove-empty-clock-drawer)
 org-refile-targets '((nil :maxlevel . 3) (org-agenda-files :maxlevel . 3))
 org-html-format-inlinetask-function 
'org-html-format-inlinetask-default-function
 org-enforce-todo-dependencies t
 org-edit-src-content-indentation 0
 org-odt-format-headline-function 'org-odt-format-headline-default-function
 org-imenu-depth 8
 org-download-annotate-function 'org-download-annotate-default
 org-ascii-format-inlinetask-function 'org-ascii-format-inlinetask-default
 org-clock-report-include-clocking-task t
 org-modules '(org-habit ol-w3m ol-bbdb ol-bibtex ol-docview ol-gnus 
ol-info ol-irc ol-mhe ol-rmail ol-eww)
 org-blocker-hook '(org-block-todo-from-children-or-siblings-or-parent 
org-depend-block-todo)
 org-archive-location "%s_archive.org::"
 org-id-link-to-org-use-id 'create-if-interactive
 org-mode-hook '((lambda nil (setq paragraph-start "\f\\|[ ]*$" 
paragraph-separate "[     \f]*$")) turn-on-auto-fill org-tempo-setup 
org-clock-load
                 #[0 "\300\301\302\303\304$\207" [add-hook 
change-major-mode-hook org-show-all append local] 5] flyspell-mode 
spacemacs/org-setup-evil-surround spacemacs/load-yasnippet
                 #[0 "\301\211\207" [imenu-create-index-function 
org-imenu-get-tree] 2] toc-org-enable org-superstar-mode 
org-download-enable dotspacemacs//prettify-spacemacs-docs 
spacemacs//org-babel-do-load-languages
                 #[0 "\300\301\302\303\304$\207" [add-hook 
change-major-mode-hook org-babel-show-result-all append local] 5] 
org-babel-result-hide-spec org-babel-hide-all-hashes 
spacemacs//evil-org-mode org-eldoc-load
                 spacemacs//init-company-org-mode company-mode)
 org-clock-persist t
 org-odt-format-drawer-function #[514 "\207" [] 3 "\n\n(fn NAME CONTENTS)"]
 org-outline-path-complete-in-steps nil
 org-archive-hook '(org-attach-archive-delete-maybe)
 org-agenda-restore-windows-after-quit t
 org-superstar-headline-bullets-list '("■" "◆" "▲" "▶")
 org-confirm-elisp-link-function 'yes-or-no-p
 org-startup-indented t
 org-startup-with-inline-images t
 org-clock-history-length 20
 org-download-file-format-function 'org-download-file-format-default
 org-agenda-before-write-hook '(org-agenda-add-entry-text)
 org-hide-block-startup t
 org-metaup-hook '(org-babel-load-in-session-maybe)
 org-bibtex-headline-format-function #[257 "\300
\236A\207" [:title] 3 
"\n\n(fn ENTRY)"]
 org-latex-format-drawer-function #[514 "\207" [] 3 "\n\n(fn _ CONTENTS)"]
 org-expiry-handler-function 'org-toggle-archive-tag
 org-babel-pre-tangle-hook '(save-buffer)
 org-file-apps '((auto-mode . emacs) 
("\\.\\(xlsx?\\|docx?\\|pptx?\\|pdf\\|ods\\|png\\)\\'" . default))
 org-tab-first-hook '(org-babel-hide-result-toggle-maybe 
org-babel-header-arg-expand)
 org-babel-load-languages '((dot . t) (calc . t) (plantuml . t) (sql . 
t) (js . t) (shell . t) (emacs-lisp . t))
 org-log-done 'time
 org-export-backends '(confluence md ascii html icalendar latex odt)
 org-ascii-format-drawer-function #[771 "
\207" [] 4 "\n\n(fn NAME 
CONTENTS WIDTH)"]
 org-trigger-hook '(org-depend-trigger-todo)
 org-priority-lowest 68
 org-agenda-loop-over-headlines-in-active-region nil
 org-src-lang-modes '(("arduino" . arduino) ("redis" . redis) ("php" . 
php) ("C" . c) ("C++" . c++) ("asymptote" . asy) ("bash" . sh) ("beamer" 
. latex) ("calc" . fundamental) ("cpp" . c++) ("ditaa" . artist) ("dot" 
. fundamental)
                      ("elisp" . emacs-lisp) ("ocaml" . tuareg) 
("screen" . shell-script) ("shell" . sh) ("sqlite" . sql))
 org-agenda-prefix-format '((agenda . " %i %-12:c%?-12t% s") (timeline 
. "  % s") (todo . " %i %b") (tags . " %i %-12:c") (search . " %i %-12:c"))
 org-occur-hook '(org-first-headline-recenter)
 org-log-into-drawer t
 org-protocol-protocol-alist '(("capture-eww-readable" :protocol 
"capture-eww-readable" :function 
org-protocol-capture-html--capture-eww-readable :kill-client t)
                               ("capture-html" :protocol "capture-html" 
:function org-protocol-capture-html--with-pandoc :kill-client t))
 org-after-tags-change-hook '(org-expiry-insert-created)
 org-cycle-hook '(org-cycle-hide-archived-subtrees 
org-cycle-hide-drawers org-cycle-show-empty-lines 
org-optimize-window-after-visibility-change)
 org-todo-keywords '((sequence "TODO(t)" "NEXT(n)" "|" "DONE(d!/!)") 
(type "WAIT(w@/!)" "DELEGATED(e@/!)" "IDEA(i!/!)" "|" "CANCELED(c@/!)" 
"PASSED-ON(p@/!)"))
 org-speed-command-hook '(org-speed-command-activate 
org-babel-speed-command-activate)
 org-clock-persist-file "/home/standard/.emacs.d/.cache/org-clock-save.el"
 org-attach-id-dir "org-attach/"
 org-odt-format-inlinetask-function 
'org-odt-format-inlinetask-default-function
 org-latex-prefer-user-labels t
 org-publish-timestamp-directory 
"/home/standard/.emacs.d/.cache/.org-timestamps/"
 org-export-async-init-file 
"/home/standard/.emacs.d/layers/+emacs/org/local/org-async-init.el"
 org-refile-use-outline-path t
 org-export-before-parsing-hook '(org-attach-expand-links)
 org-id-locations-file "/home/standard/.emacs.d/.cache/.org-id-locations"
 org-confirm-shell-link-function 'yes-or-no-p
 org-link-parameters '(("eww" :follow org-eww-open :store 
org-eww-store-link) ("rmail" :follow org-rmail-open :store 
org-rmail-store-link) ("mhe" :follow org-mhe-open :store org-mhe-store-link)
                       ("irc" :follow org-irc-visit :store 
org-irc-store-link :export org-irc-export) ("info" :follow org-info-open 
:export org-info-export :store org-info-store-link)
                       ("gnus" :follow org-gnus-open :store 
org-gnus-store-link) ("docview" :follow org-docview-open :export 
org-docview-export :store org-docview-store-link)
                       ("bibtex" :follow org-bibtex-open :store 
org-bibtex-store-link) ("bbdb" :follow org-bbdb-open :export 
org-bbdb-export :complete org-bbdb-complete-link :store 
org-bbdb-store-link) ("w3m" :store org-w3m-store-link)
                       ("attachment" :follow org-attach-follow 
:complete org-attach-complete-link) ("id" :follow org-id-open)
                       ("orgit-rev" :store orgit-rev-store :follow 
orgit-rev-open :export orgit-rev-export :complete orgit-rev-complete-link)
                       ("orgit-log" :store orgit-log-store :follow 
orgit-log-open :export orgit-log-export :complete orgit-log-complete-link)
                       ("orgit" :store orgit-status-store :follow 
orgit-status-open :export orgit-status-export :complete 
orgit-status-complete-link) ("file+sys") ("file+emacs") ("shell" :follow 
org-link--open-shell)
                       ("news" :follow #[514 "\301\300\302Q\"\207" 
["news" browse-url ":"] 6 "\n\n(fn URL ARG)"]) ("mailto" :follow #[514 
"\301\300\302Q\"\207" ["mailto" browse-url ":"] 6 "\n\n(fn URL ARG)"])
                       ("https" :follow #[514 "\301\300\302Q\"\207" 
["https" browse-url ":"] 6 "\n\n(fn URL ARG)"]) ("http" :follow #[514 
"\301\300\302Q\"\207" ["http" browse-url ":"] 6 "\n\n(fn URL ARG)"])
                       ("ftp" :follow #[514 "\301\300\302Q\"\207" 
["ftp" browse-url ":"] 6 "\n\n(fn URL ARG)"]) ("help" :follow 
org-link--open-help) ("file" :complete org-link-complete-file) ("elisp" 
:follow org-link--open-elisp)
                       ("doi" :follow org-link--open-doi))
 org-html-postamble nil
 org-latex-format-headline-function 
'org-latex-format-headline-default-function
 org-expiry-inactive-timestamps t
 org-present-mode-hook '(spacemacs//org-present-start)
 org-link-translation-function 'toc-org-unhrefify
 org-link-elisp-confirm-function 'yes-or-no-p
 org-todo-keyword-faces '(("NEXT" . "yellow") ("WAIT" . "steel blue") 
("DELEGATED" . "steel blue") ("IDEA" . "khaki") ("DONE" . "PaleGreen3") 
("CANCELED" . "PaleGreen3") ("PASSED-ON" . "DimGrey"))
 org-latex-format-inlinetask-function 
'org-latex-format-inlinetask-default-function
 org-tab-before-tab-emulation-hook '(org-tempo-complete-tag)
 org-html-format-drawer-function #[514 "\207" [] 3 "\n\n(fn NAME 
CONTENTS)"]
 outline-mode-hook '(windows-scripts/dos-outline-hook)
 org-clock-into-drawer "CLOCK"
 org-image-actual-width nil
 org-clocktable-defaults '(:maxlevel 2 :lang "en" :scope file :block 
nil :wstart 1 :mstart 1 :tstart nil :tend nil :step nil :stepskip0 nil 
:fileskip0 nil :tags nil :match nil :emphasize nil :link nil :narrow nil 
:indent t :hidefiles
                           nil :formula nil :timestamp nil :level nil 
:tcolumns nil :formatter nil)
 org-present-mode-quit-hook '(spacemacs//org-present-end)
 org-html-format-headline-function 
'org-html-format-headline-default-function
 org-attach-dir-relative t
 org-use-sub-superscripts '{}
 org-agenda-custom-commands '(("d" "Clock report for today" agenda "" 
((org-agenda-span 1) (org-agenda-start-with-clockreport-mode t) 
(org-agenda-start-with-log-mode (quote clockcheck)))))
 org-download-link-format-function 
'org-download-link-format-function-default
 org-agenda-clockreport-parameter-plist '(:link t :maxlevel 3)
 org-ascii-text-width 999999
 org-attach-use-inheritance t
 org-download-abbreviate-filename-function 'file-relative-name
 org-agenda-todo-ignore-scheduled 'future
 org-default-notes-file "/home/standard/org/notes.org"
 org-insert-heading-hook '(org-expiry-insert-created)
 )