bug#59415: 29.0.50; [feature/tree-sitter] c-ts-mode fails to fontify a portion of a large C file

[Theodor Thornhill]

Hi and thanks for cc.
>
> Observe that fontifications stop at this line for some reason.
> Fontification reappears on line 209271.  Maybe it's because of the many
> braces that appear in warning face?  Why does TS think there are syntax
> errors here?  The C++ TS parser doesn't have that problem, btw.
>

It seems the c parser definitely can't handle what it's seeing.

> P.S. Btw, isn't the treesit-max-buffer-size limit too low?  4 MiB?
>

It might be!  IIRC treesit uses 10x the buffer size to store the ast, so
it'll be some more memory usage.  I'll do some more digging, but in the
meantime I attach this profiler report that shows font-locking as the
culprit:

In this profile I followed your repro, and did some more movement around
the buffer after.  This isn't from emacs -Q, but I believe the results
will be just the same, considering where the slowness seems to be


       16695  85% - redisplay_internal (C function)
       16695  85%  - jit-lock-function
       16695  85%   - jit-lock-fontify-now
       16695  85%    - jit-lock--run-functions
       16695  85%     - run-hook-wrapped
       16695  85%      - #<compiled -0x156eddb48a262583>
       16695  85%       - font-lock-fontify-region
       16695  85%        - font-lock-default-fontify-region
       16679  84%         - treesit-font-lock-fontify-region
        2080  10%            treesit-buffer-root-node
        2689  13% - command-execute
        2689  13%  - call-interactively
        2380  12%   - funcall-interactively
        1576   8%    - scroll-up-command
        1525   7%     - scroll-up
        1525   7%      - jit-lock-function
        1525   7%       - jit-lock-fontify-now
        1525   7%        - jit-lock--run-functions
        1525   7%         - run-hook-wrapped
        1525   7%          - #<compiled -0x15bd2ea490f7f983>
        1525   7%           - font-lock-fontify-region
        1525   7%            - font-lock-default-fontify-region
        1525   7%               treesit-font-lock-fontify-region
         633   3%    - end-of-buffer
         628   3%     - recenter
         628   3%      - jit-lock-function
         628   3%       - jit-lock-fontify-now
         628   3%        - jit-lock--run-functions
         628   3%         - run-hook-wrapped
         628   3%          - #<compiled -0x14388b9914c40883>
         628   3%           - font-lock-fontify-region
         628   3%            - font-lock-default-fontify-region
         628   3%               treesit-font-lock-fontify-region
           5   0%       push-mark
         128   0%    - project-find-file
         128   0%     - project-find-file-in
          86   0%      - project--read-file-cpd-relative
          86   0%       - project--completing-read-strict
          86   0%        - completing-read
          86   0%         - completing-read-default
          86   0%          - apply
          86   0%           - vertico--advice
          86   0%            - apply
          86   0%             - #<compiled -0x2e553dfe9f75520>
          79   0%              - read-from-minibuffer
          37   0%               - vertico--exhibit
          26   0%                - vertico--update
          22   0%                   redisplay
           4   0%                 - vertico--recompute
           4   0%                  - vertico-sort-history-length-alpha
           4   0%                   - mapcan
           4   0%                    - #<compiled -0x1cada1a01280ac5f>
           4   0%                       sort
          11   0%                - vertico--display-candidates
          11   0%                   vertico--resize-window
          15   0%               - timer-event-handler
          10   0%                - apply
           7   0%                 - battery-update-handler
           7   0%                  - sit-for
           7   0%                   - redisplay
           7   0%                      redisplay_internal (C function)
           3   0%                   #<compiled 0x12c58df73848dc86>
           2   0%               - internal-timer-start-idle
           2   0%                  timerp
           2   0%               - command-execute
           2   0%                - call-interactively
           2   0%                 - funcall-interactively
           2   0%                  - vertico-exit
           2   0%                   - vertico--match-p
           2   0%                    - test-completion
           2   0%                     - #<compiled -0x1464df124877e5c8>
           2   0%                        complete-with-action
          27   0%      - find-file
          27   0%       - find-file-noselect
          24   0%        - find-file-noselect-1
           4   0%         - insert-file-contents
           4   0%          - set-auto-coding
           4   0%           - find-auto-coding
           4   0%              sgml-html-meta-auto-coding-function
           4   0%         - after-find-file
           4   0%          - normal-mode
           4   0%           - set-auto-mode
           4   0%            - set-auto-mode--apply-alist
           4   0%             - set-auto-mode-0
           4   0%              - c-ts-mode
           4   0%                 treesit-ready-p
           3   0%        - find-buffer-visiting
           3   0%           abbreviate-file-name
          15   0%      - project-files
          15   0%       - apply
          15   0%        - #<compiled -0x7a9f28e22b82f80>
          15   0%         - mapcan
          15   0%          - #<compiled 0x14d13416934a6c69>
          15   0%           - project--vc-list-files
          11   0%            - apply
          11   0%             - vc-git--run-command-string
          11   0%              - #<compiled 0x88854d79be8a>
          11   0%               - kill-buffer
          11   0%                - replace-buffer-in-windows
          11   0%                 - unrecord-window-buffer
          11   0%                    assq-delete-all
           4   0%              split-string
           5   0%    - next-line
           5   0%     - line-move
           5   0%        line-move-visual
           4   0%    - execute-extended-command
           4   0%     - command-execute
           4   0%      - call-interactively
           4   0%       - funcall-interactively
           4   0%          profiler-stop
           2   0%    - digit-argument
           2   0%     - universal-argument--mode
           2   0%        set-transient-map
         309   1%   - byte-code
         309   1%    - read-extended-command
         309   1%     - read-extended-command-1
         309   1%      - completing-read
         309   1%       - completing-read-default
         309   1%        - apply
         309   1%         - vertico--advice
         309   1%          - apply
         309   1%           - #<compiled -0x2e553dfe9f75520>
         276   1%            - read-from-minibuffer
         253   1%             - vertico--exhibit
         249   1%              - vertico--update
         240   1%               - vertico--recompute
         236   1%                - vertico--all-completions
         236   1%                 - apply
         236   1%                  - completion-all-completions
         236   1%                   - completion--nth-completion
         236   1%                    - completion--some
         236   1%                     - #<compiled -0x18735a95ea969dbf>
         163   0%                      - completion-basic-all-completions
         163   0%                       - completion-pcm--all-completions
         163   0%                        - all-completions
         163   0%                         - #<compiled -0xf2f3e8a19f62ad2>
         163   0%                          - complete-with-action
           4   0%                           - all-completions
           4   0%                            - #<compiled 0xadd42c29ce50255>
           4   0%                               #<compiled 0x1a1dcc3780af9553>
          73   0%                      - completion-substring-all-completions
          73   0%                       - completion-substring--all-completions
          64   0%                        - completion-pcm--all-completions
          64   0%                         - all-completions
          64   0%                          - #<compiled -0x1464df124877e5c8>
          64   0%                             complete-with-action
           4   0%                - test-completion
           4   0%                 - #<compiled -0xf2f3e8a19f62ad2>
           4   0%                    complete-with-action
           7   0%                 redisplay
           4   0%              - vertico--display-candidates
           4   0%                 vertico--resize-window
           4   0%             - redisplay_internal (C function)
           4   0%              - eval
           4   0%                 unless
         201   1% + timer-event-handler
          50   0% + ...
           4   0%   set-message-functions