[PATCH] Re: cc-mode fontification feels random

[Alan Mackenzie]

Hello, Daniel.

On Thu, Jun 03, 2021 at 20:16:53 -0700, Daniel Colascione wrote:
> As long as I can remember, cc-mode fontification has felt totally 
> random, with actual faces depending on happenstance of previously-parsed 
> types, luck of the draw in jit-lock chunking, and so on. Is there any 
> *general* way that we can make fontification more robust and consistent?

> For years and years now, I've been thinking we just need more 
> deterministic parser-and-based mode support, and I still think that, but 
> on a realistic level, that doesn't seem to be coming any time soon.

> In the meantime, is there any general approach we might be able to use 
> to get stuff like the attached to stop happening?

Here, "stuff like the attached" was having some types correctly
fontified, others not.  This was due to the order, somewhat random, in
which a type is recognised as such and entered into a CC Mode table, and
its use being scanned in a jit-lock chunk.

The following patch is an attempt to improve this situation.  It is best
used with jit-stealth-lock enabled.  I have tried it out with the
following settings:

    jit-lock-stealth-load: 200 ; i.e. inactive.
    jit-lock-stealth-nice: 0.1 ; 100 ms between fontifying stealth
                                 chunks.
    jit-lock-stealth-time: 1   ; 1 second idle time before stealth kicks
                                 in.

Whenever a new found type is entered into the CC Mode table, it marks
all occurrences of the type in the buffer for fontification (by setting
the 'fontified text property to nil on it), and causes an immediate
redisplay when there are occurrences of the new type in a window.

I think stealth lock could be enhanced by having it fontify several
500-byte chunks together, say until 0.05s time has been taken up.  This
could speed up stealth fontification while still leaving the keyboard
responsive to the user.

Anyhow, could you try the patch please, in particular on the source code
which you posted a picture of back in June, and see how well it,
together with stealth fontification, helps with the random
fontification.  (The patch is still a fairly rough sketch, not a
finished patch.)

Thanks!



diff -r a811a06c82c2 cc-engine.el
--- a/cc-engine.el      Sat Aug 21 10:14:48 2021 +0000
+++ b/cc-engine.el      Mon Aug 30 18:23:44 2021 +0000
@@ -173,6 +173,9 @@
 (cc-bytecomp-defvar c-doc-line-join-end-ch)
 (defvar c-syntactic-context)
 (defvar c-syntactic-element)
+(defvar c-new-id-start)
+(defvar c-new-id-end)
+(defvar c-new-id-is-type)
 (cc-bytecomp-defvar c-min-syn-tab-mkr)
 (cc-bytecomp-defvar c-max-syn-tab-mkr)
 (cc-bytecomp-defun c-clear-syn-tab)
@@ -6839,21 +6842,47 @@
   (setq c-found-types
        (make-hash-table :test #'equal :weakness nil)))
 
+;;;; OLD STOUGH, 2021-08-23
+;; (defun c-add-type (from to)
+;;   ;; Add the given region as a type in `c-found-types'.  If the region
+;;   ;; doesn't match an existing type but there is a type which is equal
+;;   ;; to the given one except that the last character is missing, then
+;;   ;; the shorter type is removed.  That's done to avoid adding all
+;;   ;; prefixes of a type as it's being entered and font locked.  This
+;;   ;; doesn't cover cases like when characters are removed from a type
+;;   ;; or added in the middle.  We'd need the position of point when the
+;;   ;; font locking is invoked to solve this well.
+;;   ;;
+;;   ;; This function might do hidden buffer changes.
+;;   (let ((type (c-syntactic-content from to c-recognize-<>-arglists)))
+;;     (unless (gethash type c-found-types)
+;;       (remhash (substring type 0 -1) c-found-types)
+;;       (puthash type t c-found-types))))
+;;;; NEW STOUGH, 2021-08-29
+(defun c-add-type-1 (from to)
+  ;; FIXME!!!
+  (let ((type (c-syntactic-content from to c-recognize-<>-arglists)))
+    (unless (gethash type c-found-types)
+      (puthash type t c-found-types)
+      (when (and (eq (string-match c-symbol-key type) 0)
+                (eq (match-end 0) (length type)))
+       (c-fontify-new-found-type type)))))
+
 (defun c-add-type (from to)
-  ;; Add the given region as a type in `c-found-types'.  If the region
-  ;; doesn't match an existing type but there is a type which is equal
-  ;; to the given one except that the last character is missing, then
-  ;; the shorter type is removed.  That's done to avoid adding all
-  ;; prefixes of a type as it's being entered and font locked.  This
-  ;; doesn't cover cases like when characters are removed from a type
-  ;; or added in the middle.  We'd need the position of point when the
-  ;; font locking is invoked to solve this well.
+  ;; Add the given region as a type in `c-found-types'.  If the region is or
+  ;; overlaps an identifier which might be being typed in, don't record it.
+  ;; This is tested by checking `c-new-id-start' and `c-new-id-end'.  That's
+  ;; done to avoid adding all prefixes of a type as it's being entered and
+  ;; font locked.  This is a bit rough and ready, but now covers adding
+  ;; characters into the middle of an identifer.
   ;;
   ;; This function might do hidden buffer changes.
-  (let ((type (c-syntactic-content from to c-recognize-<>-arglists)))
-    (unless (gethash type c-found-types)
-      (remhash (substring type 0 -1) c-found-types)
-      (puthash type t c-found-types))))
+  (if (and c-new-id-start c-new-id-end
+          (<= from c-new-id-end) (>= to c-new-id-start))
+      (setq c-new-id-is-type t)
+    (c-add-type-1 from to)))
+;;;; END OF NEW STOUGH
+;;;; END OF NEW STOUGH
 
 (defun c-unfind-type (name)
   ;; Remove the "NAME" from c-found-types, if present.
diff -r a811a06c82c2 cc-fonts.el
--- a/cc-fonts.el       Sat Aug 21 10:14:48 2021 +0000
+++ b/cc-fonts.el       Mon Aug 30 18:23:44 2021 +0000
@@ -2253,6 +2253,48 @@
     ;; defvar will install its default value later on.
     (makunbound def-var)))
 
+;;;; NEW STOUGH, 2021-08-29
+;; `c-re-redisplay-timer' is a timer which, when triggered, causes a
+;; redisplay.
+(defvar c-re-redisplay-timer nil)
+
+(defun c-force-redisplay (start end)
+  ;; Force redisplay immediately.  This assumes `font-lock-support-mode' is
+  ;; 'jit-lock-mode.  Set the variable `c-re-redisplay-timer' to nil.
+  (jit-lock-force-redisplay (copy-marker start) (copy-marker end))
+  (setq c-re-redisplay-timer nil))
+
+(defun c-fontify-new-found-type (type)
+  ;; Cause the fontification of TYPE, a string, wherever it occurs in the
+  ;; buffer.  If TYPE is currently displayed in a window, cause redisplay to
+  ;; happen "instantaneously".  These actions are done only when jit-lock-mode
+  ;; is active.
+  (when (and (boundp 'font-lock-support-mode)
+            (eq font-lock-support-mode 'jit-lock-mode))
+    (c-save-buffer-state
+       ((window-boundaries
+         (mapcar (lambda (win)
+                   (cons (window-start win)
+                         (window-end win)))
+                 (get-buffer-window-list (current-buffer) 'no-mini t)))
+        (target-re (concat "\\_<" type "\\_>")))
+      (save-excursion
+       (save-restriction
+         (widen)
+         (goto-char (point-min))
+         (while (re-search-forward target-re nil t)
+           (put-text-property (match-beginning 0) (match-end 0)
+                              'fontified nil)
+           (dolist (win-boundary window-boundaries)
+             (when (and (< (match-beginning 0) (cdr win-boundary))
+                        (> (match-end 0) (car win-boundary))
+                        (c-get-char-property (match-beginning 0) 'fontified)
+                        (not c-re-redisplay-timer))
+               (setq c-re-redisplay-timer
+                     (run-with-timer 0 nil #'c-force-redisplay
+                                     (match-beginning 0) (match-end 
0)))))))))))
+
+;;;; END OF NEW STOUGH, 2021-08-29
 
 ;;; C.
 
diff -r a811a06c82c2 cc-mode.el
--- a/cc-mode.el        Sat Aug 21 10:14:48 2021 +0000
+++ b/cc-mode.el        Mon Aug 30 18:23:44 2021 +0000
@@ -173,6 +173,16 @@
   (when c-buffer-is-cc-mode
     (save-restriction
       (widen)
+;;;; NEW STOUGH, 2021-08-23
+      (let ((lst (buffer-list)))
+       (catch 'found
+         (dolist (b lst)
+           (if (and (not (eq b (current-buffer)))
+                    (with-current-buffer b
+                      c-buffer-is-cc-mode))
+               (throw 'found nil)))
+         (remove-hook 'post-command-hook 'c-post-command)))
+;;;; END OF NEW STOUGH
       (c-save-buffer-state ()
        (c-clear-char-properties (point-min) (point-max) 'category)
        (c-clear-char-properties (point-min) (point-max) 'syntax-table)
@@ -728,6 +738,9 @@
   (or (memq 'add-hook-local c-emacs-features)
       (make-local-hook 'after-change-functions))
   (add-hook 'after-change-functions 'c-after-change nil t)
+;;;; NEW STOUGH, 2021-08-23
+  (add-hook 'post-command-hook 'c-post-command)
+;;;; END OF NEW STOUGH
   (when (boundp 'font-lock-extend-after-change-region-function)
     (set (make-local-variable 'font-lock-extend-after-change-region-function)
         'c-extend-after-change-region))) ; Currently (2009-05) used by all
@@ -1936,6 +1949,45 @@
        ;; confused by already processed single quotes.
        (narrow-to-region (point) (point-max))))))
 
+;;;; NEW STOUGH, 2021-08-22
+;; The next two variables record the bounds of an identifier currently being
+;; typed in.  These are used to prevent such a partial identifier being
+;; recorded as a found type by c-add-type.
+(defvar c-new-id-start nil)
+(make-variable-buffer-local 'c-new-id-start)
+(defvar c-new-id-end nil)
+(make-variable-buffer-local 'c-new-id-end)
+;;;; NEW STOUGH, 2021-08-29
+;; The next variable, when non-nil, records that the previous two variables
+;; define a type.
+(defvar c-new-id-is-type nil)
+(make-variable-buffer-local 'c-new-id-is-type)
+;;;; END OF NEW STOUGH
+
+(defun c-update-new-id (end)
+  ;; Fill this in.  FIXME!!!
+  (save-excursion
+    (goto-char end)
+    (let ((id-beg (c-on-identifier)))
+      (setq c-new-id-start id-beg
+           c-new-id-end (and id-beg
+                             (progn (c-end-of-current-token) (point)))))))
+
+
+(defun c-post-command ()
+  ;; If point was inside of a new identifier and no longer is, record that
+  ;; fact.
+  (when (and c-buffer-is-cc-mode
+            c-new-id-start c-new-id-end
+            (or (> (point) c-new-id-end)
+                (< (point) c-new-id-start)))
+    (when c-new-id-is-type
+      (c-add-type-1 c-new-id-start c-new-id-end))
+    (setq c-new-id-start nil
+         c-new-id-end nil
+         c-new-id-is-type nil)))
+;;;; END OF NEW STOUGH
+
 (defun c-before-change (beg end)
   ;; Function to be put in `before-change-functions'.  Primarily, this calls
   ;; the language dependent `c-get-state-before-change-functions'.  It is
@@ -2133,6 +2185,9 @@
                                                      c->-as-paren-syntax)
                    (c-clear-char-property-with-value beg end 'syntax-table 
nil)))
 
+;;;; NEW STOUGH, 2021-08-22
+               (c-update-new-id end)
+;;;; END OF NEW STOUGH
                (c-trim-found-types beg end old-len) ; maybe we don't
                                                     ; need all of these.
                (c-invalidate-sws-region-after beg end old-len)



-- 
Alan Mackenzie (Nuremberg, Germany).