Re: Selective font-locking?

[JD Smith]

Thank you!  This method works surprisingly well to fontify input at the end of the buffer with another major-mode’s keywords/syntax.  It seems to be fairly efficient even for long multi-line input text:

(defun python-shell-multiline--apply-font-lock (limit)

  (if-let ((process (get-buffer-process (current-buffer)))

  (pmark (process-mark)))

      (if (> limit pmark)

 (let ((font-lock-keywords python-shell-multiline-font-lock-keywords)

(font-lock-syntactic-face-function

#'python-font-lock-syntactic-face-function)

(start (max pmark (point))))

   (with-syntax-table python-mode-syntax-table

     (font-lock-flush start limit)

     (font-lock-ensure start limit))))))

(setq-local font-lock-keywords '(python-shell-multiline--apply-font-lock)

   font-lock-keywords-only nil

   syntax-propertize-function python-syntax-propertize-function)

(setq python-shell-multiline-font-lock-keywords

      (symbol-value

       (font-lock-choose-keywords python-font-lock-keywords

 (font-lock-value-in-major-mode

  font-lock-maximum-decoration))))

The one curious thing: it definitely requires calls to both font-lock-flush and font-lock-ensure.  Otherwise the input isn’t fontified.  Tracing through, it seems when 'font-lock-mode is enabled, 'font-lock-flush is set to `jit-lock-refontify`, which simply clears the ‘fontified property, presumably expecting it to be noticed and re-fontified.  Not sure if this is the correct/most efficient approach.

It seems to me this general technique could be useful for lots of different “mixed fontification buffers”, simply rebinding font-lock-keywords/syntax table/etc. as needed for the region under consideration. It’s much simpler than the round-trip copy + full buffer re-fontify that many modes use, with no extra buffers to manage, post-command-hooks, etc.  There may be race conditions for highly dynamic buffers, but it’s working well for this usage case. 

Thanks again.

On Apr 11, 2021, at 5:10 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

Python-font-lock-keywords is a variable defined in ‘python.el’.
Its value is
(python-font-lock-keywords-level-1 python-font-lock-keywords-level-1 python-font-lock-keywords-level-2 python-font-lock-keywords-maximum-decoration)

Not sure why jit-lock-function would be evaluating it like an sexp.

`python-font-lock-keywords` does not hold a valid value for use on
`font-lock-keywords` but a value to be used as the first element of
`font-lock-defaults`.  This "first element" is used to initialize
`font-lock-keywords` but it depends on things like the
`font-lock-maximum-decoration`.


       Stefan

On Apr 11, 2021, at 12:31 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

But then, why bother round-tripping text out to a special-use buffer anyway,
vs. just letting font-lock operate in-situ in the shell buffer itself using
python-mode’s fairly simple font-lock-defaults. The only thing needed to
make this work is asking font-lock to ignore all the text with ‘field of
‘output?  

Maybe you can try something like the following?

  (defvar python--font-lock-keywords ...)
  (defvar python-font-lock-keywords
    '(python--apply-font-lock))
  (defun python--apply-font-lock (limit)
    (while (< (point) limit)
      (let ((next-boundary (find-next-boundary limit)))
        (if (we-should-skip-this-block)
            (goto-char next-boundary)
          (let ((font-lock-keywords python--font-lock-keywords))
            (font-lock-ensure (point) limit))))))


-- Stefan