bug#71345: Feature: unleash font-lock's secret weapon; handle Qfontified = non-nil

[JD Smith]

On Jun 3, 2024, at 12:56 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

In my particular case, I've been trying for several months to achieve
a scheme for position-dependent additional fontification based on treesitter
scope: think font-lock, but respondent not just to the contents of the
buffer, but also the location of point within it.  My particular application
is drawing treesitter scope-aware indentation bars.  There are many other
applications you could envision, including many discussed here
(e.g. bug#22404).

I've been meaning to add support for that to jit-lock.
Here's the design I have in mind:

From the outside, only one change, i.e. a new function

   (jit-lock-flush BEG END FONTIFICATION-FUNCTION)

where FONTIFICATION-FUNCTION is the exact same function that was passed
to `jit-lock-register` (and is thus used as a kind of identifier of the
corresponding backend).

So in your case, when the position-dependent highlighting needs to be
updated, you'd request it by calling `jit-lock-flush`.

The way it would be implemented inside `jit-lock.el` is that
`jit-lock-flush` would set `fontified` to nil over that whole region,
but before that it would scan the region for those places where
`fontified` is non-nil, and set the `jit-lock-already-fontified` property
to the list of currently active backends (minus FONTIFICATION-FUNCTION),
so as to remember that while this region does require jit-lock action
those backends don't need to be called.

Thanks.  Very interesting approach, recycling the existing Qfontified handler.  I wonder though, if you have several different unrelated functions calling `jit-lock-flush' with different FONTIFICATION-FUNCTION's prior to jit-lock-fontify-now running on a region, won't they step on each other?  I.e it seems that you would need to look for existing jit-lock-already-fontified+fontified=nil properties over the region mentioned in jit-lock-flush and subtract the passed FONTIFICATION-FUNCTION from the various values already found there.  Of course you'd also need to handle the case where you "subtract it all the way to nil".  That starts to sound like a lot of property slinging, which might even dominate the work done.   I don't guess there is a fast hidden

 (remove-from-text-property-lists beg end property val-to-remove)

You could end also up with a quilted patchwork of different levels of the already-fontified property before you clear them, which might make jit-lock-fontify-now's job harder.  Although for my particular case I doubt this will be a problem:

1. jit-lock-after-change will set fontified=nil (and btw, will probably also need to clear the already-fontified property over the affected change region).
2. My tree-sitter based code, upon edits and position-based scope changes, will use jit-lock-flush to add already-fontified properties to the affected locations where fontified=t (if any).

I imagine that the functions may also need a way to opt-out of "deferred contextual refontification", for example if they add some other properties/overlays orthogonal to face.  Perhaps this could be done by having jit-lock-context-fontify setup the "rest of the buffer" with already-fontified = jit-lock-no-context-functions — an optional list of FONTIFICATION-FUNCTION's that don't require contextual refontification.