bug#71345: Feature: unleash font-lock's secret weapon; handle Qfontified = non-nil

[Stefan Monnier]

>>> That starts to sound like a lot of property slinging, which might even
>>> dominate the work done.
>> Indeed, this amount of work could become significant.  It's my main
>> worry, but I don't have a clear feel for how serious it would be
>> in practice.
> In my situation, the most likely scenario is that fontified=nil is noticed
> during redisplay when there is a fairly large stretch of already-fontified
> property having the same value.  So jit-lock-fontify-now will quickly find
> a nice large chunk to call my FONTIFICATION-FUNCTION=F-F with.
>
> Since jit-lock-after-change will likely clear away already-fontified and set
> fontified=nil, a single additional F-F on top of jit-lock-function will
> probably be very well handled.  A good question is how it would scale with
> more functions all operating in the same region.  One idea is to rig up
> a test file, do some fake jit-lock-flushing on it, and check performance of
> just subtracting/searching/dividing the already-fontified property as you
> add more (fake) F-F's.   For me, jit-lock-fontify-now of a 2500 char chunk
> in a heavy treesitter buffer is in the 2-5ms range.  Individual F-F's could
> be much lighter weight.

I must say that I can't follow you.  I suspect we're not talking about
quite the same thing.  Could you clarify what is the costs you imagine
could be significant?  What you compare it to?

You seem to be comparing "a single big jit-lock backend" vs "several
jit-lock backends", which is a completely different worry from mine.
Splitting a backend into several backends comes with many more issues
(such as the issue of fighting over which one controls which properties,
or removing internal dependencies such that none of them needs to look
at the properties set by the others, ...) but that seems largely
orthogonal to the question at hand: if you want to be able to refresh
the position-dependent highlighting separately from the rest of the
highlighting you need that position-dependent highlighting to be
independent anyway (e.g. you need to be able to remove it without
affecting the position-independent highlighting).

> But things like `text-property-any' will be quickly defeated by the
> combinatorics of a large F-F set.

`text-property-any` only tests `eq`ness so it works just as quickly with
a property made up of a million-element list as with a property made of
a boolean.

IOW, I again can't follow you.

> So here's an idea.  You could invert the logic, and have a set of
> `fontified-pending' properties which jit-lock-flush adds to as it sets
> fontified=nil,

Yes, of course, we could use the complement set.


        Stefan