Re: Last steps for pretesting (font-lock-extend-region-function)

[Stefan Monnier]

>>>> I'm not saying the hook is a kludge: I'm saying the code that uses it is
>>>> a kludge, compared to the code that uses font-lock-multiline.

>>> OK.  I feel that existing code in AWK Mode is natural.  (OK, I wrote it,
>>> so I'm not in a position to judge it dispassionately, but ...).  It notes
>>> the logical EOL before a change.  It notes the logical EOL after a
>>> change.  The later one is the end position for fontification.
>> Why doesn't the BOL before change matter?
> Maybe it does/would do in some major modes.  It happens not to have any
> effect in AWK.
>> Why does the EOL before change matter?
> Because a buffer change can alter this EOL's syntactic context.

But why does it matter?
Honestly, neither explanation seems very clear to me.  I don't know what
others think, but this is an example of the complexity in your code.
You end up having to discuss various scenarios to justify your code, which
leaves the reader with the lingering question "but are all the relevant
scenarios covered?".

Clearly, we 're biased, but I find that the code using font-lock-multiline
doesn't suffer from such problems.  It does suffer from the fact that you
have to trust font-lock's use of the font-lock-multiline property, but if
you assume it works, then all you have to do is "place the property on the
chunk of text that needs to be refontified if any part gets changed".

> font-lock-keywords.  The purpose of this structure is to fontify strings
> matching specific regular expressions.  Using it to expand the region to
> be font-locked at a future fontification seems utterly disconnected with
> this purpose.

My sample code using the ".*\\(\\\\\n.*\\)*" regexp was just a quick way to
show you what it could look like without having to delve into your code.
Check my cc-awk patch: it shows what typically happens in practice, which is
that you place the put-text-property right where you add the face (or
syntax-table property), so it's a lot more natural.

>> There two issues about multiline elements:
>> - properly notice them when they appear
>> - properly *re*fontify them when they change

> I cannot bring myself to consider the extension of a font lock region as
> "the manipulation of a multiline element".

I don't understand what you're saying here.

> It just feels too contorted.  I have always understood the term "multiline
> element" to mean an element within the font-lock region which crosses
> a line boundary.

IIUC, that's how I understand it as well, except I wouldn't restrict it to
the "font-lock region".

>> The first is taken care of by font-lock-fontify-region-function.

> The AWK Mode regular expressions, even those that can straddle line
> breaks, match just fine without having to turn on the f-l-multiline
> variable.

I don't see how my text and yours are related here.

>> The second talks about *re*fontification, so it's natural that it would
>> involves saving some info during fontification for the next time around.

> I think I would agree, except that font-lock brutally erases these
> properties at every fontification, only to have to recalculate them.
> This wastes processor time, probably quite a lot of it.

No it doesn't *particularly* "waste processor time quite a lot":  it's just
what font-lock does for a living: erase text-properties and the re-add them
(usually it only does it for `face', sometimes it does it for `syntax-table'
and/or `font-lock-multiline', and it can do that for other properties (see
font-lock-extra-managed-props)).

> What is the point of doing this, unless there has been a buffer change?

If font-lock really happens to remove a font-lock-multiline property, it
means that somebody placed it there before.  Since this property is normally
placed during font-locking, it implies that we're actually *re*fontifying.
*Re*fontification normally takes place in response to a buffer change.

So, the "removal + re-addition" of the font-lock-multiline property (just
like any other property managed by font-lock) indeed only happens if there
has been a buffer change.

>> Doing it in a-c-f is fundamentally not the right place since it doesn't
>> know whether and how the changed text was fontified: it works, of
>> course, but it's not particularly more natural.

> To know how text is to be refontified, you need to know how it has been
> changed.

The font-lock-multiline property is a "living" proof that this is not true.


        Stefan