bug#66732: tree-sitter fontification doesn't update multi-line syntax reliably

[Yuan Fu]

> On Dec 12, 2023, at 7:45 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
>>> Anyway, could you try my patch? Like I said, I'm not sure if the
>>> insufficient fontification I'm observing in c-ts-mode is due to the
>>> problem with the solution, or due to the other redisplay-related problems
>>> on my system.
>> 
>> Yeah, it doesn't solve the problem in c-ts-mode regarding block comments.
> 
> I must admit I don't understand what the problem may be here.

The problem is, when you type /* RET foo RET */ the block comment is not 
fontified in comment face

That’s because before the user types the final “/“, there isn’t a complete 
comment node in the parse tree. When the user types the “/“, we need to mark 
the whole block comment for refontification.

We actually have that, when the parser reparses, it’ll also compute the 
affected region, the region that changed during the last reparse, and it’ll 
call the “notifiers” with that region. We install a font-lock-notifier, which 
simply sets fontified text property to nil in that region, so redisplay would 
call jit-lock to fontify that region.

In our example, the region would be the block comment.

>> We might need to run (progn (force-parse) (update-ranges) (force-parse))
>> before jit-lock-fontify-now and sytax-ppss.
> 
> Why would we need that?

The first force-parse is just to make the parser reparse the latest buffer 
content, then we need to update ranges of any embedded code, then we would want 
to make the parser for the embedded language to reparse, if their range has 
changed. After this process, all the parsers has the up-to-date parse tree, and 
more importantly have called their notifiers, those notifiers should apply 
fontified text prop, and apply any syntax text prop.

Yuan