Re: cc-mode fontification feels random

[Stefan Monnier]

> I've been thinking of a new core facility for helping modes implement this
> kind of incremental buffer analysis. Basically, it works like this:
> fontification logically proceeds from bob to eob in fixed-size chunks. After
> each chunk, we checkpoint the state of the fontification engine in a text
> property. Whenever we modify the buffer, we invalidate chunks that the
> modification might have affected and proceed from the last
> known-valid checkpoint.

[ I assume that what you mean by "fontification" is not literally
  placing faces (which is typically what font-lock does), but only
  a subset of that job (the subset that needs to proceed sequentially
  from BOB).  ]

You mean like what we do for `syntax-ppss` (except we keep the
checkpoint data in an alist indexed by positions, rather than in
text-properties)?

I think it would be fairly easy to add some way to keep extra data in
`syntax-ppss-wide/narrow`.

> It's more subtle than it sounds though.
>
> First, we need to support lookahead. Fontification of region [A, B) might do
> lookahead and depend on text in region [B, C).

For `syntax-propertize` we handle this via a `syntax-multiline` text
property, so that changes in the B region cause re-propertization of the
A region.

> Second, writing fontification as some kind of callback with explicit
> checkpoint and restore support is annoying, and nobody's going to do
> that. If it were possible to write fontification programs as coroutines, we
> would keep mode fontification routines simply and declarative and
> automatically do both the chunking and the checkpointing.

When I wrote the `syntax-ppss` code, I did expect to add facilities to
keep extra data in there, but so far the need has not really come up.


        Stefan