Re: forward-comment and syntax-ppss

[Alan Mackenzie]

Hello, Dmitry.

On Fri, Dec 16, 2016 at 03:33:45AM +0200, Dmitry Gutov wrote:
> On 11.12.2016 12:17, Alan Mackenzie wrote:

> > I don't think this is an issue.  (syntax-ppss-1 POS), when
> > new-local-variable is non-nil simulates (parse-partial-sexp
> > new-local-variable POS), and the latter would throw an error for
> > precisely the same values of POS as the former.

> Indeed, but this way the first argument will be implicit, and, unlike 
> point-min, often non-obvious to the code calling syntax-ppss.

I thought the main idea of the facility was to provide a base point for
a sub-buffer in multi-major-mode modes.  In this case, the code calling
syntax-ppss isn't going to be using positions less than
new-local-variable anyway, it's going to be working entirely inside the
pertinent sub-buffer.

> It's easy to produce a buffer position lower than the arbitrary value
> of the new variable. It's harder to produce a buffer position lower
> than point-min.

The new variable wouldn't have an "arbitrary" (i.e. random) value.  It
would be carefully set by code which knows what it's doing. I can't see
how it would be easier (accidentally) to produce an invalid position for
syntax-ppss.

> Anyway, it's a minor issue. A good error message should be sufficient.

> > Maybe, but these changes are the direct desired effect of narrowing.
> > They're not nasty unforeseen side effects.

> "nasty" or "unforeseen" is beside the point. Like described previously, 
> some code doesn't really know how to behave in the face of narrowing if 
> the user's intent that resulted in narrowing, isn't known.

You keep saying this, but I can't remember any examples of what you mean
by "the user's intent ... not being known".  Can you give such an
example of how code would do things differently on the basis of
"knowing" what the "user meant"?  And I don't mean here the misuse of
(point-min) to mean "Position 1".  That is just a bug.

Narrowing is just narrowing, and is fundamentally clear and unambiguous.
That's why it has become such a powerful and well-used tool, both
directly by users, and by Lisp code.

[ .... ]

> > That may be the case, but that has nothing to do with what I said ("In
> > particular, it doesn't change ....").  Whether abc is inside a string or
> > not doesn't vary by the narrowing.  It's determined by other factors.

> It can be determined by the narrowing (i.e. a function named 
> `inside-a-string?' could confidently know whether to rely on the current 
> narrowing or ignore it, to produce its result, if narrowing were 
> annotated with intent).

Why would anybody need such a facility?  Could you give a concrete use
case, please?

> > In particular, I was thinking of what I think is
> > called "hard widening".  I think it would be a mistake to add this
> > feature.  At the moment, narrowing is a strong, direct, simple facility
> > - it does what it says it does and no more.

>  From where I'm standing, the things you use it for (hide buffer 
> contents) can be implemented in a much better fashion using overlays 
> with invisibility (in fact, one such implementation exists already), 
> which don't introduce the difficulties of changing point-min and 
> point-max.

What difficulties?

> Thus, without making things harder for many other facilities.

Narrowing is such a fundamental feature of Emacs that attempting to
supersede it by something else would be bound to fail, or at least be
very painful for a long time.  Maybe your invisibility overlays might
cover some use cases of narrowing, but they would be bound to fail on
others.

> > Were "hard widening" to be introduced, we could easily end up with a
> > confused mess of `widen' and `hard-widen' through the code, with
> > nobody being quite sure which one to use and why.  There might be
> > lots of depressing errors where the wrong one is used.

> These are issues of design and documentation. The previous discussions 
> of hard-widen didn't come up with any real conceptual problems. Just 
> with low-level implementation ones.

It would foul up much existing code.  It would be bound to.  There will
be code around which depends on widening meaning exactly "widen".  If you
change the meaning of `widen' to `(narrow-to-region some-start some-end)'
things will go wrong.

> >> More importantly, any change you would make to the "new" function you
> >> could make to syntax-ppss instead.

> > The new function would be a different function, i.e. it would have a
> > different definition.  Just for clarity, it would not be written from
> > scratch, it would be based on the current syntax-ppss.

> And then it would be utterly useless for third-party code, for years to 
> come. Further, if it uses a separate variable for its cache, and the new 
> function is used in the code, while the old function continues to be 
> used by third-party code (and also stable packages like Gnus and 
> CC-Mode) for compatibility reasons, we'll have two syntax caches that 
> will interact in fascinating, difficult to predict ways.

I don't agree with you on any of this.  Of course, the devil would be in
the details, but I do think the Emacs team has enough experience of
changes to be able to do this one in a non-destructive fashion.  Don't
forget, the current syntax-ppss doesn't and can't work properly.

> Am I interpreting your intention correctly?

I don't think so.  `syntax-ppss-1' would be usable in the same way as
`syntax-ppss', so a backward compatibility alias would leave things
working at least as well as they do at the moment.

> >> Further, the good part of this proposal is more or less equivalent to
> >> "indexing syntax-ppss cache by point-min" as proposed by Stefan.

> > I think I agree here.  But "good part" implies the existence of a "less
> > good part".  What, in your view, constitues this "less good part"?

> Well... both your and Stefan's proposals will end up actually having to 
> maintain the index of syntax-ppss caches, and having to invalidate those 
> values somehow. I'm sure it's possible to arrive at a satisfactory 
> algorithm (like only keeping around two values, for point-min equal to 1 
> and the most recent other value), but there must be use cases which some 
> other invalidation approach would be better.

That was precisely the mechanism I was looking at a few days ago.  I
agree it could probably be improved upon, perhaps after gaining
experience with it.

> With syntax-ppss-dont-widen, the caller code can take the responsibility 
> of maintaining the values of syntax-ppss-cache and syntax-ppss-last.

The calling code shouldn't even be aware of the internal details of the
syntax-ppss cache, far less be burdened with the responsibility of
maintaining it.  Such high coupling between syntax-ppss and its users
would make it pretty much impossible to change the cache mechanism at a
later date if we wanted to do that.  syntax-ppss should be as black-box
like as possible.

> And it turns out, it's rather easy to bind them to nil around the same
> pieces of code that starts with (save-restriction (narrow-to-region
> ...)). Which currently seems to give the same result as the idea in the
> previous paragraph, with much less code.

> In short, letting the caller code control cache invalidation has its 
> benefits.

They are by far outweighed by the disadvantages.

-- 
Alan Mackenzie (Nuremberg, Germany).