Re: CC Mode and electric-pair "problem".

[Stefan Monnier]

>> >> Why?    How 'bout:
>> >>     char foo[] = "some unterminated // string
>> > Bug compatibility with the current scan-sexps.
>> I don't see why: currently, scan-sexps skips over the comment, but
>> that's not a bug: it's exactly what it is documented to do.
> There is no comment there, but scan-sexps skips to it nevertheless.

The starting point is within the string (not according to the C language
rules, of course, but according to the syntax-tables settings), and
operations like scan-sexps are documented to work under the assumption
that the starting point is outside of strings/comments, so it is very
much correct for it to consider this "// string\n" to be a comment.
I agree that it would be OK for scan-sexps in this case to consider that
\n terminates the string rather than the comment, tho.

>> When you change the syntax property of ?\n to be "> s", it changes the
>> behavior expected based on the documentation, ....
> Er, documentation?  This new flag isn't documented yet, or at least not
> in any permanent fashion.

Well, I was talking hypothetically under the assumption that "s" is
documented to mean something like "closes a string if there's one to
close".

>> .... so in the above case it should treat the \n as closing the string
>> rather than closing the comment.
> I agree.

OK, sorry 'bout the above, then, I see we agree.

>> It needs to work reliably for those languages where strings
>> are indeed terminated by newline (e.g. jgraph-mode in GNU ELPA).
> You mean, jgraph-mode is another use-case for `s'?  (I'm not familiar
> with it.)

I looked for existing use-cases and I indeed found one.  It's very much
not high-profile, tho.  Also this use-case is slightly different in that
the \n is really the normal/only way to terminate the string in jgraph.
In case you're interested:

    http://web.eecs.utk.edu/~plank/plank/jgraph/jgraph.html

>> But that brings us back to "why not use string-fence?".
> Yes.  String-fence interferes with syntactical stuff "inside" the
> invalid string, whereas the `s' flag won't.

Not sure how serious this "interferes with syntactical stuff" is
in practice.

>> But we usually don't make any effort to guess what the intended
>> closest valid state might be, except where the user is actively
>> editing the text (e.g. by proposing completion candidates for
>> identifiers).
> There's no need to guess.  The compiler defines the state, namely that
> the (invalid) string ends at the EOL, and what follows is non-string.

The compiler just makes an arbitrary choice, just like we do and that
has no bearing on what the intended valid state is (which is not
something the compiler can discover either: it's only available in the
head of the coder).

> There's no need to guess.  back_maybe_comment is in the new
> scratch/fontify-open-string branch.  It is NOT that complicated.

Unsurprisingly it introduces a complexity which I find unjustified by
the presented benefits.

But it now occurs to me that maybe we can do better: have you tried to
merge back_maybe_comment into back_comment?  After all, back_comment
already pays attention to strings (in order to try and correctly
handle comment openers appearing within strings), so there's
a possibility that back_comment might be able to handle your use case
with much fewer changes (and in that case, the performance cost would
be pretty close to 0, I think).


        Stefan