bug#24870: 26.0.50; parse-partial-sexp ignores comment-end

[Alan Mackenzie]

Hello, Noam.

On Tue, Dec 13, 2016 at 11:04:45PM -0500, npostavs@users.sourceforge.net wrote:
> npostavs@users.sourceforge.net writes:


> > I have tracked the issue down to scan_sexps_forward in syntax.c

> Applying this change which reverts part of [1] seems to fix the issue:


> --- i/src/syntax.c
> +++ w/src/syntax.c
> @@ -3192,7 +3192,11 @@ scan_sexps_forward (struct lisp_parse_state *state,

>    while (from < end)
>      {
> -      if (SYNTAX_FLAGS_COMSTART_FIRST (prev_from_syntax)
> +      INC_FROM;
> +      code = prev_from_syntax & 0xff;
> +
> +      if (from < end
> +          && SYNTAX_FLAGS_COMSTART_FIRST (prev_from_syntax)
>         && (c1 = FETCH_CHAR (from_byte),
>             syntax = SYNTAX_WITH_FLAGS (c1),
>             SYNTAX_FLAGS_COMSTART_SECOND (syntax)))
> @@ -3213,8 +3217,6 @@ scan_sexps_forward (struct lisp_parse_state *state,
>       }
>        else
>          {
> -          INC_FROM;
> -          code = prev_from_syntax & 0xff;
>            if (code == Scomment_fence)
>              {
>                /* Record the comment style we have entered so that only



> Alan, can you explain what the idea behind that change was?

We're talking about 9dcf5998935c8aaa846d7585b81f0dcfe1935b3d from Sun
Mar 20 13:19:48 2016 +0000, still?

The idea is that in a (parse-partial-sexp from to), the end position
might be in the middle of a two character comment marker, such as "/*".
Before this change, it was impossible successfully to use the result of
that operation as the old state for continuing parse-partial-sexp from
that position, since it did not contain enough info to see it was in a
comment after passing the "*"

The change 9dcf599 added an extra element onto the parse state which was
non-nil when we end up after a "/", etc.

> I think it might correspond to this part of the commit message:

>     Reformulate code at the top of the main loop correctly to
>     recognize comment openers when starting in the middle of one.

> [1]: 9dcf5998935c Sun Mar 20 13:19:48 2016 +0000
> "Amend parse-partial-sexp correctly to handle two character comment 
> delimiters"

With the above in mind, I think we should both check your proposed patch
to make sure it doesn't break the working of 9dcf599.

Thanks for Cc'ing me in on this.

-- 
Alan Mackenzie (Nuremberg, Germany).