Re: scanner_cursor and yylloc

[Paul Eggert]

[Bringing up an old conversation -- my am I behind in email!]

> From: Akim Demaille <address@hidden>
> Date: Mon, 31 Mar 2003 11:34:11 +0200

> Actually, I never quite understood why you add the missing
> terminator, I usually do this:
> 
> <STATE_STRING>{ /* Handling of the strings.  Initial " is eaten. */
>      \" {
>        BEGIN INITIAL;
>        return STRING;
>      }
> 
> // [...]
>      <<EOF>> {
>        std::cerr
>        << *yylloc << ": unexpected end of file in a string" << std::endl;
>        exit_set (exit_scan);
> 
>        BEGIN INITIAL;
>        return STRING;
>      }
> }
> 
> I don't see what rescanning brings.

In the actual parser, the shared code was more complicated than "BEGIN
INITIAL; return STRING;" and the two almost-copies got out of sync.  I
wanted to avoid code duplication, so I had the <<EOF>> case unput the
missing terminator in order to fall through to the \" case.

Unfortunately, as Peter Muir noted, unput was not portable, so in
<http://mail.gnu.org/archive/html/bison-patches/2003-04/msg00005.html>
I reverted to the method that you suggested above -- except that I
merely had the EOF case do "BEGIN INITIAL;" (without the other part of
the almost-copied code) so that the unfinished string is not reported
back to the parser.

This issue is discussed in my April 26 letter
<http://mail.gnu.org/archive/html/bison-patches/2003-04/msg00014.html>
where you suggested returning the token anyway (as above) but I
plumped for the easier-to-maintain approach of merely discarding the
unfinished string.  I don't think the change in behavior matters to
users, as we're merely talking about exactly which diagnostics are
issued for invalid Bison programs.