Re: User Token Numbers

[Hans Aberg]

At 09:34 +0200 2002/04/10, Akim Demaille wrote:
>| The lexical analyzer is not supposed to return the error token, so
>| that probably explains why historically the token was not in the .h
>| file.  The standard does not explicitly prohibit putting `#define
>| YYERRCODE whatever' into the .h file, so I guess it'd be OK to do that
>| if there's a good reason to.
>
>As noted by Hans, providing a means for the scanner to report an error
>to the parser can be useful.  But then, the question is returning
>`undefined', or `error'.

I noticed three different error situation reported by the lexer to the parser:
1. The lexer discovers an error whose nature that the parser should report.
Thus, the parser should call yyerror.
2. The lexer discovers an error whose nature it reports, but it wants to
make use of the parsers error recovery system.
3. The lexer gets a character it cannot decide whether it is parsable or
not. Thus, when the parser discovers that it neither handles it, it will
report "undefined".

Following the idea of C++ exception handling, one idea that comes to my
mind is to tokenize errors. One would then be able to write
  %error <token-name>
When <token-name> appears in the token input, it skips forward (unwinds the
parser stack) until it finds corresponding in the grammar to a line, either
only error or %error <token-name>, where <token-name> matches the thrown
error.

With this feature in hand, one might declare say
  %error lexer_error
and let the lexer return lexer_error. One should then have a means in the
%error declaration to tell that the yyerror function should not be called
before error recovery (but alternatively, one can put in the token number
as an argument of the yyerror function, and write code that ignores certain
types of errors).

  Hans Aberg