Re: `YYERROR;' should not discard lookahead token

[Akim Demaille]

 >> From: Akim Demaille <address@hidden>
 >> Date: Sun, 02 Feb 2003 11:31:59 +0100

 >> | 2002-11-24  Paul Eggert  <address@hidden>
 >> | 
 >> |   * data/yacc.c (yyerrlab1): Move this label down, so that the
 >> |   parser does not discard the lookahead token if the user code
 >> |   invokes YYERROR.  This change is required for POSIX conformance.
 >> 
 >> Is there any test related to this feature?

 Paul> Sorry, no.

Hi Paul,

Sorry for the delays, once again.  Well, I have tried to write such a
test, and ended up with the following test:

foo.y
Description: Text document

But I cannot see the difference between before and after your patch.

This is after:

| /tmp % bison foo.y; gcc foo.tab.c -o foo; ./foo                  nostromo 
14:17
| Starting parse
| Entering state 0
| Reading a token: Next token is token 'a' ()
| Shifting token 'a', Entering state 2
| Reading a token: Next token is token '!' ()
| Reducing stack by rule 4 (line 16), 'a' -> a_b
| Error: popping token 'a' ()
| Stack now 0
| Shifting error token, Entering state 1
| Next token is token '!' ()
| Shifting token '!', Entering state 6
| Reducing stack by rule 3 (line 14), error '!' -> input
| OK
| Stack now 0
| Entering state 3
| Reading a token: Now at end of input.

And this is before.  Both exhibit the "OK", and both, after shifting
the error token, do consider the current lookahead: "!".

| /tmp % ~/src/bison-1.35/tests/bison foo.y; gcc foo.tab.c -o foo; ./foo
| Starting parse
| Entering state 0
| Reading a token: Next token is 97 ('a')
| Shifting token 97 ('a'), Entering state 2
| Reading a token: Next token is 33 ('!')
| Reducing via rule 4 (line 16), 'a'  -> a_b
| Error: state stack now 0
| Shifting error token, Entering state 1
| Next token is 33 ('!')
| Shifting token 33 ('!'), Entering state 5
| Reducing via rule 3 (line 14), error '!'  -> input
| OK
| state stack now 0
| Entering state 8
| Reading a token: Now at end of input.
| Shifting token 0 ($), Entering state 9
| Now at end of input.
| 

 Paul> POSIX says that the lookahead token is discarded only under the
 Paul> following conditions:

 Paul> (1) When a syntax error is detected by the parser

 Paul> (2) no normal input symbols have been shifted since the preceding error
 Paul> was detected

 Paul> (3) the lookahead symbol is not an endmarker

 Paul> The error in the old code was that its condition (1) was this:

 Paul> (1') When YYERROR is executed or when the parser detects a syntax error

I have not observed this either.  See for instance the following patch
that I'm applying.  It passes properly with older Bisons :(  Am I
missing something?

diffs.patch
Description: Text Data

To exercise older Bisons, I ran this:


  ./testsuite -d 61 
  ../../bison-1.35/tests/bison testsuite.dir/061/calc.y -o old.c
  gcc old.c -o old
  ./old < testsuite.dir/061/input

and I correctly get 2222 which means that the error recovery
post-YYERROR worked properly.  And running:

  echo "() + (1 + 1 + 1 +) + (* * *) + (1 * 2 * *) = 1" | ./old

gives also the same results with 1.35 or with CVS.