Another error handling (Was: Reductions during Bison error handling)

[Akim Demaille]

Paul H. said:

> As further evidence that perhaps something odd is going on, here is an
> example where there is a parse error; it is recovered from; but 
> Bison never uses the error production!
> 
> 
>     prog: 'x' 'y' 'z' ';' { printf (" x y z ;\n"); }
>       | r error ';' { printf (" *err*\n"); }
>       | r 'x' ';'  { printf (" x x ;\n"); }
>       | q ';'
>       | q '.' 
>       ;
> 
>     q : 'x' 
>       ;
> 
>     r : 'x' 
>       ;
> 
> 
> With the input 
> 
>      x y x ;
> 
> we get the output
> 
>      Error: parse error
>      x x ;
> 
> indicating that 'error' was never shifted. 
> 
> Why?  Well, when we pop the stack back to the first 'x' and take the 
> reduction r : 'x' (which is indicated when the lookahead token is
> error),  the algorithm I sent in the last message has us "proceeding
> normally", meaning that we use the current lookahead symbol ('x').

If we were to try to reimplement the somewhat more powerful scheme
that was (almost) implemented before, and to fix it, doesn't your
example demonstrate that we simply need to ``yyunlex'' the lookahead
when we find an error, and pretend the current lookahead is error?  It
seems to answer this scenario.  But maybe it's too naive.

Then, you also reported:

> Ah! Now I see what problem the yyerrdefault code-suppression was
> trying to fix.  So now I can amend my comment to say that I think a 
> different fix might be indicated.  
> 
> Here's the issue.  If our erroneous configuration is
> 
>     Parse Stack:  s1  s2 ... sk ... sn  <-top   Lookahead: X
> 
> we pop back to sk, and if the indicated action is "reduce", we take
> the reduction and proceed normally.  Now, if that reduction is a
> non-defaulted reduction (again, I don't mean default ACTION, but
> default reduction, mediated by yydefact), we are guaranteed that X can
> eventually be shifted.  If, however, the default reduction was used
> because X is an erroneous symbol in state sk, then we will eventually
> return to the error-recovery code, throw away X *and then continue to
> pop the stack*, turning mere "panic-mode" into "full-fledged
> rout-mode".  

This analysis seems to suggest the same fix.