Re: error-tokens and locations?

[Paul Eggert]

Ian Badcoe <address@hidden> writes:

> Does YYLLOC_DEFAULT get called with N=0 in the new version?

Yes.

> Were empty locations arising from the middle of the input already
> automatically generated

Yes, for some grammars, in that YYLLOC(..., ..., 0) was being called
for empty nonterminals.

> (and are they defined as start = end
> which makes them work perfectly with the standard YYLLOC_DEFAULT)?

Yes, though it's not strictly start==end; it's actually more like
end<=start.

> What happens if an empty location needs to be generated for the end
> of the input?  Or does it not happen?

I think it works just as the middle of the input, except that you use
the location of $ (i.e., the location that yylex returns when it
returns an EOF) instead of the location of the next ordinary token.

> > > If you add the concept of empty locations, doesn't anybody who redefined
> > > YYLLOC_DEFAULT have to revisit their code?
> >
> >Sure.
> 
> I meant "revisit" in the sense that their old code would now be broken,
> but I think I misunderstood?

All I meant was that the author of YYLLOC_DEFAULT has to make sure
that it works correctly with empty nonterminals, so they need to revisit
their code if they haven't ensured that it works already.

> >You can see an example of one way to handle this problem in
> >src/parse-gram.y's lloc_default function.  Admittedly it's not
> >pretty.
> 
> I hadn't grasped that empty locations still had a position. I was thinking
> that the same "empty" values were going to arise wherever in the input
> the location was.
> 
> I've looked at that function and I wonder why it is necessary?
> 
> If an empty location just has start = end then surely they will
> merge happily just the same as anything else?

Yes and no.  Suppose the nonterminal N produces the string "e f g h i
j" as part of the larger program "a b c d e f g h i j k", and suppose
N does so via the production "N: O P", where O produces the empty
string.  Here is a layout of a program's terminals and nonterminals O
and P, along with the possible locations for N:

a b c d         e f g h i j k
       OOOOOOOOOPPPPPPPPPPP
       NNNNNNNNNNNNNNNNNNNN     (1)
                NNNNNNNNNNN     (2)

Both (1) and (2) are "correct" in some sense, but most users would
prefer (2).  src/parse-gram.y's lloc_default ignores the empty
nonterminal O at the start of "N: O P" precisely so that we can get
(2) instead of (1).