[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: User Token Numbers
From: |
Paul Eggert |
Subject: |
Re: User Token Numbers |
Date: |
Mon, 8 Apr 2002 11:17:45 -0700 (PDT) |
> From: Akim Demaille <address@hidden>
> Date: 08 Apr 2002 19:20:17 +0200
>
> While writing test cases I fell on some weird behaviors: 1.28 (I don't
> know for 1.3x, I don't have it at hand, but I'd say it behaves
> equally) accepts that you
>
> %token FOO 2
>
> although 2 is used for $undefined.
That is a clear violation of POSIX, I'm afraid.
> But I would like to know more from POSIX, in particular, I
> would like to make sure that EOF (= 0) is the only number that is
> reserved.
It depends on what you mean by "reserved". If you you mean "yacc
cannot assign these token numbers", then POSIX reserves all
nonpositive integers, all token numbers used for literals in the
grammar, and all user-assigned token numbers.
Here are the relevant quotes:
* "The end of the input is marked by a special token called the
endmarker, which has a token number that is zero or negative. (These
values are invalid for any other token.) All lexical analyzers shall
return zero or negative as a token number upon reaching the end of
their input."
* "All token numbers assigned by yacc shall be unique and distinct
from the token numbers used for literals and user-assigned tokens."
Other things from POSIX include:
* User-defined assigned token numbers must be positive.
* Yacc can assign a token number immediately when it sees a %token
declaration. However, in my opinion it'd be a bit nicer if it
waits till end of grammar, so that it doesn't assign a token number
that collides with a later user-defined token number.
* Users can't change a token number after it has been assigned.
* Literals must be single-byte. "Multi-byte characters should be
recognized by the lexical analyzer and returned as tokens. They
should not be returned as multi-byte character literals."
* "If duplicate token numbers cause conflicts in parser generation,
yacc shall report an error; otherwise, it is unspecified whether the
token assignment is accepted or an error is reported."
- User Token Numbers, Akim Demaille, 2002/04/08
- Re: User Token Numbers,
Paul Eggert <=
- Re: User Token Numbers, Akim Demaille, 2002/04/08
- Re: User Token Numbers, Paul Eggert, 2002/04/08
- Re: User Token Numbers, Akim Demaille, 2002/04/09
- Re: User Token Numbers, Paul Eggert, 2002/04/09
- Re: User Token Numbers, Akim Demaille, 2002/04/10
- Re: User Token Numbers, Hans Aberg, 2002/04/10
- Re: User Token Numbers, Paul Eggert, 2002/04/10
Re: User Token Numbers, Florian Krohm, 2002/04/08
Re: POSIX and reduce/reduce conflicts, Paul Eggert, 2002/04/08