[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Dynamic token kinds
From: |
Frank Heckenbach |
Subject: |
Re: Dynamic token kinds |
Date: |
Mon, 17 Dec 2018 18:37:56 +0100 |
Hans Åberg wrote:
> > What I suggest to add (without changing the above), is:
> >
> > symbol_type make_symbol (token_type type);
> > // checks at runtime that type is V_FOO or V_BAR
> >
> > symbol_type make_symbol (token_type type, int &&);
> > // checks at runtime that type is I_BAZ or I_QUX
> >
> > symbol_type make_symbol (token_type type, string &&);
> > // checks at runtime that type is S_BLA
> >
> > These runtime checks might be implemented via a switch if that's
> > easier to auto-generate (it might be in fact) or with a simple
> > "if (... || ...)" statement, that's an implementation detail.
>
> Actually, I pass the semantic value through a class object to
> which the lexer function belongs, so the extra arguments are not
> needed. So I must think more about this.
Do you actually use Bison's variant? Otherwise, what you do is
irrelevant here (sorry), as this is a proposal specifically about
Bison's variant.
> >>> It's not that bad actually. Again, my lexers work fine as is.
> >>> I just brought this up because Akim proposed to call the function
> >>> "unsafe_..." which I thought was too harsh and proposed
> >>> "unchecked_..." -- but by adding the checks, it would be neither
> >>> unsafe nor unchecked. :)
> >>
> >> This worries me.
> >
> > That's why I suggest to add the check. :)
>
> There must be some guard against programming errors, I think.
That's why I suggest to add the check. :)
> >> But also having having to use something more complex to be returned by the
> >> lexer than a value on the lookup table .
> >
> > The lexer returns a token which contains the token kind (an enum)
> > and the semantic value (a union value). As mismatch is bad. The
> > make_FOO functions avoid a mismatch and are suitable for statically
> > known token kinds. The direct constructor call can be used for
> > dynamic token kinds, but allows a mismatch. The functions I propose
> > to generate instead could be used for dynamic token kinds and avoid
> > a mismatch.
> >
> > Everything clear now?
>
> Yes, it is the requirement of returning the semantic value that
> causes the issue. Then this requirement is perhaps born out the
> condition of not storing the type in the Bison variant.
Building rather than returning. And sure, it's because Bison's
variant doesn't store the type, we know this.
Regards,
Frank
- Re: Dynamic token kinds, (continued)
- Re: Dynamic token kinds, Frank Heckenbach, 2018/12/16
- Re: Dynamic token kinds, Hans Åberg, 2018/12/16
- Re: Dynamic token kinds, Frank Heckenbach, 2018/12/16
- Re: Dynamic token kinds, Hans Åberg, 2018/12/16
- Re: Dynamic token kinds, Frank Heckenbach, 2018/12/16
- Re: Dynamic token kinds, Hans Åberg, 2018/12/16
- Re: Dynamic token kinds, Frank Heckenbach, 2018/12/17
- Re: Dynamic token kinds, Hans Åberg, 2018/12/17
- Re: Dynamic token kinds, Frank Heckenbach, 2018/12/17
- Re: Dynamic token kinds, Hans Åberg, 2018/12/17
- Re: Dynamic token kinds,
Frank Heckenbach <=
- Re: Dynamic token kinds, Hans Åberg, 2018/12/17
- Re: Porting to typed C++ parser (was: Dynamic token kinds), Frank Heckenbach, 2018/12/17
- Re: Porting to typed C++ parser (was: Dynamic token kinds), Hans Åberg, 2018/12/17
- Re: Porting to typed C++ parser (was: Dynamic token kinds), Frank Heckenbach, 2018/12/18
- Re: Porting to typed C++ parser (was: Dynamic token kinds), Hans Åberg, 2018/12/18
- Re: Porting to typed C++ parser (was: Dynamic token kinds), Frank Heckenbach, 2018/12/19
- Re: Porting to typed C++ parser (was: Dynamic token kinds), Hans Åberg, 2018/12/19
- Re: Porting to typed C++ parser (was: Dynamic token kinds), Frank Heckenbach, 2018/12/22
- Re: Porting to typed C++ parser (was: Dynamic token kinds), Hans Åberg, 2018/12/22
- Re: Porting to typed C++ parser (was: Dynamic token kinds), Frank Heckenbach, 2018/12/22