Re: api.location.type is unsupported for C parsers

[Scheidler , Balázs]

Hi,


On Thu, Apr 25, 2019 at 6:13 PM Akim Demaille <address@hidden> wrote:

> Hi Balázs!
>
> > Le 25 avr. 2019 à 07:03, Scheidler, Balázs <
> address@hidden> a écrit :
> >
> > I've tried this now and api.location.type does seem to work.
>
> That's good to know, I'll install that patch in master.
>
> > I still have some other compilation issues to hunt. One of those is on
> "enum XXX_tokentype". Since I am using the same tokens in multiple
> grammars, I get duplications.
>
> Doh.  That one is a tricky one :(
>
>
I resolved this by defining MAIN_TOKENTYPE right within my main parser,
which will avoid the enum completely, getting rid off the collisions.


> > modules/native/native-grammar.c:279:25: error: expected identifier
> before numeric constant
> >  #define LL_CONTEXT_ROOT 1
> >                          ^
> > lib/cfg-grammar.h:107:5: note: in expansion of macro ‘LL_CONTEXT_ROOT’
> >      LL_CONTEXT_ROOT = 1,
> >      ^~~~~~~~~~~~~~~
> >
> > The native-grammar file is a "slave" grammar that reuses some of the
> tokens of the main one. cfg-grammar.h is the main grammar. At the end both
> grammars have overlapping "tokens", but then the 2nd includes the header of
> the first. I am not sure how to resolve this yet, but I'll look at it in
> the coming days.
>
> What is really specific here is that I suppose that you do use a single
> scanner for all the grammars, right?  So do want to use the same names in
> every parser, right?
>

I am. I am basically trying to extend the main grammar with plugins, and
bison/yacc has never been intended to do that. The plugin has its own
grammar file, which get called from a plugin "injection point" in the main
grammar. The plugin specific grammar reuses a lot of rules of the main one
and is using the same lexer.

For instance, here's my destination plugin lookup logic:

```
dest_plugin
        : LL_IDENTIFIER
          {
            ... # some code omitted for clarity
            p = cfg_find_plugin(configuration, context, $1);
            $$  = (LogDriver *) cfg_parse_plugin(configuration, p, &@1,
NULL);
            ...
          }
        ;
```

cfg_parser_plugin() will call the parse function supplied by the plugin,
passes on the lexer instance until that grammar succeeds. If it does, it
will yield a destination driver instance that we can use just as if we
parsed that in the main grammar.

The plugin has a grammar like this (from the file destination):

```
start
        : LL_CONTEXT_SOURCE source_affile                        {
YYACCEPT; }
        | LL_CONTEXT_DESTINATION dest_affile                     {
YYACCEPT; }
        ;
```

where the source_affile rule will yield the driver instance and returns it
to the main grammar. The LL_CONTEXT_SOURCE/DESTINATION tokens are
artificially inserted into the lexer stream, so we can steer the plugin
parser in the right direction, as a single grammar will parse multiple,
related objects. The source and destination file drivers for instance, this
way I only need one grammar per plugin.



>
> If you want the tokens to be disjoint, that's really easy: use different
> values of api.token.prefix.  But here you want token types that are
> different but compatible, right?
>

yup. that's why I explicitly set the token values to make sure they are the
same every time they get included.


>
> Can you tell me more about how you deal with the different scanners?


See above.

Cheers,


Bazsi