Re: [PATCH 0/4] Fix crash when generating IELR

[Akim Demaille]

Hi Dwight,

> Le 26 juin 2020 à 18:13, Dwight Guth <dwight.guth@runtimeverification.com> a 
> écrit :
> 
> Hi,
> 
> I can confirm that the version of bison downloaded from the link you provided 
> does not manifest the error in my real example, and valgrind no longer 
> reports any use-after-free errors on the larger example, so it looks like 
> it's been fixed.

Good!  I just installed the fixes in master.  Thanks!

> Since the grammar it was generated from originally, as well as the code that 
> generated it, is completely open source, I'd be happy to sign some kind of 
> waiver with respect to the intellectual property on the bison grammar I 
> provided you

Thanks, I'll send you a separate email for this.

> About the original grammar: It is an automatically generated bison file, but 
> it is generated from a manually written grammar written in a language called 
> K (https://github.com/kframework/k).

Interesting.  Thanks for the context.


> I figured it shouldn't crash, either...

Indeed :)

> Basically the way it is constructed is by taking the original grammar, 
> duplicating certain nonterminals in order to introduce an explicit precedence 
> relationship between different productions (ie, that certain productions 
> cannot appear directly under certain nonterminals in certain parent 
> productions), and then automaticlaly encoding it into bison.

I'm not sure I follow here.  Is this something resembling the way SGLR handles 
the precedence between rules? (not the old way, based on solving shift/reduce 
conflicts, which involves tokens, rather than rules only).


> This seems to work well in many cases on real-world programming languages, 
> but obviously it becomes possible to write arbitrary grammars that you would 
> have to use GLR in order to parse with any real accuracy.

Right.

> I am attaching the original bison grammar that the manually modified grammar 
> I sent you was derived from,

I see that you are using string aliases with loads of escapes, e.g.,

%token TOK_35 36 "(\\/\\*([^\\*]|(\\*+([^\\*\\/])))*\\*+\\/)"

This looks very much like a regex for C comments.  Do you use this string?  I 
really means the strings that Bison embeds in the generated parser (in 
yytname), or is this just informative, but the actual use of this regex done 
elsewhere?

I'm asking this because the handling of string aliases is changing currently, 
and I would need feedback on these changes.

Also, according to what I saw, you should consider using api.token.raw.


> but I think you will find that the minimized grammar probably works better as 
> a test case.

Yes, definitely :)

> I can also point you to the original grammar written in K if you are 
> curious...

Yes please!

Cheers!