Re: Bison 3.2.90 released [beta]

[Akim Demaille]

Hi Derek,

> Le 13 janv. 2019 à 23:40, Derek Clegg <address@hidden> a écrit :
> 
> It appears that “api.prefix” changes the definition of YYSTYPE:
> 
> %define api.prefix {cmap_yy}
> …
> union CMAP_YYSTYPE
> …
> 
> I don’t know whether this is a new change or not. I switched to “api.prefix” 
> due to this error:
> 
> cmap-parser.y:71.1-12: error: deprecated directive, use ‘%define api.prefix’ 
> [-Werror=deprecated]
> %name-prefix "cmap_yy"
> ^^^^^^^^^^^^
> 
> Unfortunately, the corresponding option in flex does not have the same 
> behavior: flex-generated files, even with the corresponding flex option set
> 
> %option prefix="cmap_yy"
> 
> refer to YYSTYPE, not CMAP_YYSTYPE, at least as of flex-2.5.35.

I don't think flex's scanner depend on YYSTYPE.  Flex does not know anything 
about yylval and yylloc.  In fact you have to define YY_DECL to tell flex what 
the signature of ylex is, and therefore the names and types of the arguments 
for the value and the location.

So I venture that somewhere you have something like

#define YY_DECL int yylex (YYSTYPE* yylval, YYLTYPE* yylloc)

which should become

#define YY_DECL int cmap_yylex (CMAP_YYSTYPE* yylval, CMAP_YYLTYPE* yylloc)


> This makes it not non-trivial to set an API prefix for flex-generated and 
> bison-generated files.

I agree the change is more demanding than just updating the grammar file, you 
also need to adjust YY_DECL.  I personally put YY_DECL in the %code provides 
section, so in the end I still only need to update the grammar file, but your 
situation might be different.

I am unhappy that this change requires more changes, however, I really believe 
that the previous situation was wrong: if you have several parsers, 
%name-prefix allows symbols in the objects to not collide, yet you cannot 
include their headers in a single compilation unit, as the types would collide. 
 IMHO, %name-prefix was buggy.

I should have provided some doc for this.  What do you think about the 
following patch?

diff --git a/doc/bison.texi b/doc/bison.texi
index c66a9746..5a724a98 100644
--- a/doc/bison.texi
+++ b/doc/bison.texi
@@ -6650,6 +6650,28 @@ For example, if you use @samp{%define api.prefix 
@address@hidden, the names become
 @code{cparse}, @code{clex}, @dots{}, @code{CSTYPE}, @code{CLTYPE}, and so
 on.
 
+Users of Flex must update the signature of the generated @code{yylex}
+function.  Since the Flex scanner usually includes the generated header of
+the parser (to get the definitions of the tokens, etc.), the most convenient
+way is to insert the declaration of @code{yylex} in the @code{provides}
+section:
+
address@hidden
+%define api.prefix @address@hidden
+// Emitted in the header file, after the definition of YYSTYPE.
+%code provides
address@hidden
+  // Tell Flex the expected prototype of yylex.
+  #define YY_DECL                             \
+    int clex (CSTYPE*y ylval, CLTYPE *yylloc)
+
+  // Declare the scanner.
+  YY_DECL;
address@hidden
address@hidden example
+
address@hidden 1
+
 The @code{%define} variable @code{api.prefix} works in two different ways.
 In the implementation file, it works by adding macro definitions to the
 beginning of the parser implementation file, defining @code{yyparse} as