[PATCH] more on prologue alternatives

[Paolo Bonzini]

Hi,

my copyright assignment is on its way back to Boston.  In the meanwhile, 
I'll start posting my patches for review.

This is not absolutely necessary for the Java parser project, but (not 
unexpectedly) it suits its need very well -- both from the perspective 
of writing grammars, and from the perspective of teaching Bison next 
semester. :-)

here is my try on making the prologue directives more straightforward.

1) there are still four sections, that is:
   - before header
   - after header
   - beginning of header (%start-header)
   - end of header (%end-header)

2) the directives are two: %{...%} and %header { ... }

3) %{...%} maps to before header if it is seen before %union or after 
the first %%, otherwise it maps to after header.

4) %header { ... } maps to beginning of header if it is seen before 
%union or after the first %%, otherwise it maps to after header.

The key of the proposal is in this hunk of the patch:

-  /* Unlike the traditional Yacc prologue blocks, the output order
-   * for the %*-header blocks is not affected by their declaration
-   * position relative to any %union in the grammar file.  */
+  /* Like for the traditional Yacc prologue blocks, the output order
+   * for the %header blocks is affected by their declaration position
+   * relative to any %union in the grammar file.  Putting prologue
+   * or %header blocks in the grammar section (after %%) has the
+   * same effect.  */


The attached patch, regtested properly, implements this suggestion. 
Most of the changes to actions.at are actually needed only for test 63, 
but I preferred to keep the autotest code as readable as possible, hence 
I moved the declarations in the grammar section for all the tests.

%{...%} and %header {...} are allowed in the grammar section without a 
trailing semicolon.  This can be changed easily.  If it is not, I would 
propose to also allow %union {...} in the grammar section without a 
trailing semicolon.

What do you think?

Paolo

2006-10-03  Paolo Bonzini  <address@hidden>

        * doc/bison.texinfo (Prologue): Document %header instead of %*-header.
        (Decl Summary): Likewise.
        (Calc++ Parser): Likewise.
        (Table of Symbols): Likewise.

        * src/parse-gram.y (%before-header, %after-header, %start-header,
        %end-header): Remove.
        (%header): New token.
        (post_blocks): New variable.
        (input): Set post_blocks.
        (prologue_declaration): Remove rules for removed token.  Move
        "%{...%}"...
        (code_declaration): ... here.
        (grammar_declaration for "%union"): Set post_blocks.
        (rules_or_grammar_declaration): Allow code_declarations without
        a trailing semicolon.
        * src/reader.c (header_augment): New.
        * src/reader.h (header_augment): Declare.
        * src/scan-gram.l (%before-header, %after-header, %start-header,
        %end-header): Remove.
        (%header): New token.

        * tests/actions.at: Adjust for changes above.
        * tests/regression.at: Likewise.

Index: doc/bison.texinfo
===================================================================
RCS file: /sources/bison/bison/doc/bison.texinfo,v
retrieving revision 1.205
diff -u -r1.205 bison.texinfo
--- doc/bison.texinfo   11 Sep 2006 18:56:58 -0000      1.205
+++ doc/bison.texinfo   3 Oct 2006 09:19:58 -0000
@@ -2674,22 +2674,20 @@
 feature test macros can affect the behavior of Bison-generated
 @code{#include} directives.
 
address@hidden %before-header
address@hidden %start-header
address@hidden %after-header
address@hidden %header
 If you've instructed Bison to generate a header file (@pxref{Table of Symbols,
 ,%defines}), you probably want @code{#include "ptypes.h"} to appear
 in that header file as well.
-In that case, use @code{%before-header}, @code{%start-header}, and
address@hidden instead of @var{Prologue} sections
-(@pxref{Table of Symbols, ,%start-header}):
+In that case, use @code{%header} instead of @var{Prologue} sections
+(@pxref{Table of Symbols, ,%header}):
 
 @smallexample
-%before-header @{
address@hidden
+  /* Does not appear in the header.  */
   #include <stdio.h>
 @}
 
-%start-header @{
+%header @{
   #include "ptypes.h"
 @}
 %union @{
@@ -2697,7 +2695,8 @@
   tree t;  /* @address@hidden is defined in @file{ptypes.h}.} */
 @}
 
-%after-header @{
address@hidden
+  /* Likewise, does not appear in the header.  */
   static void print_token_value (FILE *, int, YYSTYPE);
   #define YYPRINT(F, N, L) print_token_value (F, N, L)
 @}
@@ -4327,11 +4326,9 @@
 and to the token type codes.  @xref{Token Values, ,Semantic Values of
 Tokens}.
 
address@hidden %start-header
address@hidden %end-header
-If you have declared @code{%start-header} or @code{%end-header}, the output
-header also contains their code.
address@hidden of Symbols, ,%start-header}.
address@hidden %header
+If you have declared @code{%header}, the output header also contains its code.
address@hidden of Symbols, ,%header}.
 @end deffn
 
 @deffn {Directive} %destructor
@@ -7560,18 +7557,18 @@
 @end example
 
 @noindent
address@hidden %start-header
address@hidden %header
 Then come the declarations/inclusions needed to define the
 @code{%union}.  Because the parser uses the parsing driver and
 reciprocally, both cannot include the header of the other.  Because the
 driver's header needs detailed knowledge about the parser class (in
 particular its inner types), it is the parser's header which will simply
 use a forward declaration of the driver.
address@hidden of Symbols, ,%start-header}.
address@hidden of Symbols, ,%header}.
 
 @comment file: calc++-parser.yy
 @example
-%start-header @{
+%header @{
 # include <string>
 class calcxx_driver;
 @}
@@ -7630,15 +7627,16 @@
 @end example
 
 @noindent
address@hidden %after-header
-The code between @samp{%after-header @{} and @address@hidden is output in the
address@hidden address@hidden
address@hidden address@hidden
+The code between @address@hidden and @address@hidden is output in the
 @file{*.cc} file; it needs detailed knowledge about the driver.
 
 @comment file: calc++-parser.yy
 @example
-%after-header @{
address@hidden
 # include "calc++-driver.hh"
address@hidden
address@hidden
 @end example
 
 
@@ -8308,49 +8306,28 @@
 Start-Symbol}.  It cannot be used in the grammar.
 @end deffn
 
address@hidden {Directive} %after-header @address@hidden@}
-Specifies code to be inserted into the code file after the contents of the
-header file.
address@hidden of Symbols, ,%start-header}.
address@hidden deffn
-
address@hidden {Directive} %before-header @address@hidden@}
-Specifies code to be inserted into the code file before the contents of the
-header file.
address@hidden of Symbols, ,%start-header}.
address@hidden deffn
-
address@hidden {Directive} %end-header @address@hidden@}
-Specifies code to be inserted both into the header file (if generated;
address@hidden of Symbols, ,%defines}) and into the code file after any
-Bison-generated definitions.
address@hidden of Symbols, ,%start-header}.
address@hidden deffn
-
address@hidden {Directive} %start-header @address@hidden@}
address@hidden {Directive} %header @address@hidden@}
 Specifies code to be inserted both into the header file (if generated;
address@hidden of Symbols, ,%defines}) and into the code file before any
-Bison-generated definitions.
address@hidden of Symbols, ,%defines}) and into the code file.
address@hidden of Symbols, ,%header}.
 
 @cindex Prologue
address@hidden %before-header
address@hidden %header
 @findex %union
address@hidden %end-header
address@hidden %after-header
 For example, the following declaration order in the grammar file reflects the
 order in which Bison will output these code blocks.  However, you are free to
 declare these code blocks in your grammar file in whatever order is most
 convenient for you:
 
 @smallexample
-%before-header @{
address@hidden
   /* Bison treats this block like a pre-prologue block: it inserts it
    * into the code file before the contents of the header file.  It
    * does *not* insert it into the header file.  This is a good place
    * to put #include's that you want at the top of your code file.  A
    * common example is `#include "system.h"'.  */
address@hidden
-%start-header @{
address@hidden
+%header @{
   /* Bison inserts this block into both the header file and the code
    * file.  In both files, the point of insertion is before any
    * Bison-generated token, semantic type, location type, and class
@@ -8358,18 +8335,20 @@
    * dependencies, for example.  */
 @}
 %union @{
-  /* Unlike the traditional Yacc prologue blocks, the output order
-   * for the %*-header blocks is not affected by their declaration
-   * position relative to any %union in the grammar file.  */
+  /* Like for the traditional Yacc prologue blocks, the output order
+   * for the %header blocks is affected by their declaration position
+   * relative to any %union in the grammar file.  Putting prologue
+   * or %header blocks in the grammar section (after %%) has the
+   * same effect.  */
 @}
-%end-header @{
+%header @{
   /* Bison inserts this block into both the header file and the code
    * file.  In both files, the point of insertion is after the
    * Bison-generated definitions.  This is a good place to declare or
    * define public functions or data structures that depend on the
    * Bison-generated definitions.  */
 @}
-%after-header @{
address@hidden
   /* Bison treats this block like a post-prologue block: it inserts
    * it into the code file after the contents of the header file.  It
    * does *not* insert it into the header file.  This is a good place
Index: src/parse-gram.y
===================================================================
RCS file: /sources/bison/bison/src/parse-gram.y,v
retrieving revision 1.92
diff -u -r1.92 parse-gram.y
--- src/parse-gram.y    15 Sep 2006 16:34:48 -0000      1.92
+++ src/parse-gram.y    3 Oct 2006 09:20:03 -0000
@@ -68,6 +68,7 @@
 static symbol *current_lhs;
 static location current_lhs_location;
 static int current_prec = 0;
+static bool post_blocks = false;
 
 #define YYTYPE_INT16 int_fast16_t
 #define YYTYPE_INT8 int_fast8_t
@@ -133,18 +134,16 @@
 `----------------------*/
 
 %token
-  PERCENT_AFTER_HEADER    "%after-header"
-  PERCENT_BEFORE_HEADER   "%before-header"
   PERCENT_DEBUG           "%debug"
   PERCENT_DEFAULT_PREC    "%default-prec"
   PERCENT_DEFINE          "%define"
   PERCENT_DEFINES         "%defines"
-  PERCENT_END_HEADER      "%end-header"
   PERCENT_ERROR_VERBOSE   "%error-verbose"
   PERCENT_EXPECT          "%expect"
   PERCENT_EXPECT_RR      "%expect-rr"
   PERCENT_FILE_PREFIX     "%file-prefix"
   PERCENT_GLR_PARSER      "%glr-parser"
+  PERCENT_HEADER          "%header"
   PERCENT_INITIAL_ACTION  "%initial-action"
   PERCENT_LEX_PARAM       "%lex-param"
   PERCENT_LOCATIONS       "%locations"
@@ -160,7 +159,6 @@
   PERCENT_REQUIRE        "%require"
   PERCENT_SKELETON        "%skeleton"
   PERCENT_START           "%start"
-  PERCENT_START_HEADER    "%start-header"
   PERCENT_TOKEN_TABLE     "%token-table"
   PERCENT_VERBOSE         "%verbose"
   PERCENT_YACC            "%yacc"
@@ -206,7 +204,7 @@
 %%
 
 input:
-  prologue_declarations "%%" grammar epilogue.opt
+  prologue_declarations "%%" { post_blocks = true; } grammar epilogue.opt
 ;
 
 
@@ -221,13 +219,10 @@
 
 prologue_declaration:
   grammar_declaration
-| "%{...%}"     { prologue_augment (translate_code ($1, @1), @1, union_seen); }
-| "%after-header" braceless        { prologue_augment ($2, @2, true); }
-| "%before-header" braceless       { prologue_augment ($2, @2, false); }
+| code_declaration
 | "%debug"                         { debug_flag = true; }
 | "%define" STRING content.opt     { muscle_insert ($2, $3); }
 | "%defines"                       { defines_flag = true; }
-| "%end-header" braceless          { muscle_code_grow ("end_header", $2, @2); }
 | "%error-verbose"                 { error_verbose = true; }
 | "%expect" INT                    { expected_sr_conflicts = $2; }
 | "%expect-rr" INT                { expected_rr_conflicts = $2; }
@@ -252,13 +247,18 @@
 | "%push-parser"                { push_parser = true; }
 | "%require" STRING             { version_check (&@2, $2); }
 | "%skeleton" STRING            { skeleton = $2; }
-| "%start-header" braceless     { muscle_code_grow ("start_header", $2, @2); }
 | "%token-table"                { token_table_flag = true; }
 | "%verbose"                    { report_flag = report_states; }
 | "%yacc"                       { yacc_flag = true; }
 | /*FIXME: Err?  What is this horror doing here? */ ";"
 ;
 
+code_declaration:
+  "%{...%}"                   { prologue_augment (translate_code ($1, @1), @1,
+                                                 post_blocks); }
+| "%header" braceless         { header_augment ($2, @2, post_blocks); }
+;
+
 grammar_declaration:
   precedence_declaration
 | symbol_declaration
@@ -325,6 +325,7 @@
       body++;
 
       union_seen = true;
+      post_blocks = true;
       muscle_code_grow ("stype", body, @3);
     }
 ;
@@ -451,6 +452,7 @@
    body of the grammar.  */
 rules_or_grammar_declaration:
   rules
+| code_declaration
 | grammar_declaration ";"
 | error ";"
     {
Index: src/reader.c
===================================================================
RCS file: /sources/bison/bison/src/reader.c,v
retrieving revision 1.272
diff -u -r1.272 reader.c
--- src/reader.c        15 Sep 2006 16:34:48 -0000      1.272
+++ src/reader.c        3 Oct 2006 09:20:03 -0000
@@ -90,6 +90,19 @@
   obstack_sgrow (oout, prologue);
 }
 
+/*---------------------------------------------------------------------.
+| There are two parts of the header: one before the first %union and.  |
+|  one after.  Augment the one specified by POST.                      |
+`---------------------------------------------------------------------*/
+
+void
+header_augment (const char *header, location loc, bool post)
+{
+  char *muscle = !post ? "start_header" : "end_header";
+  muscle_code_grow (muscle, header, loc);
+}
+
+
 
 
 /*------------------------------------------------------------------------.
Index: src/reader.h
===================================================================
RCS file: /sources/bison/bison/src/reader.h,v
retrieving revision 1.55
diff -u -r1.55 reader.h
--- src/reader.h        9 Jul 2006 20:36:33 -0000       1.55
+++ src/reader.h        3 Oct 2006 09:20:03 -0000
@@ -45,6 +45,7 @@
 /* From reader.c. */
 void grammar_start_symbol_set (symbol *sym, location loc);
 void prologue_augment (const char *prologue, location loc, bool post);
+void header_augment (const char *prologue, location loc, bool post);
 void grammar_current_rule_begin (symbol *lhs, location loc);
 void grammar_current_rule_end (location loc);
 void grammar_midrule_action (void);
Index: src/scan-gram.l
===================================================================
RCS file: /sources/bison/bison/src/scan-gram.l,v
retrieving revision 1.103
diff -u -r1.103 scan-gram.l
--- src/scan-gram.l     15 Sep 2006 15:56:26 -0000      1.103
+++ src/scan-gram.l     3 Oct 2006 09:20:04 -0000
@@ -156,8 +156,6 @@
   `----------------------------*/
 <INITIAL>
 {
-  "%after-header"                   return PERCENT_AFTER_HEADER;
-  "%before-header"                  return PERCENT_BEFORE_HEADER;
   "%binary"                        return PERCENT_NONASSOC;
   "%debug"                         return PERCENT_DEBUG;
   "%default"[-_]"prec"             return PERCENT_DEFAULT_PREC;
@@ -165,7 +163,6 @@
   "%defines"                       return PERCENT_DEFINES;
   "%destructor"                            return PERCENT_DESTRUCTOR;
   "%dprec"                         return PERCENT_DPREC;
-  "%end-header"                     return PERCENT_END_HEADER;
   "%error"[-_]"verbose"            return PERCENT_ERROR_VERBOSE;
   "%expect"                        return PERCENT_EXPECT;
   "%expect"[-_]"rr"                return PERCENT_EXPECT_RR;
@@ -173,6 +170,7 @@
   "%fixed"[-_]"output"[-_]"files"   return PERCENT_YACC;
   "%initial-action"                return PERCENT_INITIAL_ACTION;
   "%glr-parser"                    return PERCENT_GLR_PARSER;
+  "%header"                         return PERCENT_HEADER;
   "%left"                          return PERCENT_LEFT;
   "%lex-param"                     return PERCENT_LEX_PARAM;
   "%locations"                     return PERCENT_LOCATIONS;
@@ -193,7 +191,6 @@
   "%right"                         return PERCENT_RIGHT;
   "%skeleton"                      return PERCENT_SKELETON;
   "%start"                         return PERCENT_START;
-  "%start-header"                   return PERCENT_START_HEADER;
   "%symbol-default"                 return PERCENT_SYMBOL_DEFAULT;
   "%term"                          return PERCENT_TOKEN;
   "%token"                         return PERCENT_TOKEN;
Index: tests/actions.at
===================================================================
RCS file: /sources/bison/bison/tests/actions.at,v
retrieving revision 1.69
diff -u -r1.69 actions.at
--- tests/actions.at    13 Sep 2006 19:39:14 -0000      1.69
+++ tests/actions.at    3 Oct 2006 09:20:05 -0000
@@ -172,7 +172,7 @@
 # helping macros.  So don't put any directly in the Bison file.
 AT_BISON_OPTION_PUSHDEFS([$5])
 AT_DATA_GRAMMAR([[input.y]],
-[[%start-header {
+[[%header {
 #include <stdio.h>
 #include <stdlib.h>
 #include <assert.h>
@@ -190,13 +190,17 @@
   int ival;
 }])
 AT_LALR1_CC_IF([%define "global_tokens_and_yystype"])
-m4_ifval([$6], [[%end-header {]], [[%after-header {]])
+
+%token END 0
+%%
+
+m4_ifval([$6], [[%header {]], [[%{]])
 AT_LALR1_CC_IF([typedef yy::location YYLTYPE;])
 [static int yylex (]AT_LEX_FORMALS[);
 ]AT_LALR1_CC_IF([], [static void yyerror (const char *msg);])
-[}
+m4_ifval([$6], [[}]], [[%}]])[
 
-]m4_ifval([$6], [%type <ival> '(' 'x' 'y' ')' ';' thing line input END])[
+]m4_ifval([$6], [%type <ival> '(' 'x' 'y' ')' ';' thing line input END;])[
 
 /* FIXME: This %printer isn't actually tested.  */
 %printer
@@ -204,34 +208,32 @@
     ]AT_LALR1_CC_IF([debug_stream () << $$;],
                     [fprintf (yyoutput, "%d", $$)])[;
   }
-  input line thing 'x' 'y'
+  input line thing 'x' 'y';
 
 %destructor
   { printf ("Freeing nterm input (address@hidden)\n", $$, RANGE (@$)); }
-  input
+  input;
 
 %destructor
   { printf ("Freeing nterm line (address@hidden)\n", $$, RANGE (@$)); }
-  line
+  line;
 
 %destructor
   { printf ("Freeing nterm thing (address@hidden)\n", $$, RANGE (@$)); }
-  thing
+  thing;
 
 %destructor
   { printf ("Freeing token 'x' (address@hidden)\n", $$, RANGE (@$)); }
-  'x'
+  'x';
 
 %destructor
   { printf ("Freeing token 'y' (address@hidden)\n", $$, RANGE (@$)); }
-  'y'
+  'y';
 
-%token END 0
 %destructor
   { printf ("Freeing token END (address@hidden)\n", $$, RANGE (@$)); }
-  END
+  END;
 
-%%
 /*
    This grammar is made to exercise error recovery.
    "Lines" starting with `(' support error recovery, with
Index: tests/regression.at
===================================================================
RCS file: /sources/bison/bison/tests/regression.at,v
retrieving revision 1.108
diff -u -r1.108 regression.at
--- tests/regression.at 13 Sep 2006 17:41:19 -0000      1.108
+++ tests/regression.at 3 Oct 2006 09:20:07 -0000
@@ -398,7 +398,6 @@
 input.y:6.1: invalid character: `%'
 input.y:6.2: invalid character: `-'
 input.y:7.1-8.0: missing `%}' at end of file
-input.y:7.1-8.0: syntax error, unexpected %{...%}
 ]])
 
 AT_CLEANUP