Re: push parser

[Bob Rossi]

Hi,

Paul, are you going to apply the patch to push.c? I think it does
pass the testsuite if you 'cp push.c yacc.c'. This would be helpful
to me if you could. Then I could give one final annotation of
yacc and push, before we copy them over.

Thanks,
Bob Rossi

On Mon, Oct 02, 2006 at 10:24:36AM -0400, Bob Rossi wrote:
> On Fri, Sep 29, 2006 at 11:30:51PM -0400, Bob Rossi wrote:
> > On Fri, Sep 29, 2006 at 05:55:08PM -0700, Paul Eggert wrote:
> > > Bob Rossi <address@hidden> writes:
> > > 
> > > > If we force the user to define that function, it add's an unnecessary 
> > > > complication to the push parser. That is, the user is mostly likely 
> > > > going to call yypushparse, and not care about yyparse at all. However,
> > > > yyparse is the function causing the user to define the yypushlex
> > > > function. What do you think I should do to resolve this?
> > > 
> > > Ah, sorry, I didn't understand this issue at all.
> > > 
> > > It sounds to me like the push parser shouldn't define yyparse.  That
> > > way, the user shouldn't have to care about any lexer function.
> > 
> > I agree. Akim, you were the main advocate for adding this functionality. Can
> > you see any other solution to the problem besides remove yyparse from
> > the generated output?
> 
> Akim, when you get around to this, let me know what your thoughts are.
> For now, I've removed yyparse again in push mode. It can be easily added
> back ...
> 
> Paul, I think we are getting closer to a userful interface for the user.
> Below is what it currently looks like.
> 
>   struct yypvars *ctx = yypvarsinit ();
>   int status;
>   YYSTYPE my_lval;
>   YYLTYPE my_lloc;
>   do {
>     status = yypushparse (ctx, yylex (&my_lval, &my_lloc), &my_lval, &my_lloc)
>   } while (status == YYPUSH_MORE);
>   free (ctx);
> 
> The user is forced to pass in NULL to my_lval. When locations are turned
> on the extra parameter will also have to be passed in. Otherwise, not.
> 
> How is this looking? I'm starting to like it much better than what was 
> previously done.
> 
> Thanks,
> Bob Rossi
> 
> 2006-10-02  Bob Rossi  <address@hidden>
> 
>         * data/push.c (yychar_set, yylval_set, yylloc_set): Delete.
>         (yypushparse): Add yynchar, yynlval, yynlloc parameters.
>         (b4_declare_parser_variables): Do not declare yynerrs for push mode.
>         (struct yypvars): Remove b4_declare_parser_variables.
>         (yypvarsinit): Remove init code for removed variables.
>         (global scope): Do not declare b4_declare_parser_variables if
>         push or pure mode.
>         (yypushparse): Add b4_declare_parser_variables.
>         Init new local variables, and remove init code for removed
>         yypvars variables.
>         (yyparse): Delete.
>         * tests/calc.at (_AT_DATA_CALC_Y): Call yypushparse for push mode
>         and yyparse for other modes.
>         * tests/local.at (AT_PUSH_IF): Added.
>         (AT_PURE_IF): Rename to AT_PURE_OR_PUSH_IF and modify accordingly.
>         (AT_PURE_AND_LOC_IF): Rename to AT_PURE_OR_PUSH_AND_LOC_IF and
>         modify accordingly.
>         (AT_YYERROR_ARG_LOC_IF): Use AT_PURE_OR_PUSH_AND_LOC_IF.
>         (AT_YYERROR_SEES_LOC_IF): Use AT_PURE_OR_PUSH_IF.
>         (AT_PURE_LEX_IF): Use AT_PURE_OR_PUSH_IF.
> 
> 

> Index: data/push.c
> ===================================================================
> RCS file: /sources/bison/bison/data/push.c,v
> retrieving revision 1.3
> diff -u -r1.3 push.c
> --- data/push.c       21 Sep 2006 17:45:21 -0000      1.3
> +++ data/push.c       2 Oct 2006 14:12:27 -0000
> @@ -161,11 +161,7 @@
>  #define yychar  b4_prefix[]char
>  #define yydebug b4_prefix[]debug
>  #define yynerrs b4_prefix[]nerrs
> -b4_locations_if([#define yylloc b4_prefix[]lloc])
> -b4_push_if([
> -#define yychar_set b4_prefix[]char_set
> -#define yylval_set b4_prefix[]lval_set
> -#define yylloc_set b4_prefix[]lloc_set])])[
> +b4_locations_if([#define yylloc b4_prefix[]lloc])])[
>  
>  /* Copy the first part of user declarations.  */
>  ]b4_pre_prologue[
> @@ -974,14 +970,12 @@
>  ]b4_push_if([
>  struct yypvars;
>  enum { YYPUSH_MORE = 4 };
> -]b4_c_function_decl([yychar_set], [void], [[struct yypvars *YYPVARS], 
> [YYPVARS]], [[int yychar], [yychar]])[
> -]b4_c_function_decl([yylval_set], [void], [[struct yypvars *YYPVARS], 
> [YYPVARS]], [[YYSTYPE yylval], [yylval]])[
> -#ifdef YYLTYPE_IS_TRIVIAL
> -]b4_c_function_decl([yylloc_set], [void], [[struct yypvars *YYPVARS], 
> [YYPVARS]], [[YYLTYPE yylloc], [yylloc]])[
> -#endif
>  ]b4_c_function_decl([yypvarsinit], [void *], [[void], []])[
>  ]b4_c_function_decl([yypushparse], [int],
> -   [[struct yypvars *YYPVARS], [YYPVARS]])[
> +   [[struct yypvars *yypvars], [yypvars]],
> +   [[int yynchar], [yynchar]],
> +   [[YYSTYPE *yynlval], [yynlval]]
> +   b4_locations_if([,[[YYLTYPE *yynlloc], [yynlloc]]]))[
>  ])[
>  
>  ]m4_divert_push([KILL])# ======================== M4 code.
> @@ -997,7 +991,8 @@
>  YYSTYPE yylval;
>  
>  /* Number of syntax errors so far.  */
> -int yynerrs;b4_locations_if([
> +]b4_push_if([],[
> +int yynerrs;])[b4_locations_if([
>  /* Location data for the lookahead symbol.  */
>  YYLTYPE yylloc;])
>  ])
> @@ -1008,7 +1003,6 @@
>  m4_define([b4_declare_yyparse_variables],
>  [[struct yypvars
>    {
> -]]b4_declare_parser_variables[[
>      int yystate;
>      int yyn;
>      int yyresult;
> @@ -1087,45 +1081,16 @@
>    pv->yyssp = pv->yyss;
>    pv->yyvsp = pv->yyvs;
>  
> -#if YYLTYPE_IS_TRIVIAL
> -  /* Initialize the default location before parsing starts.  */
> -  pv->yylloc.first_line   = pv->yylloc.last_line   = 
> ]b4_location_initial_line[;
> -  pv->yylloc.first_column = pv->yylloc.last_column = 
> ]b4_location_initial_column[;
> -#endif
> -
>    pv->yynew = 1;
>  
>  ]b4_locations_if([  pv->yylsp = pv->yyls;])[
>  
>    return (void *) pv;
> -}
> -
> -void
> -yychar_set (struct yypvars *YYPVARS, int yychar)
> -{
> -  if (YYPVARS)
> -    YYPVARS->yychar = yychar;
> -}
> -
> -void
> -yylval_set (struct yypvars *YYPVARS, YYSTYPE yylval)
> -{
> -  if (YYPVARS)
> -    YYPVARS->yylval = yylval;
> -}
> -
> -#ifdef YYLTYPE_IS_TRIVIAL
> -void
> -yylloc_set (struct yypvars *YYPVARS, YYLTYPE yylloc)
> -{
> -  if (YYPVARS)
> -    YYPVARS->yylloc = yylloc;
> -}
> -#endif])
> +}])
>  m4_divert_pop([KILL])dnl# ====================== End of M4 code.
>  
> -b4_pure_if([],
> -        [b4_declare_parser_variables])
> +b4_push_if([],[b4_pure_if([],
> +        [b4_declare_parser_variables])])
>  
>  b4_push_if([b4_declare_yyparse_variables])
>  
> @@ -1134,7 +1099,9 @@
>  `-------------------------*/
>  
>  b4_push_if([
> -b4_c_function_def([yypushparse], [int], [[struct yypvars *YYPVARS], 
> [YYPVARS]])],[
> +b4_c_function_def([yypushparse], [int], [[struct yypvars *yypvars], 
> [yypvars]], 
> +                  [[int yynchar], [yynchar]], [[YYSTYPE *yynlval], [yynlval]]
> +               b4_locations_if([,[[YYLTYPE *yynlloc], [yynlloc]]]))],[
>  #ifdef YYPARSE_PARAM
>  b4_c_function_def([yyparse], [int], [[void *YYPARSE_PARAM], [YYPARSE_PARAM]])
>  #else /* ! YYPARSE_PARAM */
> @@ -1142,6 +1109,7 @@
>  #endif])
>  {[
>    ]b4_pure_if([b4_declare_parser_variables])[
> +  ]b4_push_if([b4_declare_parser_variables])[
>    ]b4_push_if([struct yypvars *pv;])[
>    int yystate;
>    int yyn;
> @@ -1202,12 +1170,15 @@
>  
>    YYDPRINTF ((stderr, "Starting parse\n"));
>  
> -  ]b4_push_if([pv = YYPVARS;])[
> -
>    yystate = 0;
>    yyerrstatus = 0;
> -  yynerrs = 0;
> -  yychar = YYEMPTY;          /* Cause a token to be read.  */
> +]b4_push_if([  yychar = yynchar;
> +  pv = yypvars;
> +  if (yynlval)
> +    yylval = *yynlval;
> +]b4_locations_if([  if (yynlloc)
> +     yylloc = *yynlloc;])[],[yynerrs = 0;
> +  yychar = YYEMPTY; /* Cause a token to be read.  */])[
>  
>    /* Initialize stack pointers.
>       Waste one element of value and location stack
> @@ -1237,12 +1208,6 @@
>  ]])dnl
>  [  ]b4_push_if([
>       /* Initialize the locals to the current context. */
> -     yychar = pv->yychar;
> -     yylval = pv->yylval;
> -     yynerrs = pv->yynerrs;
> -     ]b4_locations_if([
> -     yylloc = pv->yylloc;])[
> -
>       yystate = pv->yystate;
>       yyn = pv->yyn;
>       yyresult = pv->yyresult;
> @@ -1382,12 +1347,6 @@
>       YYDPRINTF ((stderr, "Return for a new token:\n"));
>       yyresult = YYPUSH_MORE;
>       /* Initialize the locals to the current context. */
> -     pv->yychar = yychar;
> -     pv->yylval = yylval;
> -     pv->yynerrs = yynerrs;
> -     ]b4_locations_if([
> -     pv->yylloc = yylloc;])[
> -
>       pv->yystate = yystate;
>       pv->yyn = yyn;
>       pv->yyresult = yyresult;
> @@ -1530,7 +1489,7 @@
>    /* If not already recovering from an error, report this error.  */
>    if (!yyerrstatus)
>      {
> -      ++yynerrs;
> +]b4_push_if([],[ ++yynerrs;])[
>  #if ! YYERROR_VERBOSE
>        yyerror (]b4_yyerror_args[YY_("syntax error"));
>  #else
> @@ -1713,28 +1672,6 @@
>      return YYID (yyresult);
>  ]}
>  
> -b4_push_if([
> -#ifdef YYPARSE_PARAM
> -b4_c_function_def([yyparse], [int], [[void *YYPARSE_PARAM], [YYPARSE_PARAM]])
> -#else /* ! YYPARSE_PARAM */
> -b4_c_function_def([yyparse], [int], b4_parse_param)
> -#endif
> -{[
> -  struct yypvars *ctx = yypvarsinit ();
> -  int status;
> -  do {
> -    yychar_set (ctx, yylex ());
> -    yylval_set (ctx, yylval);
> -#ifdef YYLTYPE_IS_TRIVIAL
> -    yylloc_set (ctx, yylloc);
> -#endif
> -    status = yypushparse (ctx);
> -  } while (status == YYPUSH_MORE);
> -  free (ctx);
> -  return status;
> -]}])
> -
> -
>  b4_epilogue
>  b4_defines_if(
>  address@hidden @output_header_name@
> Index: tests/calc.at
> ===================================================================
> RCS file: /sources/bison/bison/tests/calc.at,v
> retrieving revision 1.92
> diff -u -r1.92 calc.at
> --- tests/calc.at     15 Sep 2006 15:56:26 -0000      1.92
> +++ tests/calc.at     2 Oct 2006 14:12:27 -0000
> @@ -336,7 +336,17 @@
>  
>  ]AT_SKEL_CC_IF([], [m4_bmatch([$4], [%debug],
>  [  yydebug = 1;])])[
> -  status = yyparse (]AT_PARAM_IF([&result, &count])[);
> +]AT_PUSH_IF([
> +  {
> +    struct yypvars *ctx = yypvarsinit ();
> +    YYSTYPE my_lval;
> +    ]AT_LOCATION_IF([YYLTYPE my_lloc;])[
> +    do {
> +      status = yypushparse (ctx, yylex (&my_lval]AT_LOCATION_IF([[, 
> &my_lloc]])[), &my_lval]AT_LOCATION_IF([[, &my_lloc]])[);
> +    } while (status == YYPUSH_MORE);
> +    free (ctx);
> +  }],[
> +  status = yyparse (]AT_PARAM_IF([[&result, &count]])[);])[
>    if (global_result != result)
>      abort ();
>    if (global_count != count)
> Index: tests/local.at
> ===================================================================
> RCS file: /sources/bison/bison/tests/local.at,v
> retrieving revision 1.16
> diff -u -r1.16 local.at
> --- tests/local.at    12 Mar 2006 15:26:05 -0000      1.16
> +++ tests/local.at    2 Oct 2006 14:12:28 -0000
> @@ -56,10 +56,12 @@
>  [m4_bmatch([$3], [%parse-param], [$1], [$2])])
>  m4_pushdef([AT_LOCATION_IF],
>  [m4_bmatch([$3], [%locations], [$1], [$2])])
> -m4_pushdef([AT_PURE_IF],
> -[m4_bmatch([$3], [%pure-parser], [$1], [$2])])
> -m4_pushdef([AT_PURE_AND_LOC_IF],
> -[m4_bmatch([$3], [%locations.*%pure-parser\|%pure-parser.*%locations],
> +m4_pushdef([AT_PUSH_IF],
> +[m4_bmatch([$3], [%push-parser], [$1], [$2])])
> +m4_pushdef([AT_PURE_OR_PUSH_IF],
> +[m4_bmatch([$3], [%pure-parser\|%push-parser], [$1], [$2])])
> +m4_pushdef([AT_PURE_OR_PUSH_AND_LOC_IF],
> +[m4_bmatch([$3], 
> [%locations.*%pure-parser\|%pure-parser.*%locations\|%locations.*%push-parser\|%push-parser.*%locations],
>          [$1], [$2])])
>  m4_pushdef([AT_GLR_OR_PARAM_IF],
>  [m4_bmatch([$3], [%glr-parser\|%parse-param], [$1], [$2])])
> @@ -69,12 +71,12 @@
>             [yy])])
>  # yyerror receives the location if %location & %pure & (%glr or 
> %parse-param).
>  m4_pushdef([AT_YYERROR_ARG_LOC_IF],
> -[AT_GLR_OR_PARAM_IF([AT_PURE_AND_LOC_IF([$1], [$2])],
> +[AT_GLR_OR_PARAM_IF([AT_PURE_OR_PUSH_AND_LOC_IF([$1], [$2])],
>                   [$2])])
>  # yyerror always sees the locations (when activated), except if
>  # yacc & pure & !param.
>  m4_pushdef([AT_YYERROR_SEES_LOC_IF],
> -[AT_LOCATION_IF([AT_YACC_IF([AT_PURE_IF([AT_PARAM_IF([$1], [$2])],
> +[AT_LOCATION_IF([AT_YACC_IF([AT_PURE_OR_PUSH_IF([AT_PARAM_IF([$1], [$2])],
>                                       [$1])],
>                           [$1])],
>               [$2])])
> @@ -82,7 +84,7 @@
>  # The interface is pure: either because %pure-parser, or because we
>  # are using the C++ parsers.
>  m4_pushdef([AT_PURE_LEX_IF],
> -[AT_PURE_IF([$1],
> +[AT_PURE_OR_PUSH_IF([$1],
>           [AT_SKEL_CC_IF([$1], [$2])])])
>  
>  AT_PURE_LEX_IF(
> @@ -125,7 +127,7 @@
>  m4_popdef([AT_YYERROR_ARG_LOC_IF])
>  m4_popdef([AT_NAME_PREFIX])
>  m4_popdef([AT_GLR_OR_PARAM_IF])
> -m4_popdef([AT_PURE_AND_LOC_IF])
> +m4_popdef([AT_PURE_OR_PUSH_AND_LOC_IF])
>  m4_popdef([AT_LOCATION_IF])
>  m4_popdef([AT_PARAM_IF])
>  m4_popdef([AT_YACC_IF])