Re: trying out the c++ parser skeleton / b4_post_prologue

[Akim Demaille]

>>> "Anthony" == Anthony Heading <address@hidden> writes:

hi Anthony,

 > Akim Demaille wrote:
 >> Could you please detail why you'd like to do that?  For several
 >> reasons, including recursion support, I think the proper way to extend
 >> the parser is to couple it with a parsing driver.

 > Yes, I did read the example in the docs, and thought it an odd design.
 > Perhaps you also could expand on those reasons

The main reason is to provide a reentrant parser, and that requires a
lot of interaction with the rest of the environment.  It proved to be
much simpler that capture the environment within the so-called driver,
which is passed to the parser.

Another issue was how to pass additional members to the parser, so
that they could be used in the user actions.  Trying to extend the
parser class directly is difficult and does not fit nicely with the
current interface (in particular because one has to extend the
constructor of the parser with constructors to the new members).
Obviously you can't add these new members in a subclass, since the
user actions are run in the superclass.  There was a time when we
provided the user with a means to define a *superclass* of the parser
class, letting him initialize his members this way.  But it was also
very inconvenient.

In the end, after having tried several approaches, the driver pattern
proved to be not only simpler, but also much more flexible and more
powerful.  Until very recently, there were still scares from this time
when, trying to pay for the indirection, we planned on using a more
STL-like interface: various traits.  These guys were a problem for
someone willing to subclass.  But they are no longer there, si it
*might* be feasible to subclass.  But why in the very files?  What's
wrong with subclassing in subparser.{hh,cc}?

 > - here are my thoughts as an end-user.

 > First, it is very natural in C++ for a programmer to link in some
 > types of application-specific code by providing an implementation of
 > an abstract interface. See e.g. Stroustrup C++PL ch. 24, MFC, Qt, etc.
 > I was surprised to find, for example, that in this bison-c++, the
 > interface to the lexer is not created as an abstract virtual function
 > for the end-programmer to implement, but instead remains a C macro.

Right.  Valid or not, the point was simply to avoid an indirection.
Generated parsers are expected to run fast, and it seemed like a waste
to subclass for yylex alone.  By I nevertheless agree that this is not
nice.


 > Obviously this isn't wrong per se, but there has been a decade of
 > aggregate common thinking (agree with it or not) about how ideally
 > C++ code should differ from C, and the bison-c++ design differs
 > enough in many areas from this convergence of practice that I found
 > it unintuitive and hard to use. Therefore I chose to ignore the
 > documentation example, and instead read the skeleton source code.

OK.

 > Your parsing-driver suggestion leads the semantic action code to be
 > filled with references to a driver handle.       | TOKEN_IDENTIFIER  {
 > $$ = driver.variables[*$1]; }

 > This is the main reason for my attempt to subclass.

I explained above why subclassing does not work.

 > But in C++, there is the huge benefit that this scope can
 > transparently be an _instance_ of a parser, which surely is a great
 > tool to solve many of the recursion problems.

It is not.  As we will move the implement of the push-parser scheme in
Bison, you'll see that the parser object maintains state.  For each
parse, instantiate a new object.

 > If I were starting again, I would flip your parser/parsing-driver
 > relationship, and have the semantic action code generated within
 > the scope of the parsing driver, i.e. within the scope of a
 > user-extensible class which maintains the semantic state.

So what exactly are you proposing?  That we generate code for a
subclass that the user is expected to write?  How is that simpler than
the current scheme?

Also, in case of recursive parses, while there are several parsers,
there is a unique driver.

 > I would certainly make bison not the user do the boilerplate work:
 > e.g. "$1" conceptually can be automatically expanded to
 > "yy_parser_instance_->yysemantic_stack_[1]" so that bison reaches
 > out to find the parser class, the user isn't required to reach to
 > find his driver via e.g "driver.variables".  But then I'd go
 > further to suggest that the parser-driver should inherit from the
 > parser rather than being in a strange peer relationship, because
 > there's enormous support within the C++ language for that type of
 > design.

A parser driver is not a parser.  The relationship here is not
subclassing.  But you might be referring to some new kind of
relationship that would need more description.

 > But probably we're not starting again, so I'm not sure what to
 > suggest.  I've spent another couple of hours this morning studying
 > the code while composing this response, and I begin to understand
 > the frustration expressed by e.g. Hans.  So in my now somewhat-
 > considered view, the current C++ skeleton is a poor design, and
 > I'm inclined not to use it.

OK.  If you have concrete suggestions on how to make it more useful
without making it useless to some of its users, we'll buy it.

 > That's a bit troublesome though, because a non-POD C++ YYSTYPE
 > is a very desirable thing, and I don't believe that works fully
 > with the C parser. 

I totally agree on this point.  This is real pain in the neck, but
currently there is no nice answer to this.  Hans makes a proposal
which is simply unacceptable in many situations (not everyone wants to
have all there semantics values derive from a single base class).
Discussion is underway to provide the user with a means to define
yystype on his own, and that will implement Hans' proposal.  But
someday if one has enough time to implement this, what is truly needed
to accommodate C++ requirements and its users expectations in this
respect, is a stripped down version of variants, allowing not only
PODs, but also non-PODs.

 > And as Hans mentions, bison isn't static enough for a user to
 > maintain their own skeleton.  Hmm...

I don't know what static means here.



There is one feature that Hans (and I, for one) wants for quite a
while now, and that was technically difficult until very recently:
%define for code.  It is now at hand, and it turns out I have a patch
almost ready for it (I have a s/r conflict to kill first).  That
should provide knowledgeable users with more freedom.




Much work needs to be done on the C++ skeletons, and we could use
people willing to contribute *code*:

- use <vector> instead of <queue> (at the user's choice)

- merge the loc and val stacks, and possibly the state stack too.
  In fact I think all skeletons should consider this move

- propagate Bob Rossi's changes for push parsers

- push the contexts to the mergers too