Re: [lmi] Re: Parsing, and parsimony

[Greg Chicares]

On 2005-9-24 13:33 UTC, Vadim Zeitlin wrote:
> On Tue, 06 Sep 2005 14:15:30 +0000 Greg Chicares <address@hidden> wrote:
> 
> GC> On 2005-9-6 13:47 UTC, Greg Chicares wrote:
> GC> > 
> GC> >    using boost::spirit as a parser (so we'd have RPN) and boost::phoenix
> GC> >    instead of boost::lambda (because the same author wrote spirit and
> GC> >    phoenix).
> GC> 
> GC> In 'input_sequence.?pp' we have a hand-coded recursive-descent parser.
> GC> It's a lot like Stroustrup's example in TC++PL3, which in turn is a lot
> GC> like the example in the Red Dragon Book. It's 1526 lines of code to
> GC> maintain, or 3222 if you count '*seq*.?pp'.
> ...
> GC> If it's a good idea to use spirit here,
> 
>  I've read the code now and, globally, I think it is. There are a few
> caveats though:
> 
> 1. I've never used spirit before so I could miss some unforeseen problems

Fair enough.

> 2. I'm not sure about the error detection, i.e. whether it's possible to
>    give as detailed error messages as needed in this framework, but AFAICS
>    it should be

Flexibility of error messages is a strength of hand-written parsers.
I guess the only way to tell what we can do with spirit is to try it.
I feel it's worth the trial, even if we ultimately find difficulties
with error messages that cause us to revert to the existing parser.

> 3. Spirit syntax is similar to BNF but not quite, e.g. let me rewrite your
>    some productions of your grammar in it to show it (there could be errors
>    in what I write but it should be sufficiently close to the real code to
>    illustrate what I mean):
> 
> GC>   // GRAMMAR interval-begin: one of [ (
> 
>       interval_begin = ch_p('[') | ch_p('(');
> 
> GC>   // GRAMMAR duration-scalar: @ integer
> 
>       duration_scalar = '@' >> int_p;
> 
> GC>   // GRAMMAR sequence: span ; span
> 
>       sequence = span >> *(';' >> span);
> 
> 
>    So if you want to really use BNF, you'd better switch to another parser
>    (yacc, antlr, whatever...) and not Spirit.

It's not important what metalanguage we write the grammar in: spirit's is
as good for lmi as BNF.

>  OTOH:
> 
> 1. Spirit does look cute
> 2. It has excellent documentation, it's the best doc for any boost library
>    I've seen so far
> 3. Integration with C++ code is very, of course, so it's much more elegant
>    than yacc
> 4. This grammar could be implemented trivially in Spirit AFAICS

This is persuasive. An elegant, well-documented library makes implementation
pretty trivial. And a trivial implementation is easier to keep free of
defects, and probably easier to extend. I'm sure we should try it.

>  To conclude, I think that if there are good unit tests for this code (I
> did see input_seq_test.cpp but it has a few TODOs so I don't know how
> complete is it),

I think it's a solid unit test. Most of those comments record open design
questions. Looking back at those questions, I feel that I'd need to regain
a deep understanding of *many* lines of code before I could address them.
That's probably another argument in favor of using spirit.

> then it should be possible to complete the transition to
> Spirit quite easily and quickly. I.e. writing the code with Spirit should
> be very quick, it's just testing an ensuring that it works the same as the
> old one which would take time and here the existing tests are crucial. But
> if the tests can be trusted, this shouldn't take more than a couple of
> days.

I believe the tests are quite comprehensive.

> GC> then it may be a good idea to use spirit and phoenix elsewhere, as
> GC> described above: we'd depend on fewer libraries.
> 
>  However I don't agree at all with this. AFAIK boost::lambda is much more
> popular than phoenix. And I don't see any important (for us) advantages of
> the latter compared to the former. And as you always have both libraries
> anyhow (both libraries are header only, there is nothing to compile; and
> there is no separate boost::lambda distribution, you always have to install
> the entire boost) I don't see why shouldn't we use the best tool for the
> job. IMHO Spirit should be very good for the parsing code but it doesn't
> mean at all that boost::lambda can't be used elsewhere.

Your points are sound. We shouldn't replace what works reliably (lambda)
with something unfamiliar (phoenix) without a good reason. I merely
speculate (casually, without having studied the matter well) that we
might later find that phoenix happens to fill lambda's niche well.
Or we might find the opposite.