Re: PEG Parser

[Michael Lucy]

On Fri, Jan 28, 2011 at 9:07 PM, Noah Lavine <address@hidden> wrote:
> Hello,
>
>> Indeed!  I'm looking forward to having it in Guile.
>
> Great! I don't intend to be pressuring you to merge it, by the way. I
> hope my message didn't come across that way.
>
>> The reason I didn't merge it yet was twofold: (1) the commit logs were
>> not in the standard style, and (2) I wasn't comfortable with the macro
>> binding stuff.  (2) was sufficiently big that I didn't have time to fix
>> it up.  My apologies for stalling on this...  But maybe now is the time
>> to look at it again!
>
> Yes, the binding stuff actually seemed very weird to me too, mostly
> because it's very non-idiomatic. More specifically, the peg module
> does most of its code generation in regular functions rather than
> macros. safe-bind implements hygiene for these code-generating
> functions. It's basically what you'd get with defmacro.
>
> I eventually came to the conclusion, however, that it's really a
> symptom of a larger issue.  Most likely it would be possible to take
> all of the functions that use safe-bind and rewrite them as macros
> themselves, using either syntax-case or syntax-rules. However, I am
> afraid that the result might be incredibly ugly. In order to make the
> code more readable, we use normal functions for code generation and
> provide hygiene through safe-bind.
>
> So the larger issue is that Scheme needs nicer-looking ways to define
> macros. If it had those, we wouldn't be using hacks like safe-bind.
> But we don't, and that's a huge issue to tackle.
>
> Those are my thoughts, but I would be quite curious to know if you (or
> anyone else) think the same way, or think something different is going
> on.

So, that's sort of a complicated story.  The safe-bind thing is a
standard idiom in the Common Lisp world (although it goes by many
names).  The most immediate reason it's there is that I was struggling
a bit with the hygienic macro system and fell back on what I knew,
then ran out of time trying to turn everything into a syntax-case
macro before the end of GSOC (which is why the data gets turned into
syntax at several points near the end of the process).

If we're talking about root causes, I think you're about right, at
least in my case.  It's pretty standard in the CL world to write large
macros by breaking the code out into functions, because it's
considered easier to reason about functions than macros.  I remember
Andy told me that functions can generate syntax, and I tried that, but
I had a remarkably hard time making it actually work correctly.  Part
of the problem was probably that I never really understood how syntax
worked, whereas I understand very well how lists work.

Also, macros are notoriously difficult to debug, especially when
they're generating several hundred lines of code that compiles fine
but mysteriously produces the wrong result after a seemingly trivial
change.  So porting them from list generation to syntax generation
turned out to be harder in practice than I had thought it would be.

>
> Noah
>