Am Fr., 16. Nov. 2018 um 01:01 Uhr schrieb Mark H Weaver <
Hi Marc,
Marc Nieper-Wißkirchen <address@hidden> writes:
> > Let's assume we are writing a macro that reimplements syntax (or some
> > variation thereof) and which has to check whether identifiers are
> > ellipses. For example, the following could be given:
> >
> > (with-ellipsis e
> > (my-syntax a e)
> >
> > Now, this could be a result of a macro expansion and e could carry
> > different marks than with-syntax or my-syntax. This is why I have been
> > thinking that one also needs the lexical context of my-syntax and not
> > only the context of e.
>
> I don't see what problem would be caused by 'e' carrying different marks
> than 'my-syntax'.
>
> As far as I can tell, in the end, the two instances of 'e' above will
> effectively be compared to one another using 'bound-identifier=?'. They
> must have the same name and the same marks to match. The marks on
> 'my-syntax' are irrelevant here.
>
> I have been thinking of the scope in which $sc-ellipsis is bound by
> `with-syntax'.
You've written 'with-syntax' is several places, in both this email and
in your previous email, and I'm guessing that you meant to write
'with-ellipsis' in each of those places. Is that right?
Yes! I shouldn't do things in parallel (writing emails for this list and programming syntax-case transformers, which use with-syntax but not with-ellipsis, at the same time). I hope it didn't cause too much confusion.
> If `my-syntax' is within the scope of `with-ellipsis', the binding of
> $sc-ellipsis introduced by this `with-syntax' will be relevant; if
> `my-syntax' is not in the lexical scope of `with-ellipsis', the
> binding should be irrelevant; thus my thought that we need the lexical
> information of my-syntax as well.
>
> Operationally, when (with-ellipsis e (my-syntax a e)) is expanded, 'e'
> will be added to the macro expansion environment as the innermost
> binding of the ellipsis identifier, and then (my-syntax a e) will be
> expanded within that new expansion environment. That is the expansion
> environment that will be consulted by the 'ellipsis-identifier?'
> predicate to find the current ellipsis identifier, which is compared
> with its argument (after stripping its anti-mark) using
> 'bound-identifier=?'.
>
> Aha, so maybe I have misunderstood the scope of `with-syntax'. Please
> consider the following example:
>
> (define-syntax foo
> (lambda (stx)
> (with-ellipsis e
> (syntax-case stx ()
> ((_ x e) (bar #'(x e)))))))
>
> (eval-when (expand)
> (define (bar x*)
> (syntax-case x* ()
> ((x ...) ---))))
>
> I would have thought that the `...' identifier in `bar' is recognized
> as an ellipsis,
It is.
> but from what you are saying it seems that the binding `with-syntax'
> is dynamic with respect to macro expansion (like syntax
> parameters). Is this really what we want?
I agree that it's not what we want, and if I understand correctly, it's
not what we have in Guile.
In Psyntax, lexical lookups of identifiers are done in two steps, using
two different data structures. First, the deferred substitutions in the
wrap are applied to the identifier, which yields a gensym if the
identifier is lexically bound. Next, the gensym is looked up in the
expansion environment 'r' to find the actual binding.
The deferred substitutions are applied to the inner bodies of each core
binding construct. When the macro expander encounters a core binding
construct, a fresh gensym is created for the binding, and that gensym is
effectively substituted for all free occurrences of the identifier
within the inner body. Mostly for efficiency reasons, this substitution
is done lazily, by adding it to the wrap. The expansion environment is
also extended each time the macro expander encounters a core binding
construct.
With this in mind, let's examine your example above more closely. The
ellipsis binding for 'e' is only in the transformer environment when the
'syntax-case' form is expanded. It is _not_ in the transformer
environment when your 'foo' macro is later used.
I agree and I see that my example doesn't demonstrate what it should have demonstrated because `bar' is not executed before `foo' is used as a macro. The example should have been more like the following:
(define-syntax foo
(lambda (stx)
(with-ellipsis e
(syntax-case (third-party-macro-transformer-helper-macro stx) ()
---))))
Here, the helper macro may expand into another instance of syntax-case. That instance should not recognize `e' as the ellipsis but whatever the ellipsis was where the helper macro was defined.
But let's go one step further. Let's consider what will happen if 'foo'
is used within 'with-ellipsis':
(with-ellipsis --- (foo a b))
When this is expanded, a fresh gensym will be generated, and an ellipsis
binding will be added for that gensym in the expansion environment 'r'.
Also, a substitution from #{ $sc-ellipsis }# to that gensym will be
added to the wrap of (foo a b).
Now let's consider how 'bar' will be affected by this. In the example
you give, where 'bar' uses 'syntax-case', the ellipsis identifier will
be looked up in the transformer environment where 'bar' is *defined*,
not the transformer environment where 'bar' is called.
But let's suppose that we change 'bar' to use 'ellipsis-identifier?' at
run-time, like this:
(define-syntax foo
(lambda (stx)
(with-ellipsis e
(syntax-case stx ()
((_ x e) (bar #'(x e)))))))
(eval-when (expand)
(define (bar x*)
(syntax-case x* ()
((x dots)
(ellipsis-identifier? #'dots)
#'#true)
(_
#'#false))))
We now see this behavior with my draft patch:
(with-ellipsis --- (foo a b)) => #false
(with-ellipsis --- (foo a ...)) => #false
(with-ellipsis --- (foo a ---)) => #true
I think this is what we want, right?
I think it looks correct.
When 'bar' is called, there will be a binding in the transformer
environment 'r' that maps a gensym to an ellipsis binding, which
specifies '---' as the ellipsis identifier. However, that binding will
only apply when testing identifiers that have been wrapped to include a
substitution from #{ $sc-ellipsis }# to the same gensym, so it will only
apply to identifiers that are in body of the same 'with-ellipsis' form.
> Therefore I think, we want `with-ellipsis' to be lexically scoped (in
> the macro transformer code).
Yes, that was certainly my intent.
Let's run the following example:
(eval-when (expand)
(define-syntax bar
(syntax-rules ()
((_ stx)
(syntax-case stx ()
((_ a (... ...))
#'#t)
((_ a b c)
#'#f))))))
(define-syntax foo
(lambda (stx)
(with-ellipsis e (bar stx))))
(display (foo 1 2 3))
(newline)
This one displays `#t' in Guile, which is exactly what we want. I guess the reason is that the macro invocation `(bar stx)' creates a new transformer environment, in which `{# $sc-ellipsis #}' becomes unbound again.
Now, why does the following work (i.e. why does it print `#t')?