Hi,
Post by Mark H WeaverPost by Catonanowhat's the problem with macroexpand-1 and syntax-case ?
In Guile 1.x, 'macroexpand-1' performed a single macro expansion step at
the top-level form of an expression, using its old non-hygienic macro
expander. There are several problems with trying to provide such an
interface in a Hygienic macro expander, and especially in the
'syntax-case' expander with its support for 'datum->syntax'. For one
thing, our modern macro expander doesn't even work with the plain
S-expressions which 'macroexpand-1' accepted and produced. It works
with "syntax objects", which effectively annotate every identifier with
extra information needed to determine which binding it references, and
also extra information needed to implement 'datum->syntax'. This in
turn requires detailed knowledge of the lexical environment in which
expansion is taking place, whereas 'macroexpand-1' provides no way for
the user to provide this information.
Mark
I have been reading this document about the scheme higienic macros
https://www.cs.indiana.edu/~dyb/pubs/bc-syntax-case.pdf
I stopped reading it when I read that the implementation relies on a
previously bootstrapped version of another macro expansion
implementation.
That's not an inherent limitation of the 'syntax-case' design. It's
merely an unfortunate attribute of the psyntax _implementation_ of
'syntax-case', apparently because they didn't care enough about
bootstrapping issues to write psyntax without the benefit of macros.
'syntax-case' could certainly be implemented without using a
pre-existing macro expander.
Post by Mark H WeaverBut Racket has some facilities to step and debug macros, as you can
see here https://docs.racket-lang.org/macro-debugger/index.html
Aren' t Racket macros higienyc ?
Yes, of course, and we could certainly implement similar macro stepping
facilities in Guile. But that's not what you asked about in your
previous message. You asked about 'macroexpand-1', and my answer was
specifically about that. I don't see any procedure similar to
'macroexpand-1' in the document you referenced above.
Post by Mark H WeaverIn this question I've been promptly suggested a quick solution to
perform a single macro expansion step
https://stackoverflow.com/questions/50073207/macro-expansion-in-guile-scheme/50515880#50515880
For posterity, here's the quick solution suggested in the link above:
(define-syntax (expand1 stx)
(syntax-case stx ()
[(_expand1 form)
(syntax-case #'form ()
[(id . more)
(identifier? #'id)
(let ([transformer (syntax-local-value #'id)])
(with-syntax ([expansion (transformer #'form)])
#''expansion))]
[_
#''form])]))
This is just a toy, and not very useful in practice.
Here's the equivalent formulation for Guile:
(use-modules (system syntax)
(srfi srfi-11))
(define (syntax-local-value id)
(let-values (((type value) (syntax-local-binding id)))
value))
(define-syntax expand1
(lambda (stx)
(syntax-case stx ()
[(_expand1 form)
(syntax-case #'form ()
[(id . more)
(identifier? #'id)
(let ([transformer (syntax-local-value #'id)])
(with-syntax ([expansion (transformer #'form)])
#''expansion))]
[_
#''form])])))
(I usually prefer to avoid using square brackets in this way, but for
sake of comparison, I used them in the definition of 'expand1' above.)
Anyway, it works the same way as in Racket for this simple example:
scheme@(guile-user)> (expand1 (or 1 2 3))
$2 = (let ((t 1)) (if t t (or 2 3)))
So, what's the problem? The first problem is that when quoting the
resulting expansion, the binding information associated with identifiers
in the syntax objects are lost, so hygiene is lost. For example:
scheme@(guile-user)> (expand1 (or 1 2 t))
$3 = (let ((t 1)) (if t t (or 2 t)))
Moving on, let's use this to try to investigate how 'define-record-type'
works from SRFI-9 in Guile:
scheme@(guile-user)> ,use (srfi srfi-9)
scheme@(guile-user)> (expand1 (define-record-type <box>
(box value)
box?
(value unbox set-box!)))
$4 = (%define-record-type #f (define-record-type <box> (box value) box? (value unbox set-box!)) <box> (box value) box? (value unbox set-box!))
scheme@(guile-user)> (expand1 (%define-record-type #f (define-record-type <box> (box value) box? (value unbox set-box!)) <box> (box value) box? (value unbox set-box!)))
While compiling expression:
Wrong type to apply: (%define-record-type guile-user)
scheme@(guile-user)>
So what went wrong here? The problem is that '%define-record-type' is a
private macro, used internally within (srfi srfi-9), and therefore not
bound in the (guile-user) module where I'm working. If we had been
working with syntax objects, each identifier within the expression would
have been annotated with the specific binding that it refers to, but as
I noted above, that information has been stripped.
The awkward error message is because this toy implementation doesn't
check if the identifier is a macro or not.
One way we could try to improve this is to write 'expandN', which
performs N macro expansion steps, keeping them as syntax objects during
the intermediate steps:
(use-modules (system syntax)
(srfi srfi-11))
(define (syntax-local-type id)
(let-values (((type value) (syntax-local-binding id)))
type))
(define (syntax-local-value id)
(let-values (((type value) (syntax-local-binding id)))
value))
(define-syntax expandN
(lambda (stx)
(syntax-case stx ()
((_expandN n form)
(let ((n (syntax->datum #'n)))
(and (number? n) (integer? n)))
(let ((n (syntax->datum #'n)))
(if (positive? n)
(syntax-case #'form ()
((id . _)
(and (identifier? #'id)
(eq? 'macro (syntax-local-type #'id)))
(let ((transformer (syntax-local-value #'id)))
(with-syntax ((expansion (transformer #'form))
(n-1 (datum->syntax #'id (- n 1))))
#'(expandN n-1 expansion))))
(_
#''form))
#''form))))))
Unfortunately, this is not quite right, because it fails to add "marks"
to the identifiers introduced by the macro transformers, and thus is not
fully hygienic, and variable capture may occur. However, it is better
than what we had before, and good enough to step further into
'define-record-type':
--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,pp (expandN 0 (define-record-type <box>
(box value)
box?
(value unbox set-box!)))
$2 = (define-record-type
<box>
(box value)
box?
(value unbox set-box!))
scheme@(guile-user)> ,pp (expandN 1 (define-record-type <box>
(box value)
box?
(value unbox set-box!)))
$3 = (%define-record-type
#f
(define-record-type
<box>
(box value)
box?
(value unbox set-box!))
<box>
(box value)
box?
(value unbox set-box!))
scheme@(guile-user)> ,pp (expandN 2 (define-record-type <box>
(box value)
box?
(value unbox set-box!)))
$4 = (begin
(define-inlinable
(box value)
(let ((s (allocate-struct <box> 1)))
(struct-set! s 0 value)
s))
(define <box>
(let ((rtd (make-struct/no-tail
record-type-vtable
'pw
default-record-printer
'<box>
'(value))))
(set-struct-vtable-name! rtd '<box>)
(struct-set! rtd (+ 2 vtable-offset-user) box)
rtd))
(define-inlinable
(box? obj)
(and (struct? obj)
(eq? (struct-vtable obj) <box>)))
(define-tagged-inlinable
((%%type <box>)
(%%index 0)
(%%copier %%<box>-set-fields))
(unbox s)
(if (eq? (struct-vtable s) <box>)
(struct-ref s 0)
(throw-bad-struct s 'unbox)))
(define-syntax-rule
(%%<box>-set-fields check? s (getter expr) ...)
(%%set-fields
<box>
(unbox)
check?
s
(getter expr)
...))
(define-inlinable
(set-box! s val)
(if (eq? (struct-vtable s) <box>)
(struct-set! s 0 val)
(throw-bad-struct s 'set-box!))))
scheme@(guile-user)>
--8<---------------cut here---------------end--------------->8---
Unfortunately this is as far as we can go with 'expandN', because it
only expands macros at the top-level of the expression. In this case,
the top-level expression is a 'begin' form, which is a core form. At
this point, a real macro expander descends into the core form and
expands subexpressions, but in order to do this properly, it needs to
understand the meanings of the core forms that it's descending into.
For example, when descending into a 'let' form, it needs to take note of
the variables that are bound by the 'let'. For example:
scheme@(guile-user)> (expandN 0 (or 1 2 3))
$2 = (or 1 2 3)
scheme@(guile-user)> (expandN 1 (or 1 2 3))
$3 = (let ((t 1)) (if t t (or 2 3)))
scheme@(guile-user)> (expandN 2 (or 1 2 3))
$4 = (let ((t 1)) (if t t (or 2 3)))
The last two outputs are the same because I made 'expandN' just smart
enough to notice that 'let' is not a macro, in which case it stops
gracefully without triggering an exception.
Hopefully this illustrates why the old 'macroexpand-1' procedure from
Guile 1.x, which works on plain S-expressions without extra binding
information, and which only expands macros at the top level of the
expression, cannot be usefully implemented on a modern hygienic macro
expander.
However, what certainly *could* be done is some kind of interactive tool
to incrementally step macro expansions, while keeping track of the
syntax objects behind the scenes. To be useful in realistic cases, it
would need to understand most if not all of the core forms in Guile.
Those core forms are the ones defined using 'global-extend' in
psyntax.scm.
Regards,
Mark