Lexical references in DEFMACRO

[edsel!sunvalleymall!eb@su-navajo.arpa (Eric Benson)]

The Common Lisp definition continues to surprise me.  I had not
noticed that DEFMACRO is not allowed to let its body refer to any
lexically visible entities outside the DEFMACRO form.  On the other
hand, DEFUN is specifically allowed to refer to lexically visible
entities.  This deserves some explanation.  It seems I am allowed to
do this:

(LET ((SOME-DATA-STRUCTURE (MAKE-SOME-DATA-STRUCTURE)))
  (DEFUN SOME-FUNCTION ()
    (DO-SOMETHING-WITH SOME-DATA-STRUCTURE)))

but not this:

(LET ((SOME-DATA-STRUCTURE (MAKE-SOME-DATA-STRUCTURE)))
  (DEFMACRO SOME-MACRO ()
    (MAKE-A-FORM-WITH SOME-DATA-STRUCTURE)))

To do what I want in the second case I must do this:

(LET ((SOME-DATA-STRUCTURE (MAKE-SOME-DATA-STRUCTURE)))
  (SETF (MACRO-FUNCTION 'SOME-MACRO)
	#'(LAMBDA (FORM ENVIRONMENT
	    (DECLARE (IGNORE FORM ENVIRONMENT))
	    (MAKE-A-FORM-WITH SOME-DATA-STRUCTURE)))))

Furthermore, in order to implement this restriction I must introduce a
new special form, such as that used in Spice Lisp, to force a form to
be evaluated in the null lexical environment.  Thus it is impossible
to live up to the suggestion in the implementation note on p.58, that
macros should not expand into implementation-dependent special forms.

It is obvious to me why the expansion functions defined by MACROLET
must have this restriction; there is no environment at macroexpansion
time to which they can refer.  The expansion functions created by
DEFMACRO, on the other hand, are not really very different from
ordinary functions created by DEFUN.

There is one difference, but it is outside the scope of the Common
Lisp definition.  Most (i.e. all) implementations of Common Lisp cause
macros defined by DEFMACRO to be available for use while compiling the
file in which they are defined.  This is not required by the Common
Lisp definition, but is part of the Lisp tradition.  It should be
obvious that this is only possible for macro definitions which occur
at "top level."  For example, it would be incorrect in this example to
define THIS-MACRO at compile time:

(WHEN SOME-CONDITION
      (DEFMACRO THIS-MACRO ()
	(EXPAND-INTO-SOMETHING)))

Similarly, any macro definition which refers to a lexically visible
entity cannot be defined at compile time, since it cannot be a "top
level" form.

I suggest one of the following be done in the next edition of the manual:

1. Remove the sentence in the description of DEFMACRO which disallows
references to lexically visible entities.  This is an incompatible
change, but is unlikely to affect many programs.  It is mostly an
upward-compatible change.

2. Add an explanation which justifies this restriction.  The
explanation would have to include some language explaining how this
could be implemented, since it appears to contradict the
implementation note on p.58.