macro expansion

[David A. Moon <Moon at SCRC-TENEX at MIT-MC>]

Here is my promised proposal, with some help from Alan.

MACRO-P becomes a predicate rather than a pseudo-predicate.
Everything on pages 92-93 (29July82) is flushed.

Everything, including the compiler, expands macros by calling MACROEXPAND
or MACROEXPAND-1.  A variable, *MACROEXPAND-HOOK*, is provided to allow
implementation of displacing, memoization, etc.

The easiest way to show the details of the proposal is as code.  I'll try to
make it exemplary.

(DEFVAR *MACROEXPAND-HOOK* 'FUNCALL)

(DEFUN MACROEXPAND (FORM &AUX CHANGED)
  "Keep expanding the form until it is not a macro-invocation"
  (LOOP (MULTIPLE-VALUE (FORM CHANGED) (MACROEXPAND-1 FORM))
	(IF (NOT CHANGED) (RETURN FORM))))

(DEFUN MACROEXPAND-1 (FORM)
  "If the form is a macro-invocation, return the expanded form and T.
  This is the only function that is allowed to call macro expander functions.
  *MACROEXPAND-HOOK* is used to allow memoization."
  (DECLARE (VALUES FORM CHANGED-FLAG))

  (COND ((AND (PAIRP FORM) (SYMBOLP (CAR FORM)) (MACRO-P (CAR FORM)))
	 (LET ((EXPANDER (---get expander function--- (CAR FORM))))
	   ---check for wrong number of arguments---
	   (VALUES (FUNCALL *MACROEXPAND-HOOK* EXPANDER FORM) T)))
	(T FORM)))

;You can set *MACROEXPAND-HOOK* to this to get traditional displacing
(DEFUN DISPLACING-MACROEXPAND-HOOK (EXPANDER FORM)
  (LET ((NEW-FORM (FUNCALL EXPANDER FORM)))
    (IF (ATOM NEW-FORM)
	(SETQ NEW-FORM `(PROGN ,NEW-FORM)))
    (RPLACA FORM (CAR NEW-FORM))
    (RPLACD FORM (CDR NEW-FORM))
    FORM))

The above definition of MACROEXPAND-1 is oversimplified, since it can
also expand other things, including lambda-macros (the subject of a separate
proposal that has not been sent yet) and possibly implementation-dependent
things (substs in the Lisp machine, for example).

The important point here is the division of labor.  MACROEXPAND-1 takes care
of checking the length of the macro-invocation to make sure it has the right
number of arguments [actually, the implementation is free to choose how much
of this is done by MACROEXPAND-1 and how much is done by code inserted into
the expander function by DEFMACRO].  The hook takes care of memoization.  The
macro expander function is only concerned with translating one form into
another, not with bookkeeping.  It is reasonable for certain kinds of
program-manipulation programs to bind the hook variable.

I introduced a second value from MACROEXPAND-1 instead of making MACROEXPAND
use the traditional EQ test.  Otherwise a subtle change would have been
required to DISPLACING-MACROEXPAND-HOOK, and some writers of hooks might get
it wrong occasionally, and their code would still work 90% of the time.


Other issues:

On page 93 it says that MACROEXPAND ignores local macros established by
MACROLET.  This is clearly incorrect; MACROEXPAND has to get called with an
appropriate lexical context available to it in the same way that EVAL does.
They are both parts of the interpreter.  I don't have anything to propose
about this now; I just want to point out that there is an issue.  I don't
think we need to deal with the issue immediately.

A related issue that must be brought up is whether the Common Lisp subset
should include primitives for accessing and storing macro-expansion
functions.  Currently there is only a special form (MACRO) to set a
macro-expander, and no corresponding function.  The Lisp machine expedient of
using the normal function-definition primitive (FDEFINE) with an argument of
(MACRO . expander) doesn't work in Common Lisp.  Currently there is a gross
way to get the macro expander function, but no reasonable way.  I don't have
a clear feeling whether there are programs that would otherwise be portable
except that they need these operations.