Declarations

["David A. Moon" <Moon%SCRC-TENEX@MIT-MC.ARPA>]

I have an alternate proposal for declarations.  It is mainly intended as
an easier-to-understand way of explaining things that gives the same
semantics, but it also resolves the inconsistencies in the manual (Mary
Poppins and Excelsior editions) in a way that is probably inconsistent with
the intent of the manual, but is more consistent with the Laser edition and
with the practice in previous lisps such as Maclisp and Lisp Machine Lisp.
I realize it is rather late for this kind of thing, but given that the
manual is inconsistent with itself and that some people were surprised by
the change from last year's version of the language, maybe we can reconsider.
In any case this way of explaining things seems to be easier to understand.

The basic problem is the SPECIAL declaration, which falls into both categories
of declarations: it both concerns the binding of variables and is pervasive.
This is not well explained in the manual and seems to be a great source of
confusion.  I propose to flush the idea that SPECIAL declarations are
pervasive.  Read on before judging.

There are two disjoint classes of declaration: those that are attached
to a particular variable binding, and those that are not.  Note that I
am not discussing proclamations here; they have their own scoping rules
which are different from the rules for declarations.

The scoping rule for the first kind of declaration is that it applies to
precisely the text that is in the lexical scope of the variable binding
with which it is associated.  Such declarations are shadowed by a
variable binding for the same name inside their scope.  Since the lexical
scoping rules are very well and precisely defined, we already understand
everything about this kind of declaration.

The scoping rule for the second kind of declaration is that it is
pervasive.  The declaration is attached to a lambda-expression or to a
form (one of the special forms listed on page 125).  The declaration
applies to all text inside that expression or form; there are no special
cases such as init-forms of LET or DO.  Such declarations are shadowed
by a conflicting declaration inside their scope.

Possible names for the two kinds of declaration are "lexical" and "pervasive"
or "variable" and "non-variable."

None of this is new.  What about SPECIAL declarations?  I propose that SPECIAL
declarations be put entirely into the first category, in the following way:

When a declaration, (SPECIAL X), is attached to a form (or lambda-expression)
it creates a -lexical- binding of the name X in that form.  Unlike all other
bindings, this binding does not associate a value with the name X.  Instead,
it associates a "special" marker with the name.  Any text that is in the
lexical scope of this binding, and uses the variable X, refers to the dynamic
binding of X, because it sees the "special" marker.  The scope of a SPECIAL
declaration is precisely defined by the scope of this binding.  Note how close
this definition is to the way that some interpreters actually work internally.

In addition to creating this lexical binding, a (SPECIAL X) declaration also
changes the meaning of a binding, if there is one, of the variable X in the
form to which the declaration is attached; it causes that binding to be a
dynamic binding rather than a lexical binding.  This applies only to the form
to which the declaration is directly attached, not to any subforms of it.
(This is not new.)

The declaration-allowing forms FLET, LABELS, LOCALLY, and MACROLET do
not normally create bindings of variable names (only function names), so
we have a slight exception, permitting SPECIAL declarations inside them
to create such bindings.

We now understand completely how the SPECIAL declaration works in LET.
Three examples:
	(defun foo (x)
	  (let ((x x))
	    (declare (special x))
	    (bar x)))
Reading from left to right, there are five occurrences of x.  The
declaration causes the second and fifth to refer to a dynamic variable,
while the first and third refer to a lexical variable.  (The fourth
occurrence is the one in the declaration itself).  The third occurrence
of x does not refer to the dynamic variable, because it is not in the
scope of the lexical binding of the name x created by the special
declaration, because of the way LET defines scoping in its subforms.
	(defun foo ()
	  (declare (special x))
	  (bar x))
The declaration causes the second occurrence of x to refer, freely, to
a dynamic variable, because this x is inside the lexical scope of a binding
of the name x attached to the defun.
	(defun foo ()
	  (let ((x (quux)))
	    (mumble x)
	    (locally
	      (declare (special x))
	      (bar x))))
Here the first and second occurrences of x refer to a lexical variable,
while the fourth refers to a dynamic variable.  The binding of the name
x in the LOCALLY shadows the binding of the name x in the LET.

There remains the issue of how sequential binding forms such as LET* and DEFUN
should work.  (DEFUN uses sequential binding with respect to initial value
forms for &optional, &key, and &aux variables).  When a SPECIAL declaration
creates a lexical binding of a name in such a form, at what position in the
sequence of bindings should it be inserted?  This makes a difference because
it affects whether the scope of the declaration includes or excludes the
initial value forms.  Clearly if the form includes a (dynamic) binding of the
name being declared, then the (lexical) binding created by the declaration
should be inserted at that point.  Otherwise we can either insert the
declaration's binding at the beginning of the sequence of bindings, giving it
the largest possible scope, or at the end, giving it the smallest possible
scope.  I suggest putting it at the beginning, because that is what people
seem to expect with DEFUN (see the example at the bottom of page 126.)
Two examples:
	(defun foo (x)
	  (let* ((x (bar x))
		 (y (quu x)))
	    (declare (special x))
	    ...))
The first and third occurrences of x refer to a lexical variable, while
the second and fourth refer to a dynamic variable.  (bar x) is outside
the scope of the special declaration while (quu x) is inside its scope.
	(defun foo (x)
	  (let* ((w (bar x))
		 (y (quu x)))
	    (declare (special x))
	    ...))
The first occurrence of x refers to a lexical variable, while the second and
third refer, freely, to a dynamic variable.  Both (bar x) and (quu x) are
inside the scope of the special declaration.  If let had been used instead of
let*, neither of them would be inside the scope of the declaration.

For explanatory purposes, the above two examples could be written on
paper as
	(defun foo (x)
	  (let* ((x (bar x))
		 (x @I[special])
		 (y (quu x)))
	    (declare (special x))
	    ...))

	(defun foo (x)
	  (let* ((x @I[special])
		 (w (bar x))
		 (y (quu x)))
	    (declare (special x))
	    ...))