Proposed change to SPECIAL: quick poll

[MOON at SCRC-TENEX]

I started to reply to this yesterday, saying "yes, that's the right thing."
Then I stopped and thought about it.  I also talked it over with Glenn Burke
and then slept on it.  This is going to have to be a very long letter and
I apologize in advance.

I am convinced that you have got hold of the wrong end of the stick.  First
of all, none of this has anything to do with the SPECIAL declaration per se.
You are just thinking in terms of the SPECIAL declaration because that is
the only one the interpreter looks at.  Note, for instance, that if the
UNSPECIAL (or LEXICAL or LOCAL) declaration hadn't been (temporarily?) removed
from the language as noted on page x of the Laser edition of the manual,
the interpreter would need to look at those declarations as well, and
precisely the same issues would arise.

My first reaction was "yes, the concept of top-level is ill-defined, let's
get rid of it."  But "top-level" is perfectly well-defined!  See the first
sentence under DECLARE on page 101 of the manual:  A DECLARE form at the
beginning of the body of certain special forms is a local declaration;
all other DECLARE forms are either top-level or errors.  Now, what may
be ill-defined is when you should signal the error versus when you treat
it as top level; but this is entirely an issue for DECLARE, has nothing
to do with SPECIAL, and would not be affected in any real way by renaming
SPECIAL to GLOBALLY-SPECIAL.

I feel reluctant to pontificate about this, since you have written a
Common Lisp interpreter and I have not.  But I see no way that one could
possibly implement the interpreter without having lambda-expressions,
and each special form that does bindings and hence allows local declarations,
specially check the beginning of their body for DECLARE forms, or macros
that expand into DECLARE forms.  It is necessary to know what the declarations
are before doing the bindings; if you go ahead and do the bindings and let
EVAL find the DECLARE while doing the body, it is too late to change
a binding from local to special, isn't it?  Thus there is no issue of
distinguishing local and top-level declarations in the interpreter; all
DECLARE forms found by this special check are local, and all others are
top-level, or errors.

Glenn points out that it isn't really as impossible as all that to
implement declaration in the interpreter according to the specifications in
the Laser edition, since he has already done it.

In fact I am quite willing to believe that it is impractically difficult
to implement the second-to-last sentence on page 101 (that an error will
be signalled if a DECLARE appears out of place) in the interpreter, and
that this would have to be left to the compiler.  The various issues
in Fahlman's letter having to do with searching the stack and so forth
seen to be really about this.

If it is important to distinguish erroneous DECLAREs from top-level
DECLAREs in the interpreter, then we should give the two kinds of DECLARE
(local and top-level) different names.  Of course people might then expect
the top-level one, when used inside a function, to have a scope restricted
to that function....

Seventeen months ago we decided to get rid of GLOBAL-DECLARE (issue 68 from
18 November 1981).  Perhaps this was a mistake.  If GLOBAL-DECLARE is a
poor name, we could call it PROCLAIM.  Then DECLARE would signal an error
if EVAL ever saw it, while PROCLAIM would work the same way anywhere in the
interpreter but would be required to appear only at top level by the
compiler, exactly in analogy with DEFUN.

If there is "an ugly and confusing exception" here, it is not the behavior
of SPECIAL declarations at top level.  The exception is the peculiar syntax
for local declarations, which are tucked inside the special form they apply
to instead of being wrapped around it like everything else in Lisp.
Compare the LOCAL-DECLARE special form of the Lisp machine.  Of course, the
wrapping-around syntax for declarations has its own deep problems: in LET
one could not apply a declaration to the variables being bound but not to
the forms to which they were bound; the wrapping-around syntax would be
highly misleading in combination with Common Lisp's non-pervasiveness of
local declarations (which is different from Maclisp), since it doesn't make
it obvious that the declarations are attached to one particular binding,
not to everything lexically enclosed in a LOCAL-DECLARE.  There seem to be
good reasons for non-pervasive local declarations, on grounds of both
substitution semantics (we don't want inline functions to inherit declarations
of their container's variables) and interpreter efficiency (we don't want
the interpreter to have to maintain a dynamic declaration list and search
it when binding a variable in order to figure out whether the variable
should be special or local).

Another possible syntax for local declarations is to attach them even more
directly to the binding, say something like
	(let ((x (frobdicate y) special)) ...)
This has been discussed before, and it has its own set of problems, which
appear to be insuperable.  (Different syntax for every binding form, doesn't
fit nicely into DEFUN, doesn't provide a reasonable place to put non-binding-
associated local declarations).

I have to bring up one other issue which Fahlman's letter glossed over.
This part I think is specific to the SPECIAL declaration and applies to no
other.  It is legal to make a local SPECIAL declaration without binding the
variable.  This isn't explained very well in the Laser edition, but the
idea is that such a SPECIAL declaration affects exactly the same references
as it would if the variable had been bound by the form to which the
declaration is attached.  It's pretty easy to see how to implement this in
the interpreter: by putting the same marker as you normally would into the
lexical environment structure, saying "use the dynamic value", but avoiding
the creation of a dynamic binding.  Scott's letter might be interpreted as
trying to get rid of this unusual usage of the SPECIAL declaration, or it
might be interpreted as a mere oversight.  I don't see how it could
possibly be reasonable to get rid of it, so I assume it's just an
oversight.  A typical example of the use of this might be:

	(defun foo (x y)
	  (let ((foo-bar-communication x))
	    (declare (special foo-bar-communication))
	    (frob #'bar y)))

	(defun bar (z)
	  (declare (special foo-bar-communication))
	  (cdr (assoc z foo-bar-communication)))


Conclusions:

I suggest that we either

(1) Leave everything the way it is in the Laser manual, except specify
that the interpreter cannot be expected to detect misplaced declarations
and will treat all declares that aren't local as top-level, even if they
are inside a form.  A misplaced declaration will alter your global
environment with no warning.

or

(2) Say that DECLARE is allowed only at the front of the bodies of the
forms listed on page 101 (possibly preceded by a documentation string
in the places where one is allowed), and introduce a new special form,
PROCLAIM, which makes top-level declarations no matter where it is used.
It might make sense to make PROCLAIM a function rather than a special
form; this would certainly clarify its difference from DECLARE.

I have no personal preference between these.