Proposed change to SPECIAL: quick poll

[Scott E. Fahlman <Fahlman@CMU-CS-C>]

My earlier proposal to Guy was an attempt to solve the "top-level"
problem with minimal changes to the language.

I guess I read the manual differently than Moon on one point: I don't
see the business about allowing SPECIAL declarations at the start of forms
that do not bind the variable in question.  If it is really necessary to
use the special value of a symbol that is not globally special, there is
always SYMBOL-VALUE.

Of Moon's two solutions, I much prefer the second: reserving DECLARE for
local declarations and using PROCLAIM for global ones.  This change
would make no difference for most forms, but would clearly separate the
two uses of SPECIAL, just as my GLOBALLY-SPECIAL proposal would have.
Moon's first proposal, that we just trat all non-local uses of SPECIAL
as globally special without flagging an error seems a bit dangerous to
me.

As long as we're worrying about this, let me raise a problem that I
hesitated to raise earlier.  In defining Common Lisp, we've been pretty
cavalier about including constructs that would slow the interpreter
down, as long as the compiler could produce fast code.  However, the
current rules for SPECIAL slow the interpreter down a lot, probably more
than we can afford.  SPECIAL (and its inverse if this makes a comeback)
is the only declaration that the interpreter has to worry about.
Because of specials, every time we enter a lambda in the interpreter we
have to grovel down into the body looking for instances of (declare
(special ...)) and macros that might expand into the same.  THEN
we can do the binding and go on with the call.  That's all very
expensive.  Maybe there's a better way.

Currently Common Lisp recognizes two classes of declarations: those that
pertain to some particular binding of a variable and all the others.
The former class of declarations really ought to be in the lambda list,
closely associated with the variable-instances they modify.  This was
ruled out on the grounds that lambda lists need to be human-readable and
that much extra hair would push them over the brink.  So we do the next
best thing and hide these declarations in a DECLARE right after the
lambda list.  The situation is further complicated by documentation
strings, multiple DECLARE forms, and macros that might expand into
declarations.  All of these add to the cost of finding the declarations
relevant to a given binding, but that's OK in the compiler.  It is not
OK in the interpreter, however, and we can't ignore these declarations
since there may be a SPECIAL hiding in there.  This will be rare, but we
pay the price for this possibility on every call.

Suppose we recognize the fact that SPECIAL (and its inverse, if any) are
fundamentally different kinds of declarations from the rest.  Since they
matter to the interpreter, they must be easily accessible.  I would
propose something like the following:

1. (declare (special foo)) is allowed only at top level -- it is
undefined what it does elsewhere.  This form declares foo to be
globally special wherever it appears from that time forward.  (This is
easily implemented by putting some sort of SPECIAL property on the
symbol.)  If we want, we can define a (declare (unspecial  ...)) that
undoes this.

2. We may sometimes want a particular binding of FOO to be special,
without declaring the symbol FOO to be globally special.  We do not use
an embedded declare for this.  Instead, we mark this variable as special
right in the lambda list where it occurs.  We need some syntax to make
this non-obnoxious.  Perhaps #^foo would be good.  This expands into
something like (&special foo) internally, and the interpreter can now do
the binding without all the added hair.  Any reference to FOO that is
lexically within the scope of this binding form and that is not shadowed
by a lexical binding of FOO would refer to the special value.  We could
allow #^ to be used with variables to force the (symbol-value ...)
interpretation.  

So, in doing a binding the interpreter would look first for #^ (or
rather whatever this expands into), then for a special property on the
symbol, and if it finds neither of these the binding is local.  In
evaluating a symbol, the interpreter looks in the local lexical
environment, where there may be a notation to use the special value of
the symbol instead.  #^FOO is equivalent to (symbol-value foo) in all
contexts.

I suppose we could also define another bit of syntax, #_ maybe, for
making a particular value lexical when it would otherwise be special.
I've never had much use for this or for UNSPECIAL, but someone may have
an argument for why this is necessary.  (These sharpm characters are
somewhat mnemonic to me: #^ says fly off to find the value, while #_
says stay on the ground.  Neither seems to be in use right now.)

I know that this is pretty radical, especially at this late date, but it
seems to follow inevitably from our earlier radical decision to make the
interpreter pay attention to specialness.  I hate to see even this much
syntax added to lambda lists, but it may be preferable to a very
substantial slowdown in the interpreter.  #^ would be very rare in real
programs, since the vast majority of specials would be globally declared
by DEFVAR and friends.

-- Scott