compiling anonymous functions

["Scott E. Fahlman" <Fahlman@C.CS.CMU.EDU>]

Disclaimer: All of the following is a suggestion for how the language
specification might be improved, once we have a mechanism in place for
considering such improvements.

Clearly the whole section on top-level forms needs to be reconsidered
and tightened up.  We got off on the wrong foot on that one, and never
recovered.  Probably we should flush the whole notion of top-level forms
and just provide some rules about what the compiler does.  We can
probably go ahead and assume that there is a compiler of some sort and
that it is written in Common Lisp without saying much about what code it
generates or how intensively it hacks things.  What we need to specify
more clearly are which things get compiled (whatever that may mean) and
which forms are "seen" by the compiler in the sense of causing side
effects -- macro definitions, constant definitions, declarations, and
the like.

I think that the following are more or less standard practice now, and
should probably be required (or at least strongly suggested and assumed
by portable code libraries) at some future time:

1. Occurrences of (function (lambda ...)) within forms that are being
compiled are themselves compiled.  So if a function definition is

(defun foo () (setf x #'(lambda () (mumble))))

then if we compile foo and then call it, X will contain a function
object that is compiled as if it had been a top-level defun.  Note that
if you want to set X to an uncompiled expression, you can use '(lambda
...). 

2. Embedded Defuns get compiled, roughly as if they were embedded
(setf (symbol-function <name>) (function (lambda ...))) forms.
We don't want to force users to write the long form in order to ensure
compilation.  If an implementation has defun doing additional things,
such as maintaining data structures for the programming environment,
then it is up to the implementor to do the right thing when compiling
embedded Defuns, but the actual definition still should end up compiled.

3. "Random" Top-level forms (anything whose semantics are not special to
the compiler) are compiled and then executed at load time.  So if a file
being compiled contains

(mumble ...)

it is compiled more or less as if it were

(defun <made-up-name> () (mumble ...))
(<made-up-name>)

The point is not to force all implementations to execute these things
fast at load time, though that is certainly a plus for any
implementation.  The point of including this in the standard is to
standardize what happens when the random form contains constructs that
might have side effects on the compiler or sub-forms that might want to
be compiled.  (This was discussed at some length before the manual came
out, but I don't think the manual currently takes a stand on this.)

So, if all the above were to become standard, a user could employ an
idiom like

(let ((x 0) (y 1024))
  (defun foo () ...)
  (defun bar () ...))

and be sure that FOO and BAR would be compiled, whatever that means for
a given implementaiton.  I don't think this is guaranteed now, though it
is an idiom I use a lot.

I think that this subsumes Snyder's suggestions for extending the list
of forms whose interiors are considered to be "top-level", at least on
the issue of what gets compiled.  The question of where you can put
PROCLAIM forms and the like might be more complicated.

-- Scott