"lexical" packages

[masinter.pa@Xerox.ARPA]

Rebuttals:

Fahlman:

"When we're speaking of the scope variables, lexical scoping means that
the scope of the variable is one contiguous chunk in the file being
read, and once you leave the lexical block you can never again get hold
of that variable by using its name (though function closures defined
within the block can still get at the variable).  You cannot sneak in
one more function later that refers to the same X.  That's awkward at
times, but also an advantage, since the compiler can be sure that after
it has reached the end of a block, it knows all about how a lexical
variable in that block can be referenced."

Of course this isn't true, lest we be reduced to using core dumps to
debug. Of course, the debugger and the environment can reasonably be
expected to provide a way to access "lexical" variables. But such
mechanisms are part of CLTE rather than CLTL. The language has semantics
with invariants (if you (setq x --) here and call a function, you can
still expect x to have the same value), and the environment gives you
tools to break those invariants. Thus, while I propose making the
"package" a lexical property (more on that later), any reasonable
environment would provide ways of dynamically switching.

Moon:
"T has some reasonable responses to these issues, and I think if Common
Lisp were being designed in 1987 we might decide differently after
studying languages such as T and how their users use them.  But Common
Lisp was designed in 1982.  We decided to go with the well-understood
package philosophy that the Lisp machine had been using since about
1978, substantially redesigned to fix its well-understood bugs without
radically changing the whole philosophy.  It's hard to achieve
perfection in this mortal vale.  For additional insight, see the James
Madison quotation on the second page of the manual."

This is one of the weakest arguments to make -- it can be applied to any
of the discussions on this or any of the other mailing lists about
Common Lisp. If there's no chance to remove some of the mistakes in
Common Lisp, there seems to be no grounds for further discussion.

- - - - - - -

Restatement of proposal:

Packages should rightfully be lexical (determined by character sequences
in the file) rather than a dynamic (determined by what you've recently
executed). 

Let me make a specific proposal:

Just as (DECLARE --) is a special form, and "it is an error" to attempt
to evaluate a declaration, I propose that 

provide, in-package, shadow, export, require, use-package, import

all be documented as as special forms; they may only occur at the
beginnings of files. They are special markers that load, the compiler,
and the editor etc. can look for. (The wording for this can be taken
almost directly from the documentation of "declare" in CLTL p 153).

This turns what was formally a convention into a requirement. It means
that editors/file packages/etc. can actually look at a file and
determine (by scanning the beginning of the file) what packages might be
used/exported/etc. by the file. It makes illegal most of the odd (bad
programming practice) things that can go wrong with the package system.
It removes the requirement that the whole file be read, parsed and
executed in order to get a particular function. Many of the caveats in
the package section could be removed.

For those who already have an implementation of the current package
mechanism, the burden of this new scheme is minimal -- they can add
error checking, but need not.