package proposal comments

[DILL@CMU-CS-C]

Here are some thoughts about alternatives in the design of the CL
package system, after looking over the original Fahlman proposal and the
comments by Symbolics.  I've tried to separate out considerations that
are mostly independent, and tried to make explicit any assumptions about
how packages are to be used.  I won't present a lot of detailed
alternatives, because Scott has probably seen them before in previous
proposals.

TRANSITIVE VS. INTRANSITIVE "INHERITANCE":

As a general principle, users should be able to ask for only the symbols
(definitions, whatever) they need from a package.  This reduces the
chances of an accidental name conflict among symbols that the user
didn't need anyway.  If transitive inheritance is the primary way for
package A to gain access to package B's definitions, it violates this
principle, because it automatically re-exports all sorts of symbols that
may have nothing to do with the facilities that package A wants to
provide.

For a really huge system (e.g. MACSYMA or the LISP system itself), the
implementors may want to think of the system as being made up of a bunch
of reasonable-sized packages, while users may want to think of it as
a single package (MACSYMA or LISP).  For this purpose, it may be useful
for a package to group together symbols from other packages and export
them in a bunch.  It is not clear that you need more than one level of
"inheritance" for this, though.

By the way, there seems to be a reasonably clean way to do multiple
interfaces to the same package if it is possible to re-export a symbol
that has been imported from somewhere else.  Export the union of the
symbols for all of the interfaces from the package that does the
actual definitions, then import these into a separate new package
for each interface.  Then each of these new packages can export
only those symbols that are relevant to the particular interface
it is associated with.  If this is really worth doing...



"COPY" VS. "POINTER" SEMANTICS FOR IMPORTED SYMBOLS

There have been two general types of package proposals within the
Spice Lisp group in the last two years: when package A uses package B's
symbols, does it get the symbols exported by package B at one particular
time ("copy"), or does it have some sort of pointer to B, so that it can
track changes in B's exported symbols over time ("pointer")?  "Pointer"
semantics seem to require an implicit or explicit order for searching
export lists.

When it is desirable to track changes, pointer semantics is sometimes
more convenient.  The two cases (that I can think of) where changes in
export lists might happen are when someone is debugging a package and
wants to correct a mistake in an export list, and when someone has
decided to export a new symbol from an old package (only really
legitimate when exporting from the LISP package, I think, if then).

This does not necessarily work as well as it ought to.  Usually,
symbols will have been read and interned using one configuration
of export lists, and when the export lists change, there will still
be no easy way of correctly updating the old symbols to be what
they should have been.

The most serious disadvantage of the pointer semantics is that
accidental shadowing is inherent when you attempt to track changes in
export lists.  Suppose package A searches (in order) the export lists of
packages B and C.  If a symbol is added to the export list of package B,
it could shadow a symbols in C that A was previously using.  Since there
are a lot of packages that could be searching B and C in different
orders (and a lot of different values for B and C), this situation is
very difficult to check for and correct.

With copy semantics, a package always knows what symbols it has gotten
from another package immediately after it gets them.  If a conflict
arises, it can be resolved at that time, and no shadowing can occur
without explicit participation by the packages affected.  If symbols are
added to an export list, it is relatively easy to ask for all the new
symbols (but you still have to ask).  It may be more difficult to forget
symbols that have been removed from an export list, since it may be
difficult to tell where they were imported from (not necessarily the
home package, due to re-exporting).

The current proposal seems to use "copy" semantics for IMPORTing
an individual symbol, and "pointer" semantics for using all the
symbols exported by another package: IMPORTed symbols will not
be tracked if they are removed from an export list.  In pure
"copy" semantics, getting the export list of another package is
equivalent to individually importing all those symbols.

Some of these opinions are based on experience I have had using
the MUDDLE package system, which is basically a "pointer" system
with intransitive inheritance.  In my experience, changing an
export list almost never "worked" except when exporting from the
MUDDLE equivalent of the LISP package.


PACKAGES AS A UNIT OF LOADING

If it is assumed that packages correspond closely to a set of
definitions that should be loaded together, we could provide a function
to load a package, given only the package name, only if it has not
already been loaded.  The system can find the appropriate file to load
from, since the user probably doesn't care as long as the package gets
loaded.  Loading only if not already loaded is a win because many
different packages may import from the same package -- it would be a
pain to have to check in each case whether to load it.  This also makes
mutually referential packages (package A imports from package B which
imports from A) easy.

IMPORTing from a package could cause it to be loaded if necessary, as
above.  MACLISP style autoloading is not necessarily a substitute for this
capability; if there are "autoload" properties on symbols, the packages
defining those symbols will have to be defined to the extent of having
created the package object and storing the relevant symbols somewhere.


PACKAGES AS A UNIT OF COMPILATION

A package system that clearly separates out the external and internal
symbols can make it possible for a compiler to do a better job, since
it can assume that internal definitions will not be accessed outside
of the package.  There are a lot of "block compilation" kinds of
optimizations that could be done.

This is not something that is generally done in the Maclisp/Lisp Machine
world now, but I think we ought to leave open the possibility of doing
it in some implementations in the future.  Two ways to avoid losing
this opportunity are to have a warning in the manual that you should
only use internal symbols from another package if you absolutely know
what you're doing, and having a different form of qualification for
internal symbols (I favor "::" but basically don't care).

I think I can generate a list of interesting optimizations if anyone is
interested.
-------