Package Proposal (moby message)

[David A. Moon <Moon%SCRC-TENEX%MIT-MC@SU-DSN>]

Since the mailing list seems to be working again, here's the response to Fahlman's
package proposal from people at Symbolics.  I think we are very close to converging,
in spite of the voluminous nature of these comments.

A summary of points made is at the end.
-------
List the standard consistency (sanity) rules for package systems:

  If the value of *package* is not changed:

  - read-read consistency
    Reading the same print name always gets you the same (EQ) symbol.

  - print-read consistency
    An interned symbol always prints as a sequence of characters which,
    when read back in, yields the same (EQ) symbol.

  - print-print consistency
    If two interned symbols are not EQ, then their printed representations
    will not be the same sequence of characters.

  These consistency rules remain true in spite of any amount of implicit
  interning caused by typing in Lisp forms, loading files, etc.  This has
  the important implication that results are reproducible regardless of the
  order of loading files or the exact history of what symbols were typed
  in when.  The rules can only be violated by explicit action, such as
  changing the value of *PACKAGE*, calling one of the functions UNINTERN,
  IMPORT, SHADOW, INHERIT-PACKAGE, UNINHERIT-PACKAGE, or continuing from
  an error signalled by IMPORT or by reading an illegal qualified name.

  The purposes of the package system are two: to prevent naming conflicts,
  and to make the modular structure of programs more explicit.

The tree-structured directory of names, allowing two packages to have
the same name, has been invaluable for certain things (including
bootstrapping our present Common Lisp implementation), but does not work
completely.  The fact that this has subtle problems suggests that it is
better to leave it out of Common Lisp for now.  The incompatibilities
required to add it as an extension later can be made very small.

[I agree with the proposal as it stands.  We have had a lot of problems
with the current hierarchical package namespace stuff as it stands.
Most of our problems were because the very same hierarchy controlled
both package inheritance and controlled the namespace of package names
themselves.  Thus, if you wanted to inherit from SYS, which is under
GLOBAL, then your name isn't globally accessible.  So you had to invoke
a special mechanism to make your name globally accessible, which in
practice most things did.  The Common Lisp bootstrapping is about the only
case I can think of where we have actually used the hierarchical naming,
and boy is it a zoo keeping track of what's going on.  --DLW]

[In fact we have used it for other things.  Some of the zoo is not inherent
but is due to brain-damage in programs that weren't written with the
hierarchical naming in mind.  But I think we should leave it out of
Common Lisp, at least for now.--Moon]

Synonyms are very valuable, since people want to type short package
prefixes but things should have long descriptive names.  If the package
system doesn't remember the full name, then people will have to consult
documentation to find out what a package name abbreviation means.
However this can be added later.

[Our present package system uses synonyms heavily.  On the one hand,
it's desirable to use long, descriptive names, so that packages can
print out as, e.g., #<PACKAGE DEBUGGER>.  On the other hand, it's nice
to have short abbreviations, because you type them a lot when using the
"qualified name" syntax, e.g., dbg:frame-active-p,
dbg:with-erring-frame, etc.  This is because you usually get at symbols
in other packages by using qualified names, rather than by inheritance.
--DLW]

[BSG's only additional comment was that the balance of arguments is
on the side of synonyms.]

[Specifying synonyms in the recipient can replace hierarchical naming.
(In common lisp, SI means what is CL-SI globally).--MMcM]

Document that the package cell of an uninterned symbol contains NIL.

It would be very useful to have two kinds of qualified name, one for
externals and a second for internals.  This way externals need not be
put directly into one's main "name space," where the possibility of name
conflicts can arise.  Externals can be referenced by a qualified name,
but the important externals/internals distinction that tells you which
symbols in a module are interfaces you are allowed to use is retained.
A different kind of qualified name is used for sticking your fingers
where they don't belong.  Reading an external qualified name should
signal an error if you try to use an external that doesn't exist yet;
continuing from the error should fake an EXPORT to create the desired
external in the desired package.  However the keyword package is a
special case; you can always create new keywords.  I suggest #: for the
internal qualifier (keyword has no internals so this doesn't conflict
with uninterned syntax).  Double colon can remain reserved for future
extensions (we have some in mind).  There should be a function that does
what the external-qualified-name syntax does.  INTERN is the function
corresponding to the internal-qualified-name syntax.  This suggests an
obvious name for the function, which will be confusing to MACRO-10
programmers.

[Here we get to what is probably the most tricky issue in any package
system that distinguishes between internal and external symbols.  This
issue has been the major thing that has held up major changes in the
Zetalisp package system for so long.  I don't have any breakthroughs but
I think this issue deserves a good explanation and I threw in a few
comments.

Suppose I load into my Lisp environment a utility program containing
subroutines that deal with magtape I/O stuff.  This program is written
using the package system.  It lives in its own package, named TAPE, and
all of its internal functions are internal symbols of the TAPE package
so that they're hidden from the rest of the Lisp world.  It has several
entrypoints, which are external symbols of the TAPE package.  Now, I am
writing my own program (a backup dumper/loader) and I want to invoke the
magtape utilities.  There are two alternative schemes.  In scheme 1, I
inherit all of the external symbols of TAPE, and then I can just refer
directly to those symbols.  In scheme 2, I use a qualified name, saying
tape:do-this-or-that to get at the tape.

The proposal only appears to have scheme 1 in mind.  The qualified names
are referred to as being "desirable when typing an expression to be read
by the Lisp reader", with the implication that programs aren't really
supposed to use qualified names, that they're really more like a
debugging tool.  The qualified names search the internal symbols as well
as the external symbols.  Moon's comment is asking for some new
mechanism to help allow scheme 2 work as well; that is, I think Moon is
trying to design the package system in such a way that either scheme can
be used.

The advantage of scheme 2 shows up when you write a program that needs
to call functions that reside in two different packages.  Suppose we
need to call a function named FOO from the A package, but we also need
to call a function named FOO in the B package.  In scheme 1 we can't do
it at all (or need to resort to the colon, which is considered bad in
scheme 1).  A more subtle but more important problem is if we need a
function FOO from the A package and a function BAR from the B package,
but don't realize that B also has an external symbol named FOO, and so
we don't worry enough about the order of the inheritance list; then we
end up calling the wrong function at run-time.  Probably this is pretty
easy to find at run-time, as long as you understand about the problem.
You fix it by rearranging the order of packages and trying again, hopin
that there isn't any symbol on A with the name of a symbol that you need
on B.  If there is, again you're in trouble.

Knowing which scheme a package system uses has an effect on how you
choose your function names.  In the Lisp Machine right now, there are
two functions names chaos:make-stream and tape:make-stream.  The fact
that they have the same name doesn't cause any trouble, because the
present Zetalisp package system only uses scheme 2.  But if we belived
in scheme 1, we'd have chosen the names differently, probably using
make-chaos-stream and make-tape-stream.  (If you decide you want both
schemes, you have to use the latter set of names, and put up with the
redundancy of tape:make-tape-stream.)  In fact, in scheme 1, probably
just about all of the external functions of the TAPE package would have
"tape" in their names, whereas with scheme 2 none of them would.

Of course, the presence of the IMPORT function is another way to solve
the problems that scheme 2 addresses, since you can always explicitly
IMPORT only exactly those symbols that you want to use.

My present opinion isn't strong.  I lean towards feeling that having
both schemes is too hairy; we should go with one or the other and live
with the minor problems, rather than having a facility that is capable
of giving you a tradeoff between one set of problems and a different set
of problems but which is harder to figure out how to use.  But I don't
feel convinced either way.  And it will be a pain if we have to change
all the names of the external functions in our existing TAPE package.

  --DLW]

[Everyone who writes a program isn't going to put it in its own package,
we have to face that.  So, we still need to distinguish what symbols
are entry points from those programs.  This is the problem DCP keeps having
with LMFS.  #: could be only in SCL only, but then you couldn't write
a CL program not in its own package that used some yellow pages program.--MMcM]

[I think what this means is that scheme 2 is preferable, because it doesn't
interfere with the operation of other programs in the same package, as
scheme 1 does.  It seems to me that the only viable position is to support
both schemes, since some people will want one, some will want the other,
and some will want both depending on circumstances.  I don't think
having both schemes available makes anything much worse. --Moon]

SCL will of course continue to support package prefixes in front of
structured object p.r.'s, such as lists, as well as in front of symbols.
Just an abbreviation to avoid repeating a prefix many times.

[Yes.  This is an upward-compatible extension and shouldn't cause any
problems.  --DLW]

If IMPORT finds a distinct symbol already present in the ancestors
of the current package, it is presumably safe to assume that the user
knows what he's doing and wants to shadow that symbol.  If IMPORT finds
a distinct symbol already present in the current package itself (either
internal or external), we have a violation of the read-read consistency rule,
because for that symbol to have gotten there someone must have read it.
There may be programs or data already loaded containing references to
that symbol, so IMPORT can't safely just replace it.  A correctable
error must be signalled.  Upon continuing, IMPORT should do the hacks
that Zetalisp GLOBALIZE does (it should do some checking before signalling
the error, and list possible conflicts in the error message).  This
consists of checking for multiple distinct values, function definitions,
and properties (check separately for each indicator) and complaining if
any are found.  If the symbol that is going to be thrown away has a value,
function definition, or property that the symbol that is being imported
does not have, it is copied onto that symbol, and vice versa.  On the Lisp
machine one can also forward the value, function, and property cells so
that future changes to either symbol will propagate to the other one.
This makes some screw cases non-fatal, e.g. where you were only using the symbol as
the name of a function, and you got an undefined-function error because
you forgot to import the symbol.  Of course nothing can save you when you
were depending on the EQ identity of symbols, other than fixing your
package setup and loading the program (i.e. re-interning every name)
over again.

[The hacks that Moon describes (that GLOBALIZE does) are very ad hoc and
obviously should not be part of the language.  Having them in an
error-proceeding handler is probably OK, although I greatly doubt that
we want to make this part of the language definition.  As far as Common
Lisp goes, I think that to IMPORT a symbol when there is already a
symbol of that name present should just be defined to signal an error.
--DLW]

EXPORT of a new symbol that wasn't exported before, into an externals
that is somebody's ancestor, should check for name conflicts in a
similar way.  This is difficult because it doesn't know whether the
descendant package shadowed the exported symbol accidentally or on
purpose; maybe SHADOW should remember what it did.  EXPORTing into an
externals that hasn't been used as an ancestor yet, the usual case, is
no problem.

[Before we get into this level of hair, I'm still not sure what the
definition of EXPORT is.  My impression of the proposal was that every
symbol is either uninterned or interned, and every interned symbol has
exactly one home package, and every interned symbol is either an
internal symbol of its home package or an external symbol of its home
package.  My impression was that EXPORT never changes the home package
of a symbol; it merely "sets a bit" saying that this symbol is an
external symbol of its own home package.  But the @Defun says that the
symbol becomes an external symbol in the current package.  Is this just
a mistake, or does EXPORT go around changing the home packages of its
arguments? --DLW]

[I can't tell from reading Fahlman's manuscript whether EXPORT requires
that the symbols it is given already be in the current package, either
internal or external, and just "sets their bit", or whether it is as if
every package has attached to it a list of symbols, and EXPORT merely
does a PUSHNEW onto that list.  Of course it's really stored in some
more efficient way than a list.  If it's the former, what happens when
try to export a symbol that you inherited from an ancestor?  Is this
a bug in the former scheme, or a useful detection of an error? --Moon]

IMPORT does not do an UNEXPORT if somehow given an external of the
current package.  (Its @defun would suggest this.)

[Probably the definition should be changed explicitly to say that IMPORT
of a symbol that is already in the current package is a no-op.  --DLW]

I'm not sure about changing the ancestors list dynamically.  This is a
dangerous operation, but presumably it's only done "consciously."

[I think our new collective opinion is that it's OK to have dangerous
operations, and you take the consequences.  What Moon means by
"dangerous" is that it can have side-effects that are awfully hard to
figure out. --DLW]

Re: SI package.  What you actually want is the SYS package, whose
externals are the low-level and system-dependent interfaces that don't
belong in the LISP package's externals.  Normal packages, in particular
USER, take LISP as an ancestor but don't take SYS as an ancestor.  The
old SI disappears into the internals of SYS.

[Is there really any point in mentioning an SI package or a SYS package
in the Common Lisp definition, if the definition doesn't ever define any
symbols that live in those packages?  Probably neither should be
mentioned at all. --DLW]

[I think it's a win for the name of this to be standardized.  Admittedly
that is somewhat bogus, since some implementations will decide they need
more than one package of this kind and there is only one standard name.
The CL manual shouldn't really say more about this than "SYS is the standard
name for system-dependent things that don't go in LISP".--Moon]

I think there are many cases in real life where a single package, with
one internal set of symbols, wants to have several external sets of
symbols that are distinct interfaces for different users.  The external
sets of symbols are not necessarily disjoint of course.  To the outside
world this looks like multiple packages, but the important difference is
that there is only set of internals exporting to all of those packages.
LISP and SYS are an example of this.  It would save some juggling around if
EXPORT took an optional second argument which was the package to export
into.  More about this below ("search lists").

[This is one of the things we've talked about for a long time, but I
think it would be a big change to the proposal.  I can't see how to
incrementally modify the proposal to support this feature.  --DLW]
[See below--Moon]

MAKE-PACKAGE must take keyword arguments, because it will certainly be
extended in the future.  The existing optional argument should be a
keyword argument.

There is a problem with your scheme with calling package setup functions
such as IN-PACKAGE, IMPORT, and SHADOW at the front of a file that
contains code to load into that package.  At least in our system
compiled binary files work by interning each symbol used in the file
only once and then referencing it via a symbol table associated with the
file.  Any system that did this by first interning all the symbols used
in the file, before evaluating any of the top-level-forms, would lose,
because the symbols would get interned in the wrong package when the file
was loaded.  Our system interns symbols on first reference, so as long as
they aren't accidentally used before the package setup forms you are all
right.  But note that the compiler can put symbols in the "header" at
the front of the binary file that don't appear explicitly in the source
file, e.g. because it records the place and date of compilation.  I
haven't figured out yet whether this is actually a problem in practice.
Possibly the compiler needs to detect use of these package setup functions
at top level and do something special with them.  We have always preferred
to put package setup things in a separate file from the main code.
Of course, putting these package setup functions someplace in the middle
of a file, rather than at the front, is virtually guaranteed to shaft you.

[I'm not sure I understand all that you are saying here.  I think that
the issue is just like having (setq ibase ...) in a file, or like
defining a reader-macro in a file and then using it.  I presume that
Common Lisp intends to allow the latter to work, and so LOAD is required
to evaluate each form before it reads the next form.  The compiler,
likewise, has to process each for before it reads the next one, and the
default eval-when-itude of in-package probably has to explicitly be
compile-load-eval.  I don't see any problem with the compiler's putting
symbols in the header of a binary file; presumably the binary file
format remembers what package these symbols were interned in.  --DLW]

[No, the issue isn't what happens at compile time (read time).  Indeed
at compile time setting the package is just like setting the base.  The
issue is what happens at load time; the package is different from the ibase
and the readtable, because it has an effect at load time.  The goal is
to get the same symbols at load time as were gotten at compile time,
without prohibiting the loader from encaching the results of INTERN
in order to make loading faster.  This is, I believe, only a real problem
if the package-modifying functions are used in what I would consider
an "unreasonable" way.--Moon]

SHADOW is the only function that takes strings as proxies for symbols
(assuming I have disambiguated the typos in its description in the way
you intended).  Note that the strings must be in upper-case.  It is
actually perfectly all right for SHADOW to take symbols as arguments,
since any "accidental" interning this causes will be the interning that
SHADOW would do anyway.  Except there is the above-mentioned compiled-files
issue, where the "accidental interning" in the loader's symbol table
would not be all right.  Again this might best be handled by saying that
these functions should be used in files to be compiled only at top-level
and only at the front of the file, where the compiler can detect them
and handle them appropriately.

ANCESTORS-LIST => PACKAGE-ANCESTORS, this is the standard naming
convention.  Is this first level only or is it the transitive closure of
ancestor?

There is a general issue here.  I'm not at all sure that I believe in
this depth-first search of the ancestors of the ancestors.  Suppose I
write a package called CHAOSNET, whose external symbols are the entry
points I want people to call (CONNECT, DISCONNECT, and network things
like that).  Now to implement my chaosnet routines I need to call some
hardware driver primitives, and I don't feel like writing package
prefixes so I make the HARDWARE package an ancestor.  When people use
the CHAOSNET package as an ancestor, they shouldn't get those hardware
things, which they are not supposed to use, thrown into their namespace.

    [Yes!  This is the thing that most bothered me when I read the proposal.
    --DLW]

On the other hand, we want to write yellow pages packages that add
new general-use features.  The way you had in mind to do this seems to
have been to put some symbols in the externals of the YELLOW package,
then add YELLOW to the ancestors of LISP, so that everyone who inherits
from LISP will inherit from YELLOW also.  This is surely better than
YELLOW randomly going and interning its symbols into LISP so that
people will get them.  (let ((*package* (find-package "LISP")))
			 (import '(yellow:dog yellow:rain))
			 (export '(yellow:dog yellow:rain)))
This is a translation into Common Lisp terms of the way we do it
now in the Lisp machine, which everyone agrees is a crock of soil.

[Oh, is THAT why there's the transitive inheritance of ancestors?  This
has the same name-conflict problems.  If I want to use program A, and it
loads a subroutine package P that adds its external symbols to the
ancestor list of LISP, and then I want to use program B, and program B
has an external or internal symbol with the same name as any external
symbol of P, I'm in trouble. --DLW]

[This paragraph is rephrased immediately below.  But the original
is of some interest, too.]
Maybe what this shows is that the LISP package is really qualitatively
different from other packages.  Or in other words, perhaps there
are two kinds of object in the world: packages and search-lists.
They both have names and you can inherit from either.  Inheriting
from a package gets its own externals only; the ancestors list of
a package is purely intended to affect that package's internals,
not its externals.  Inheriting from a search list gets the externals
of all the packages listed in the search list, and if the search
list is altered, what you inherit is altered.  LISP is a search
list of all the packages that go to make up the complete Lisp system.
You can blithely inherit them all, or you can go ask what they
are and pick and choose which ones you want to inherit, listing
them all by name.  Adding a new package to a search list should
probably warn you if that would introduce two symbols with the
same name in that search list.  I.e. it's a bad idea for the
order to matter, within one search list.
In our discussion of packages two years ago, what I'm calling
search lists here were called "conglomerates."

[I would like to see packages and search lists distinguished.  In addition
to INHERIT-PACKAGE, EXPORT could take a search list, providing for more
than one set of external symbols from the name package.--MMcM]

Proposal to modify the proposal:

The "package system" defines two kinds of named objects, called
"packages" and "search lists".  The names are strings, they are globally
unique, and multiple names (nicknames) are allowed on an object.
Each package has two search lists associated with it; a package's
"external search list" has the same name as the package.  A package's
"internal search list" has no name.  There are other search lists
in the world as well.

A package is an object that controls how READ and PRINT translate
between printed representations of symbols and the actual symbol
objects.  A search list is a set of symbols that somehow belong
together; the set is given a name so that it can be referred to.  In the
qualified name FOO:BAR, FOO is the name of a search list and BAR must be
the name of some symbol in that search list.  In the qualified name
FOO#:BAR, FOO is the name of a package and BAR is translated into a
symbol according to that package.

Part of the difference between packages and search lists is that the
contents of a search list are established very carefully and consciously.
Often every symbol in a search list will be part of the interface to
some module, and therefore will be mentioned in a piece of documentation.
Conversely, the contents of a package are established haphazardly,
according to whatever symbols happened to be contained in a program
source file or typed in at top level, including spelling mistakes,
typographical errors, etc.

In addition to its internal and external search lists, a package has an
internal symbol table and a shadow list.  No two symbols in an internal
symbol table have the same name.  All symbols in a package's external
search list (directly, not via ancestors) and in its shadow list also
appear in its internal symbol table.  Here is how a print-name is
translated into a symbol, with respect to a certain package:
  1. If a symbol with that print-name is in the internal symbol table, use it.
  2. If a symbol with that print-name is in the internal search list, use it.
  3. Create a new symbol with that print-name, put it in the internal
     symbol table, and use it.

Every interned symbol is in at least one internal symbol table.  One of
those internal symbol tables, usually the only one, is considered its
"home internal symbol table" and the corresponding package is its "home
package".  The SYMBOL-PACKAGE function returns the home package of an
interned symbol, or NIL for an uninterned symbol.  A symbol does not
directly remember what search lists it is in.

To print a symbol with respect to a package:
  If the value of *PRINPACKAGE* is non-NIL, go directly to step 3.
  1. If the symbol is in the package's internal symbol table, just print it.
  2. If the symbol is in the package's internal search list, and does not
     have the same name as a symbol in the package's shadow list, just print it.
  3. Find the symbol's home package and print its name.  If the symbol is in
     that package's external search list, print a colon, otherwise print
     a sharpsign and a colon.  For uninterned and keyword symbols, print
     the appropriate prefix instead.  Now print the symbol's print name.

Note: An "obarray", a table which the system uses internally to map
from character strings to symbols, is not a package and not a search
list, and in fact is not visible to the user at all and not discussed in
the Common Lisp manual.  Every implementation will organize its obarrays
in a different fashion, according to its own needs and hardware tradeoffs.

A search list is defined by a list of symbols and a list of search
lists, called its ancestors.  The symbols in a search list are the
listed symbols, plus all the symbols in the ancestors.  This includes
symbols in the ancestors of the ancestors, recursively to all levels.
It is a rule that a given search list never contains two distinct symbols
with the same print name; it is permissible for the same symbol to
be in a search list for two different reasons (e.g. because it is in
two ancestors, or because an ancestor is seen twice).

Note that a search list does -not- contain a -copy- of its ancestors.
If a symbol is added to or removed from an ancestor, the contents of
the search list changes accordingly.  This "late binding" is desirable
so that the contents of a search list are not dependent on the order
of operations in a non-obvious way.  This "late binding" is viable
because symbols are only added to search lists consciously, not
accidentally as a side-effect of reading in an expression.

Note that search lists are dual-purpose objects.  A module uses a search
list, e.g. its package's external search list, as a way to advertise its
entry points for other modules to use.  A module also uses search lists,
e.g. its package's internal search list, as a way to locate the advertised
entry points of other modules.  This dual nature of search lists is a
feature, not a bug, although the reasons may not be immediately apparent.

MAKE-SEARCH-LIST name &key nicknames symbols ancestors if-exists
Make a search list.  nicknames, symbols, and ancestors default to NIL.
if-exists controls what happens if a search list with this name
already exists, and must be one of the following:
	:ERROR (the default) - signal an error.
	NIL - don't make a new search list, return the old one.
	:SUPERSEDE - make a new search list which replaces all
		     uses of the old one as an ancestor.
	:RENAME - make a new search list.  Rename the old one to
		a generated name.  Uses of the old one as an
		ancestor continue to use it.

MAKE-PACKAGE name &key nicknames ancestors if-exists
Make a package and its two associated search lists.  The external
search list has the same name and nicknames as the package, no
symbols, and no ancestors.  The internal search list has no name,
no symbols, and the specified ancestors, which defaults to (LISP),
i.e. a single ancestor which is the search list containing all the
symbols that define the Lisp language.
if-exists is as for MAKE-SEARCH-LIST; :SUPERSEDE and :RENAME refer
to the package's external search list.

INTERN name &optional (package *package*)
Do what READ does.

UNINTERN symbol &optional (package *package*)
Remove a symbol from the internal symbol table, and also from the external
search list and from the shadow list, if present in them.

EXPORT symbol(s) &optional (search-list (package-name *package*))
Insert symbols into a search list.  Error if this would cause a name
conflict, i.e. there would be two distinct symbols with the same name in
the specified search list, in some search list of which it is an
ancestor, or in a package's internal symbol table and internal search
list, provided that the symbol in the package's internal symbol table is
not also in its shadow list.

Some programs have more than one external search list, because they
present more than one interface to the outside world.  MAKE-SEARCH-LIST
is used to create the additional search lists, and EXPORT with two
arguments is used to place symbols into them.

UNEXPORT symbol(s) &optional (search-list (find-search-list (package-name *package*)))
Remove symbols from a search list, without otherwise affecting them.

IMPORT symbol(s) &optional (package *package*)
Add symbols to the package's internal symbol table.  Error if this
would cause a name conflict (see EXPORT).

SHADOW symbol(s) &optional (package *package*)
Ensure that symbols with the specified names are in the package's
internal symbol table, and on its shadow list.
UNINTERN serves to undo SHADOW.

IMPORT-AND-SHADOW symbol(s) &optional (package *package*)
Same as import, but only gets a name conflict error if there is already
a different symbol with the same name in the internal symbol table, not
if there is one in the internal search list.  Adds the imported symbols
to the shadow list so that we don't later think there is a name conflict
with the internal search list.  IMPORT-AND-SHADOW cannot be done by
combining separate IMPORT and SHADOW operations in the case where there
is already a different symbol with the same name in the internal search
list, e.g. importing a symbol that shadows a basic Lisp function in order
to install an incompatible or improved version.

FIND-PACKAGE name
Get the package with the specified name, or NIL.

FIND-SEARCH-LIST name
Get the search list with the specified name, or NIL.

PACKAGE-INTERNAL-SEARCH-LIST package
Get the internal search list object, which has no name, for the package.

SEARCH-LIST-ANCESTORS search-list
A list of the (immediate) ancestors.

SEARCH-LIST-ALL-ANCESTORS search-list
A list of all the (recursive) ancestors.

ADD-SEARCH-LIST-ANCESTORS search-list &rest ancestors
Add ancestors to a search list, checking for name conflicts (see EXPORT).
The order of ancestors to a search list never matters, since there are
no duplicate symbols.

REMOVE-SEARCH-LIST-ANCESTORS search-list &rest ancestors
Remove ancestors from a search list.

[Should the above two functions be deleted in favor of SETF of
SEARCH-LIST-ANCESTORS?  It is still necessary to check for name conflicts
when adding ancestors.  But this would allow the use of sequence
tools, e.g. PUSHNEW, on the ancestors list.]

*PRINPACKAGE*			variable, default NIL
If T, always print package prefixes on symbols regardless of the value
of *package*.  This is typically used by package-related tools that
want to be very explicit about what package a symbol is in.

DO-SEARCH-LIST-SYMBOLS (var search-list-or-name result-form) body...
Iterate over all the symbols in a search list.

DO-PACKAGE-SYMBOLS (var package-or-name result-form) body...
Iterate over all the symbols in a package's internal symbol table and
internal search list.

DO-PACKAGE-INTERNAL-SYMBOLS (var package-or-name result-form) body...
Iterate over all the symbols in a package's internal symbol table.

DO-ALL-SYMBOLS (var result-form) body...
Iterate over all the symbols in all packages' internal symbol tables.

[Most of the above functions accept names of packages and search lists
in place of the objects themselves.  This needs to be made explicit
in the manual, I left it out for brevity.]

Note: the use of the word "list" in the above discussion is not to be
taken to imply that the implementation uses lists in preference to some
other table organization.

End of proposal to modify the proposal.


Why is LIST-PACKAGES a function rather than a (read-only) variable?

Is it intentional that INTERN only accepts a string, not a symbol?
In the laser edition it accepts either, as in Maclisp and Zetalisp.

[Note that the existing SHADOW is what Zetalisp calls INTERN-LOCAL, and
it accepts strings (but not symbols!). --DLW]

This chapter of the manual -must- include a complete example of how to set
up a toy yellow-pages package.  Include a discussion of the issues with
mutually-recursive (in the externals) packages, such as in the example
on page 292 of the laser edition.  Note the dependency on package setup
(including declaration of all externals) happening first, before any code
is loaded, and before any files that create other packages that use this
one as an external are loaded.

Include WHERE-IS.  See the Lisp Machine manual, or try it on one of
your machine(s!).  This is handy for debugging package problems.
For each symbol it finds with that name, it tells you where it found it,
its home package if that is different, and all the packages that inherit
it from that place.  In this new package scheme it would tell you also
whether it was internal or external in the place that it found it.
The same symbol can be listed more than once if it's been imported.

In addition to WHERE-IS, a user command that prints out and so forth,
there should be a primitive, FIND-ALL-SYMBOLS, which takes a print name
string and returns a list of all the symbols, interned in any package,
with that print name.  The system can probably do this much more
efficiently than the user could with the primitives available in your
proposal.

In EXPORT and related functions, is NIL as an argument NIL or ()?  It doesn't
really matter which, but you must specify.

Note that there are ways to do the "searching" without multiple hash-table
lookups by INTERN.  These aren't necessarily desirable, depending on the
particular system's space and time tradeoffs.  But it's probably worth noting
in the manual that the implementation need not correspond directly to the
way it's described; otherwise people might imagine that it's very inefficient.

[In the file, where it says "@Defvar[Var {package}]", I think you really
mean "@Defvar[Var {*package*}]".  Also, at this point in the file, it
says that it's OK for the value to be the name of a package, whereas
earlier it says explicitly that it must be a package object.  I think
that it should be required to be a package object, and assume that's
what you meant.  --DLW]
-------

Summary of points made:

1. Include the consistency rules in the documentation.

2. Clarify all ambiguities and loosenesses in the documentation; this
stuff is difficult enough without the documentation adding to the
problems.  I'd be happy to help with this.  Include a complete example
of the use of packages.

3. Give MAKE-PACKAGE keyword arguments.  Actually you can't go wrong with
a general rule that anything that is named MAKE-xxx should take keyword
arguments, along with some number of required arguments.

4. Put in synonyms (:NICKNAMES keyword in MAKE-PACKAGE).

5. Have four kinds of qualified name (two new kinds):
	:foo		keyword
	#:foo		uninterned symbol
	foo:bar		external-symbol reference (error if not exported)
	foo#:bar	internal-symbol reference (just like setting *package*)

6. Provide a function (or arguments to an existing function) to allow a
program to do what each piece of syntax does.

7. Signal an error when there is a possibility of name conflict in
IMPORT, EXPORT, or INHERIT-PACKAGE (and the proposed new functions).
Continuing from the error can attempt to fix things up.  Provide a
WHERE-IS function to allow users to figure out what's going on, and a
FIND-ALL-SYMBOLS function which is the underlying primitive on which
WHERE-IS is built.

8. Separate the notions of package and search-list in order to clarify the
issues of internal vs. external symbols.