package changes

[DILL@CMU-CS-C]

I'll give one more push for major changes before I decide that I'm
obstructing progress.

It is not clear to me how an implementation can reasonably assure that
"no two symbols accessible to a package have the same name" with
"reference style" USE-PACKAGE.  Consider the following simple case of
accidental shadowing:

Pn is a package; Sm is a symbol.

  1. P3 exports S1.
  2. P1 uses P3
  3. P1 uses P2.
  4. P2 exports	S1' (a non-EQ symbol with the same pname as S1).

Read-Read consistency is violated, since the result of interning "S1"
before step 4 will not be EQ to interning it after step 4.

Similarly, Print-Read and Print-Print consistency are violated: try
printing S1 immediately before step 4, Read the result or print it again
after step 4.

This problem can occur no matter what the searching order of the various
packages is.

It can be very expensive to check for. With INCLUDE-PACKAGE, you have to
find all packages using P2, and check that there is either a symbol with
pname "S1" appearing in the use list ahead of P2 or that there is no
conflicting symbol appearing after P2 in the use list to be assured that
this has not happened.  With INCLUDE-PACKAGE, it is much more
complicated, since step 4 may cause S1' to be exported to many different
packages.

A much worse problem is that after detecting this situation, the user
will then have to deal with potentially a lot of questions demanding
that he resolve conflicts between many symbols in many packages, perhaps
for packages he knows nothing about.

----------------------------------------------------------------

I propose modifying Fahlman's proposal as below.

1. Flush INCLUDE-PACKAGE and UNINCLUDE-PACKAGE.

  The only reason I can see to have these functions is to build big
packages out of little ones.  Here is a more tasteful solution
that requires no additional functions:

a) Define a package to represent the whole system, and export from it
all the symbols that users will want to import from it.  

b) For the little packages composing the system, build them in the
natural way, except IMPORT all the symbols from the package of part a.
These packages will define the various functions, properties, and values
associated with these symbols.  Each of these "internal" packages can
hide its own internal symbols, and communicate with other internal
packages by exporting symbols and importing them in the ordinary way.
The user won't see any of these symbols unless he imports them
intentionally from one of the internal packages.  So, the user can
see it as a single large system by just importing the package defined
in part a.

With the addition of a structured package name space of some kind, the
names of the "internal" packages could also be isolated in order to
avoid conflicts with user-level package names.

Another way to group a bunch of packages would be to build another
package which imports symbols from the grouped package, and then
selectively exports them for importation by users.


2.  Flush DECLARE-PACKAGE.  Require that ALL symbols exported by a
package be exported before being imported (in any way) into other
packages (modulo debugging).


3.  Define (USE-PACKAGE P) to IMPORT the symbols in package P's export
list at the time that USE-PACKAGE is called ("copy" style).  Mistakes in
export lists will have to be remedied by tracking down the packages that
have imported them.  This may seem like a pain, but is probably 
inconsequential compared with the pain of tracking down wrong symbols
that were read by a package importing from a bogus export list.

The disadvantage of this definition that can be inferred from comments
on proposals is that it won't be possible for packages to export new
symbols from LISP and have them automatically appear in other packages.
This is not a serious problem (I believe).  No package author in his
right mind would use the new LISP features without taking steps to
ensure that they have been loaded -- and if he does this, he can equally
well do a (USE-PACKAGE LISP), after doing so.  The only potential
exception is the USER package; for this, packages exporting from LISP
can also explicitly IMPORT the symbols into USER.

The combined effect of these changes greatly simplifies the explanation
of how symbols are looked up, which is the closely related to the
conceptual simplicity of the system.  It is also safer.

----------------------------------------------------------------

I also propose the following function:

@defun[Fun {load-package} Args {@i[package-name]} @Optional {@i[file-name]}]

If the package @i[package-name] exists, do nothing.  Otherwise, load the
file named by @i[file-name].  If this argument is not supplied, load the
package from a file established by the system as the standard source for
that package, signalling an error if there is no file established for
@i[package-name].  If loading the file did not cause the package to
exist, signal an error.

USE-PACKAGE should call this function if the package it needs is not
defined.  Add an optional filename argument to USE-PACKAGE.

Note that a load can be forced by calling LOAD explicitly.

----------------------------------------------------------------

Here's how these changes would affect the example Fahlman gave in
his proposal [my comments in square brackets]:

[The init file would be simplified by not having to load the packages.
USE-PACKAGE and LOAD-PACKAGE would be sufficient.]

The file "alchemy.lisp":

@begin [lisp]
;;; Alchemy functions, written and maintained by Merlin, Inc.

;;; This file uses some functions from "Phogiston.lisp".
(declare-package "phlogiston"
  :export ("heat-flow" "cold-flow" "make-fire-bottle"))
[delete this previous s-expression]

;;; All of the following is in the ALCHEMY package.
(in-package "alchemy")

;;; Here is our external interface.
(export '(lead-to-gold gold-to-lead vms-to-unix pawn-to-king4
	  elixir-of-life))


;;; We use this a lot, so import it.  [insert: (load-package
phlogiston).  The phlogiston package will be able to use our symbols,
because we have already created the package and exported them.  It will
not try to load this package again because the package will already
exist.  Note that we have not used any symbols from phlogiston, yet, so
it's okay to load it now, if necessary.]

(import '(phlogiston:make-fire-bottle))

(defun lead-to-gold (x)
  "Takes a quantity of lead, returns gold."
  (when (> (phlogiston:heat-flow x) 3)	        ; Using a qualified symbol.
	(make-fire-bottle x))			; Using an imported symbol.
  (gild x))

;;; And so on ...
@end [lisp]

The file "phlogiston.lisp":

@begin [lisp]
;;; Phlogiston functions, written and maintained by Thermofluidics, Ltd.

;;; This file uses the ALCHEMY package from file "alchemy.lisp".
;;; The symbols listed are the only ones actually referenced.
(declare-package "alchemy"
  :export ("gold-to-lead" "elixir-of-life"))
[delete this]

;;; All of the following is in the PHLOGISTON package.
(in-package "phlogiston")

;;; Here is our external interface.
(export '(heat-flow cold-flow mix-fluids separate-fluids
	  burn make-fire-bottle))

;;; We use this a lot, so import it.
[same comments apply here as in the phlogiston.  If we load this one
first, it will not be loaded again, and it's symbols can be used by
phlogiston]
(use-package "alchemy")


(defun heat-flow (amount x y)
  "Make some amount of heat flow from x to y."
  (when feeling-weak
	(quaff (elixir-of-life)))		; No qualifier needed.
  (push-heat amount x y))

;;; And so on ...
@end [lisp]


[In Fahlman's example, he has an init file that loads both alchemy and
phlogiston -- a call to HEAT-FLOW will not work unless both are loaded.
In this proposal, the user can always (USE-PACKAGE "phlogiston") and call
HEAT-FLOW.  If it was pre-loaded in the init file, this will be faster,
but in either case it will happen exactly once.]

	-Dave
-------