Problems with packages

[Kent M Pitman <KMP@SCRC-STONY-BROOK.ARPA>]

I ran into a problem the other day in Symbolics (extended) Common Lisp
where I was doing a DEFPACKAGE which specified some shadowed symbols and
lossage resulted due to a bug where the package creation wasn't done atomically.

For those not familiar with DEFPACKAGE, we can write:
 (DEFPACKAGE "FOO"
   (:USE "FRED")
   (:SIZE 10000.)
   (:IMPORT HACKS:FOO COLOR:PAINT MATH:SOLVE ...)
   (:SHADOW "JOE" "MARY" "SALLY")
   (:EXPORT "JOE" "PAINT"))

The problem was that I was trying to read into the editor in one process
a file on the `FOO' package while I was loading the system in another
process which defined the `FOO' package. For those not familiar with the Lisp
Machine architecture, both processes share the same global environment 
(including, therefore, package declarations). I was expecting that it was 
going to say that `FOO' wasn't defined and I was going to tell it to use
`USER' for my editing purposes now while the system loaded. Something 
much worse happened...

What happened was that I hit a bad timing window and it tried to resolve 
the `FOO' package at the instant the package was created but before the
special stuff (like shadowing, importing, etc.) had been done. The editor
managed to add some symbols to the `FOO' package before the process in the
other window managed to process the shadows, resulting in lossage when the
shadow information finally got processed and the system realized that the
symbols in question had already been interned without knowing about the
shadow information.

In our environment, we can treat this as a simple bug and just have DEFPACKAGE
run without interrupts or have it take an appropriate lock on the package
during its creation so that any other process trying to do an intern or
lookup blocks waiting for the operation to finish.

However, in normal Common Lisp, because of the absence of DEFPACKAGE, 
there would be no way to do this because you can't really tell when you're 
done creating a package. IN-PACKAGE can't leave the package locked because 
it might be that no calls to SHADOW and friends follow. Even if they did
follow, you can't tell when they're done following. What a mess. CL 
should have DEFPACKAGE.

While I'm on the subject of packages, I have a few other gripes to air...

I'm really bothered that I have to say symbols as arguments to shadow. The 3600
implementation allows me to use strings, which CL doesn't define. CL should
-require- strings as arguments to this function and should not allow symbols.
Giving symbols as arguments to this function forces things to get created wrong
and then hopes that things will get cleaned up correctly afterward. This reduces
the ability to error-check if the symbol to be shadowed had already been interned
for some reason other than the call to shadow. The same situation applies to
EXPORT, etc. -- except for IMPORT, for which it would seem perfectly reasonable 
to require symbols (and forbid strings).

Finally, it bothers me that there isn't a version of IN-PACKAGE that doesn't 
require the package to exist. Macsyma has about 200 source files and every one
of them starts with IN-PACKAGE. It also shadows about 20 symbols and I'm not
(for reasons of modularity) going to repeat the SHADOW declaration at the top 
of every one of those files. I define the package early on and everything works 
mostly ok, but if someone were to read the files out of order, the package would 
get created incorrectly because no shadow information would get seen. Yet no 
error would get signalled because IN-PACKAGE would still create the package. I 
really want the file that declares the package to contain a call to IN-PACKAGE
and all the other files to call some function IN-EXISTING-PACKAGE which 
errs if the package doesn't exist, but which otherwise does what IN-PACKAGE does.