Conservation of nits

[David A. Moon <Moon%SCRC-TENEX%MIT-MC@SU-DSN>]

    Date: Wed, 18 May 1983  00:08 EDT
    From: Scott E. Fahlman <Fahlman@CMU-CS-C>

	From: Moon

	I hope you realize that IN-PACKAGE is incompatible with incremental
	re-compilation of portions of files (after editing them), a Lisp machine
	productivity feature that users are uniformly enthusiastic about.  I
	would think that this would be something that Spice Lisp would definitely
	want, too, once you have a compiler that runs natively in your Lisp.
	I'm not sure yet whether we will strongly discourage people from using
	IN-PACKAGE, or whether we will provide a kludgey way for the editor to
	know that it must parse the beginning of the file looking for IN-PACKAGE
	in order to determine in what package to read forms from the file.

    I don't completely understand this.  I looked for incremental
    compilation in the Confederate edition of the Chine Nual and couldn't
    find anything about it.  What exactly do you folks do?  I would have
    assumed that if you do incremental re-compilation on an ASCII file, you
    take some notice of the -*-Package:foo-*- business, no?  IN-PACKAGE
    would have the same set of problems as that.  I would hate to have you
    tell users not to use IN-PACKAGE.  Do you have a specific proposal?

The Chine Nual doesn't document the editor.  Incremental compilation =
designate some portion of the buffer in the editor (in the simplest case
it's the top-level form surrounding the editor cursor) and compile it
putting the results into the current environment.  You can also just
read/eval/print it.  To do this the editor needs to know the base, readtable,
and package required to read the text (it also needs to know what language
it's in, but that's a different story).  The way we tell the editor this
information is to put it all in the file attribute line, which is the first
non-blank line in the file and contains the characters "-*-".  The important
point about this is not the particular representation of this information,
but the fact that it is the same for everything in the file.  When recompiling
something halfway down the file, you don't need to read everything from the
beginning of the file up to there to find out what package (etc.) is needed
to read the one form being compiled.

The only thing that is important about the particular representation of
the information is that you can find it, or determine that it is absent,
without reading through the whole file, and in fact there is a general
facility for dealing with these file attribute lines, used by the
editor, the compiler, the loader, and any other programs that care.
Doing it with Lisp forms that you evaluate, such as IN-PACKAGE, works
for the compiler and the loader, but for other programs (the editor is
an example) whose business is not to evaluate Lisp forms it doesn't fit
in so smoothly.

So one issue is allowing the package (etc.) to change in the middle of a
file.  This seems of little worth to me, so I doubt we will make any
effort to support incremental compilation of files that do this.  We
will of course support non-incremental compilation and loading of such
files if Common Lisp allows them.  The other issue is whether we want
(not in Common Lisp, but in our own Lisp machine user interface) to
introduce a new syntax for saying what package (etc.) a file is in, when
we already have one.  Probably we will just say that you put the
information in both places if you want your file to be incrementally
compilable and you want it to work in other Common Lisp implementations.

My concern is not that we (Symbolics) can't accept IN-PACKAGE, I was more
trying to point out that you (Spice) might be backing yourselves into a corner
without realizing it, by adopting a style of programming that is inimicable
to incremental compilation.

	The writeup doesn't say why certain functions should be used at top
	level.  One reason is that if changes are being made to *package* with
	the expectation that they will affect how forms later in the file are
	read, then those changes had better happen at compile time before
	further forms are read from the file.  The other reason is that in many
	implementations the output from the compiler is designed so that the
	loader can look symbols up once, rather having to call INTERN every time
	a symbol is referenced.  This means that the compiler must know when the
	package state is changing (i.e. the read-read consistency rule doesn't
	apply).  It would probably be best to include an Implementation Note
	pointing out the restrictions this places on the compiler/loader
	implementation: the compiler may not read the whole file before compiling
	any of it (as the Lisp machine used to do in order to improve virtual
	memory locality); the loader may not do all its INTERNs before evaling
	any of the top-level forms (as Multics Maclisp does).

	The writeup should say explicitly which functions get an automatic
	(eval-when (compile) ...) wrapped around them when they are seen at
	top level by the compiler.  The example at the end is assuming this
	implicitly.

    Well, both of the above comments assume that the compiler represents
    symbols and their packages internally just by reading them into the
    right package.  Certainly the compiler has to keep track of what
    packages the various symbols want to end up in, but this might be done
    by some more complex mechanism if the compiler wanted to keep its guts
    somewhat separate from the stuff it is reading.  I'll see if I can say
    something sufficient about this without over-specifying the mechanism.

I don't think macros can reasonably run at compile time if the compiler
doesn't represent symbols in the code to be compiled as symbols.  Your
last sentence above sounds like the right thing.

	I can find no justification in the previous discussion for the suddenly
	introduced restriction that EXPORT and UNEXPORT are not allowed in
	packages that have been USE-PACKAGE'd, unless this is a Solomonian
	attempt to resolve the copy vs reference issue.  If this is trying to
	deal with the case of exporting a symbol too late, after someone who
	thought they were getting it via USE-PACKAGE made an internal symbol
	instead, the naming conflict checks I suggested earlier detect this and
	are able to tell you exactly what happened.  If this is trying to deal
	with the case where exporting a new symbol, that people using
	USE-PACKAGE didn't expect to see exported, causes what was expected to
	be two distinct symbols, internal in different packages, to become one
	(external) symbol, it doesn't deal with it.  That can just as well
	happen if all the EXPORTs are done before all the USE-PACKAGEs, and is
	basically indetectable.  This screw case is really a lot like someone
	changing what a function does without telling everyone who uses it.

    I was just trying to do something simple and safe that might end the
    debate.	 I figured that I could get this "package locking" rule right on
    the first try, and that some tricky rule to signal an error only in the
    case of a conflict with some symbol in the inferior package that is not
    on that package's shadow list would lead to N more rounds of revision.

    You seem to understand this better than I do.  How about if you propose
    the exact wording for the error-checking rule you would like to see here
    (both for EXPORT and for UNEXPORT)?  That would be the quickest way to
    converge, I think.

Okay.  But not in this message, I'm too tired to write coherently.  Tomorrow.

    Just out of curiosity, why in the world would anyone compile an INIT
    file?

To make it load faster, or to make the dozens of functions defined in it
run faster.  You should see the init files of some of our people!