Compilation and package system

[David A. Moon <Moon@SCRC-STONY-BROOK.ARPA>]

    Date: Tue, 9 Apr 1985  22:14 EST
    From: Skef Wholey <Wholey@CMU-CS-C.ARPA>

    There are a number of ways to let the compiler know about the package that a
    file will be loaded into.  The portable, Common Lisp way to do this is with a
    call to In-Package at the beginning of a file, as suggested in CLTL, on page
    189.  The Lisp Machine -*- file comment hack (which is used by the editor,
    compiler, and loader) has a few drawbacks.

    First of all, it is not portable.  People write Common Lisp programs with a -*-
    package specification, but these programs must be modified (by adding a real
    In-Package to the program text) when they are moved to non-Lisp Machine Common
    Lisps.

I guess it's a matter of viewpoint which way is non-portable, depending on which
way you're trying to port.  But let's not get into that.

    Secondly, one cannot put all of the information about the packages Use'd or
    symbols Import'ed in the file comment.  One cannot accurately fix up the
    package environment by looking at the file comment.  I suppose one could hair
    it up to include imports and shadows and whatnot (maybe someone already has),
    but that just leaves one with more stuff to "translate into Common Lisp" when
    he moves his program off of the Lisp Machine.

We believe that complex package setups should go in their own files.

    Finally, the Symbolics Common Lisp Compatibility Package will give an error
    when one attempts to load or compile a file with a header something like that
    presented below, because it interprets the file comment, and wants to find any
    package specified there before reading anything in the file.  In fact, the
    first form read may be the one that creates the package, so it may not be there
    for the file comment gremlin to interpret.  In-Package is a nice language
    feature that Does The Right Thing, but has been rendered useless on the Lisp
    Machine by zealous use of file comment magic.

I don't think it would be constructive for me to respond to the above paragraph
in the same tone as it was written, so I will refrain.

    We have adopted the convention that any -*- package specification is for the
    editor only, and that the compiler and friends should get stuff out of Lisp
    forms.  Thus, a typical Spice Lisp system file might begin with something like:

	    ;;; -*- Package: Lisp; Log: Code.Log -*-
	    ;;;
	    ;;; ...blah blah blah...
	    ;;;

	    (in-package "LISP" :use '("SYSTEM"))

	    (export '(things that this file defines))

    The editor knows which package to associate with a buffer created when visiting
    this file, but the compiler and loader and everyone else just read Lisp forms,
    the way Father Lisp wanted them to.

This all sounds at first like it makes a lot of sense, but wait a minute.  I
fail to see how the editor needs any less information about packages than the
compiler and the loader.  How does the editor know the names of the functions
in the file if it doesn't know what package they are relative to?  How can the
editor understand forms in the file, telling you about arguments wanted by
functions, documentation of variables, syntax of macros, and so forth, if it
can't intern symbols whose printed representations it finds there?  How can you
incrementally compile or load portions of the file being edited if the machine
doesn't know what package to read the forms in?  Or does you editor make a
special pass through the file when it reads it in, looking for "(IN-PACKAGE "?

In our system all programs that read Lisp forms from files get the package
information from the same place and they all work consistently.

    Since the -*- convention is not part of Common Lisp (and perhaps for good
    reason), I would encourage EVERYONE writing Common Lisp programs to begin each
    file in the way suggested in CLTL.  Smart editors need to know things like the
    package (so that functions compiled in the text editor end up in the right
    package, for example), however, so it might be worth some effort to standardize
    the semantics of -*- file comments.  Unfortunately, I imagine neither the Lisp
    Machine community nor the Spice Lisp community is willing to give in.

Probably not, considering that IN-PACKAGE was put into Common Lisp over my dead body.
This is my fault for not doing an adequate job of explaining why it was a bad idea
and only workable in the short run.  As you say, it might be worth some effort to
standardize the semantics of -*-.