packages, lexical variables, and compiling

[David C. Plummer <DCP@SCRC-QUABBIN.ARPA>]

    Date: Thu, 3 Apr 86 00:01:56 est
    From: Steven Haflich <smh@mit-ems.ARPA>

[Bottom line: I don't think this has anything to do with the language.
Instead, I think we are slopping over into program environment issues
(again).]

    Consider the file foo.lisp:

	    (IN-PACKAGE "MYPACKAGE")
	    (DEFUN FOO (X)
	      (LET ((X-PLUS-1 (1+ X)))
		(LIST X X-PLUS-1)))

    Loading foo.lisp into any common lisp has the side effect of creating
    package MYPACKAGE if it does not yet exist, and also creating symbols
    MYPACKAGE::FOO, MYPACKAGE::X, and MYPACKAGE::X-PLUS-1 if any of these
    do not exist.  So much is clear.

    If foo.lisp is compiled to produce foo.bin, and foo.bin is later loaded
    into a different lisp world, clearly package MYPACKAGE and symbol
    MYPACKAGE:FOO must be created if they do not yet exist.  But what about
    symbols MYPACKAGE::X and MYPACKAGE::X-PLUS-1 ?

The symbolics implementation will create the symbols MYPACKAGE::X and
MYPACKAGE::X-PLUS-1.  They are stored in something called the
debugging-info info.  The function ARGLIST (not part of CL, one of our
extensions) will pluck the arglist stored in the debugging-info.  In
addition, it is possible to declare what the values are; any symbols
there will be in the resulting environment and ARGLIST will also return
them as the second value.  Furthermore, X-PLUS-1 will be stored in a
part of the debugging-info that the debugger can find.  Thus if there
was an error, the debugger would know that X-PLUS-1 not only IS a
variable, but where it is (on the stack, in a lexical environment,
whatever) and actually >>build an environment that has the current
lexical state of the function in it<< and use that environment for the
read-eval-print loop.  What does this have to do with CLtL??  Absolutely
NOTHING!

    The existence of symbols naming these lexical variables is not required
    for execution of the function.  (A clever compiler could even optimize
    X-PLUS-1 away.)  Still, it is easy for a program to tell whether these
    symbols exist, and it is possible to construct package interlocutions
    which will fail depending on whether or not the symbols are created.

    There are three possibilities:
     1 - CLtL requires that the symbols are created (although some
	 implementations fail to do so), preserving interpreter-compiler
	 consistency.
This is not needed for simple systems that do not have a lexical
debugger or something like the ARGLIST function.  It may be a bit tricky
for those systems that do renames of local variables and functions (as
discussed in the scoping problems of inlinifying FLET and LABELS).
     2 - CLtL requires that the symbols not be created, which saves storage
	 but loses interpreter-compiler consistency.
All implementations that include a somewhat powerful programming
environment will therefore be incompatible with CLtL.
     3 - CLtL explicitly leaves it up to the implementation.
This gets my vote.

    Unless I have missed something, CLtL is silent on the matter.  This is
    *not* quite the same as leaving it up to the implementation.  Should
    the language definition be more specific?

Maybe yes, maybe no.  I vote no for the reasons stated above.  I think
the real question is this: What programs will work incorrectly if they
depend on package non-polution?  Isn't that what you are really worried
about?  X and X-PLUS-1 will not have a global value nor a global
definition, nor a property list.  It is as if I typed 'X and 'X-PLUS-1
at the interpreter.  Typing at READ (probably of the read-eval-print
loop) will either use existing symbols, create new ones if you are
writing new code, or >>create new ones if you happen to make a typo<<.
Are you careful to UNINTERN your typos? 

    Date: 3 Apr 86 09:03:00 EST
    From: "BACH::GREEK" <greek%bach.decnet@hudson.dec.com>

    A couple of comments on Mr. Haflich's little program.

    First of all, I don't think it's absolutely clear that the compiler is
    obliged to create MYPACKAGE if it can somehow avoid it.  After all, it's 
    really nice if the compiler leaves the global environment alone.
    Implementation of this nicety is another matter, however.
I don't think this is what Steven Haflich meant, or at least said.  He
said the compiler had to cause MYPACKAGE to be created >>in the lisp
environment the compiled file is later loaded into<<.

    Secondly, I don't think it's so easy for the interpreter to detect the 
    presence of the lexical symbols.  At least not in a correct program.  
    We've carefully constrained all the symbol-hacking functions to work 
    only on the dynamic (global) aspects of symbols.  Therefore the compiler 
    need not create real symbols for lexical variables.
I agree with this except for when you "enter the debugger," whatever
that means in various systems.  In the Symbolics implementation, it
means quite a bit as I stated above.  The program is correct, and the
read-eval-print loop (what I think you mean by the interpreter) DOES get
the lexical symbols into its environment.  Again, the debugger is not
part of CLtL, and therefore CLtL says nothing and can say nothing about
its 'correctness'.