Re: Standardization

[Jeff Dalton <jeff%aiva.edinburgh.ac.uk@cs.ucl.ac.uk>]

   From: Liz Allen <liz@edu.umd.brillig>
   
   I once tried to port some code that ran fine under DEC Common Lisp
   to LMI's Common Lisp and had *lots* of trouble -- in fact, I finally
   had to give up. ... Has anyone else tried porting common lisp code
   from one implementation to another?

(I imagine that the implementors of things like CommonLoops could also
comment on this subject.)

I have ported several programs of varying sizes (up to 150 pages or so)
from Spice Lisp to Kyoto Common Lisp (KCL) and then to Sun (Lucid) Common
Lisp.  Some of this code was originally written in Spice Lisp, but other
parts came from Franz Lisp or even T.  I have also written some new code
that was to run in both KCL and Sun Lisp; most of this has not been tried
in Spice.  Finally, I have ported some of things to Poplog Common Lisp.
(The KCL and Poplog cases are particularly enlightening in that they were
developed by people outside the central Common Lisp community and hence
had to depend more on the published description of the language.)  None of
my code depended on things outside the Silver Book, and except where blocked
by bugs I was always able to get something that worked everywhere in the end.
Nonetheless, it was a rare program that could be ported as-is.

In general, incompatibilities could be attributed to any of several reasons,
here listed roughly in order of decreasing frequency:

1. Incorrect interpretations of the language specification.

   These are cases that *I* think are wrong but where (a) the implementa-
   tion seems to be meant to work this way and (b) it is possible to see
   how someone might have understood the CLtL in this way.  Many of these
   involve packages.

2. Bugs in the implementations.

   The cases I think are wrong that don't meet criteria (a) and (b).  This
   includes faulty handling of incorrect programs.  (Incidentally, this
   category is becoming more common and may be number one any day now.
   Perhaps I initially used parts of the language that were more thoroughly
   debugged.)

3. Allowed differences.

   Implementations of Common Lisp are allowed to differ in certain ways
   apart from internal implementation details.  Examples would be potential
   numbers (p. 341) and symbols like "::" (p. 176).

   Some implementations enforce a stricter interpretation of the language
   than others in the sense that programs that work in the stricter system
   will work in less strict ones but not the other way around.  In this case,
   it helps to start in the stricter system because then you will be
   warned about things like potential numbers that you might otherwise
   have overlooked in CLtL.

   Extensions to Common Lisp can also be a problem.  For example, if a system
   defines a new external symbol in the Lisp package, and I have a program
   that also defines that symbol, I will have to do something to resolve the
   conflict.  (Would it be reasonable to require that extensions be defined
   in a different package?)

4. Implementation or environment differences.

   Some programs will have difficulties in some systems because of the
   machine or the operating system involved or because things like storage
   management are done differently.  Sometimes it is helpful (or necessary)
   to rewrite parts of the program.

   This differs from category (3) in that there isn't much that the language
   definition can do.  We could say what must be done with potential numbers,
   but we could hardly require that all systems must have the same
   performance.  Cases like the treatment of tail-recursion may fall
   somewhere in between.

5. Ambiguities, inconsistencies, or mistakes in the specification.

   This is similar to case (1) except that the specification is (more)
   clearly at fault.  Note that many of the inconsistencies are due to
   uneven revision, things missed when parts of the language were changed;
   others have origins that are less obvious.

6. Uneven development.

   Some implementations are more up-to-date than others in that they may
   have responded to changes or clarifications that were discussed on this
   mailing list.

Of these categories, (1), (3), and (5) have implications for standardization,
(1) and (5) because a standard needs a clear, precise, and consistent
specification, and (3) because it shows that some incompatibilities are
inherent in Common Lisp; after all, Common Lisp is meant to be a common
subset.  

It is also important to note that all of this can be improved.  There do
not seem to be any "fatal inconsistencies" that would bring down a large
part of the language.  To me, this means that a standard can be based on
Common Lisp, but also that improvements in the specification are needed.

(A final note: Clearly the number of cases in each category, and hence
their order, reflects the selection of dialects involved (how could it be
otherwise?), but many of the obvious conclusions about these dialects do
not hold.  I say this, without saying what those conclusions would be, to
avoid giving a false, negative impression of any of these systems.  I don't
feel this is the right place to make such comparisons.)

-- Jeff