Re: Common Lisp for SUNS

[ibuki!dbp@labrea.stanford.edu (David Posner)]

Some comments on John Foderaro's comments on "C-based" Lisps

This is in response John Foderaro's comments on "C based" Lisps on the
Sun 4.  (IBUKI Common Lisp, a derivative of Kyoto Common Lisp, is such 
a "C based" lisp.)  First, a general remark, comparisons with Franz Lisp
are simply not valid.  Professors Yuasa and Hagiya, the original designers
of KCL, had the benefit of 3 decades of work on Lisp implementation and 
compiler development (including Franz Lisp and all the more recent work 
on compiler technology) on which to base their work.  They knew all of 
this technology.  In addition they are first class computer scientists 
and awesome hackers.  They were extremely concerned with performance 
and they built a system which competes quite favorably with "direct" 
implementations. (John should consider not what he did those many years 
ago but rather what he would be capable of doing today.)

With respect to John's specific comments.

1) The use of global registers on the Sun 4.  

   "The Sun4 has 7 global registers.  Allegro stores nil and a
   pointer to our global function table in one of them, and
   uses another for tracking asynchronous interrupts. Another
   is used to pass argument/result counts.  C doesn't permit
   global register declarations so you can't take advantage
   of this nice feature of the Sun4 in a C-based Lisp."

Allegro seem to have 4 uses for the global registers:

  (a) pointing to a "global function table";
  (b) asynchronous interrupt "tracking";
  (c) nil testing.
  (d) multiple value handling;

(a) is not necessary.  I assume that the "global function table" is a transfer 
vector used to call Allegro kernel functions.  In IBCL and KCL calls to 
kernel functions are directly linked and so there is no need to call indirect
through a transfer vector.  (This is in fact an implicit win for the C based 
Lisps.) 

(b) is of marginal value.  As I recall, The Allegro compiler inserts code
to poll for interrupts in which case the register is obviously of value.  The
only interrupt polling in IBCL however occurs when entering critical sections.

(c) is of marginal value.  Not having nil in a register costs at most a couple
of cycles in doing a nil test (2 immediate instructions to build the constant.)
The only case in which this could possibly be significant is in an extremely 
tight loop but in that case the IBCL back-end optimizer (the C-compiler!) can 
simply reserve a register for nil and store nil once on loop entry.  

(d) is hard to assess because I do not know much about how Allegro handles 
multiple values and lambda list parsing.  Conceivably this could amount to 
a few percent in function call intensive code.  The next release of IBCL will
include considerable performance enhancements for function calling. (One of 
the beauties of the original work at Kyoto is the modularity and extensibility
of the implementation -- in particular the compiler.)

In short I strongly doubt that Allegro's use of global registers gives it
anything more than a marginal advantage over the C based Lisps.

(2) The ability to exploit tagged arithmetic.  IBCL and KCL do not currently
do any open coding of generic arithmetic.  This loses when generic
operators are used to do fixnum arithmetic.  IBCL will be making considerable 
improvements in this area in its next release but even so in most cases the
right thing to do is to use fixnum operators to do fixnum arithmetic, and 
floating point operators to do floating point arithmetic, i.e., add 
appropriate type declarations to the code.  When declarations are in place 
IBCL and KCL compile Lisp arithmetic to equivalent C arithmetic and from 
that point it is doubtful that the Allegro compiler will do better than the
optimizing C-compiler.

I would like to add an Amen to Jim McDonald's points about benchmarking
and interpretation of benchmarks.  [I saw an ad recently touting one lisp
as running  "23% faster" than another on the Gabriels (except it didn't call
them that) without any explanation of how the benchmarks were made and 
how they summarized the results.]  Advertized benchmarks are typically 
run on stand alone maxed out hardware.  On real systems the small size 
of the C based Lisps (the total image size is about 2 megs 
including compiler) is likely to be just as important as how registers
get used.

Another area where "C-based" lisps win is in the ability to tightly integrate
with "foreign" code.  IBCL and KCL use the systems object format and their
Lisp data structures are also C data structures.  The term "foreign" in this
case is essentially a misnomer.  

Another issue that benchmarking doesn't account for is the lead time
between the availability of hardware and the availability of a Lisp.
For the six months or so when IBCL was the only lisp available 
on the Sun 4 it might have been reasonable to compare IBCL on a Sun 4
with other Lisps on a Sun 3.   This is the current situation on HP
Spectrum series and the MIPS based platforms (e.g., the Silicon Graphics
4D).  This will undoubtedly be the situation for the new Apollo and
Motorola based RISCs in the future.  In each case the companies
producing direct implementations are going to be in the business of 
trying to duplicate all of the back end complexity of the manufacturer's 
compilers.  With the new architectures the complexity entailed in doing
decent instruction scheduling and register allocation and exploiting
low level parallelism is enormous.  I doubt that this duplication 
of effort will prove to be cost effective for most applications.

Dave Posner
IBUKI

ibuki!dbp@labrea.stanford.edu