Splicing reader macros

[David A. Moon <Moon at SCRC-TENEX at MIT-MC>]

I think I forgot to bring this up at the meeting last Saturday.

The cute kludge by which splicing reader macros (reduced to just
reader macros that don't read anything, e.g. comments and unsatisfied
conditionals) identify themselves, namely returning (VALUES), doesn't
work so nicely now that the multiple-value-taking forms with &OPTIONAL
and &REST have been removed from the language.

Someone (Eric Benson?) suggested that such macros call READ tail recursively.
Of course this doesn't work, since you can have situations like
	(COND ((FOO-P A)
	       (BLATZ B)
	       ;Okay, that takes care of B
	       ))
where the macro is followed by a special token, not by an S-expressions.
However, clearly there is a function inside the reader which such macros
could call tail recursively if they could only get at it.  For instance,
in the Lisp machine this is called SI:XR-READ-THING.

I suggest that we come up with a reasonable name for this function, perhaps
something like READ-INTERNAL-TOKEN, document it, and allow reader macros
to call it.  We could document the two values it returns (an object and what
kind of token it is), or we could say that all you can do with the values
is return them to the reader and they are its internal business.