Yegging Me On

I just can't stay away from Mr. Yegge's critique of Lisp. What is it about critiques of one's favorite language that makes one feel so righteously indignant? Perhaps it's some primordial fear that our favorite language will be taken away from us. Sort of like the basic fear by aged monolingual English speakers in Miami that as Spanish grows in visibility they will be unable to make themselves understood. As an aging almost-monolingual English speaker, I can easily imagine myself in their position and feel panic. The feeling that Lisp might vanish is very similar.

Anyway, putting these feelings aside, let me address one more little Yeggish point —

This is a problem. It's not a little teeny one, either. The Lisp communities (yeah, there are a bunch) are going to have to realize that if Lisp is ever going to be massively successful, it needs an overhaul. Or maybe a revolution. Contrary to what some might tell you, it doesn't need a committee, and it doesn't need a bunch of money. Linux proved exactly the opposite. Lisp needs a benevolent dictator. Lisp needs to ditch the name "Lisp", since it scares people. And Lisp needs to learn from the lessons of the 45 years of languages that have followed it.

At this point I begin to scratch my head. The point he started to try to argue was that Lisp is not an acceptable Lisp. But an acceptable Lisp would be an "overhaul" of the current language. As the paragraphs before the one quoted show, the overhauled Lisp would not have the "worthless spec" the current one has, it would not have CLOS, it would not have macros, and it would have a different type system. It would have a dictator (its own Guido van Rossum? or Larry Wall?). It would reflect 45 years of history somehow. And it wouldn't be called "Lisp."

To which I respond: Would we really want a Lisp that was an acceptable Lisp in this sense?

There is a lot I don't like about Common Lisp. None of it has much to do with Yegge's rather general observations. The only specific point he made that I agree with is that case-insensitivity is a big mistake. What were the Common Lisp people thinking? It might have been difficult to convert legacy code then, but it's not getting any easier, and sooner or later Lisp will change. Franz is already agitating for "modern" mode; I've adopted it, and I can't believe that it isn't spreading faster.

There are plenty of other flaws in CL. Brooks and Gabriel wrote a critique in 1984, when CL was proposed, that hit several nails on the head. I've posted a couple of essays (NIL Considered Harmful" and FORMAT Considered Ugly) on my website about flaws in CL, and I wrote the Nisp package because I think, contrary to the Lispish conventional wisdom, that static typing is a good idea.

But in spite of all that, Lisp embodies several good features that (a) I like, nay, passionately love; and (b) are so essential to the language that removing them would make it not Lisp any more:

1. S-expressions
2. Programs expressed as S-expressions
3. Macros
4. The read-eval-print loop

Let me elaborate.

S-expressions are symbols, strings, and numbers, plus lists of S-expressions and (why not?) arrays of S-expressions. That's their abstract syntax. Their concrete syntax is:

• Symbols, strings, and numbers What you would expect.
  


• Lists: (e1 ... eK)
  


• Arrays: #(e1 ... eK) (Actually, this is just the 1-dim case, but let's not worry about that.)
  

The thing to notice is that the abstract syntax and the concrete syntax are essentially the same, so that S-expressions provide the same thing XML that provides: A notion of hierarchy that is representable and parseable before the hierarchy is classified as being an instance of this or that particular syntactic category, data structure, or whatever. (But S-expressions are more concise than XML, at the expense of being less general in some ways.)

It's important that programs are S-expressions, but not just Lisp programs. The JScheme system, by Ken Anderson, Tim Hickey, and Peter Norvig, is an implementation of Scheme in Java that allows java to be called from Scheme, using what it inventors call Javadot notation. The expression object.field becomes (.field$ class). The expression "new class(a b)" becomes (class. a b). This notation was developed prior to the addition of generic classes and methods to Java (version 1.5). It is now possible in Java to write, for instance, "(new List<String>())" to make an empty list of strings. But it's clear that augmenting the notation in an elegant way is not going to be difficult, because the S-expression notation allows us to capture the hierarchical structure of the Java expression without constraining how we do it. We might write (List< String)), or ((<> List. String)). I leave it to the JScheme people to arrive at an extended notation that actually fits their original conception.

Another example is the Common SQL notation developed by Lispworks. When sending an expression to the database, the expression might look like (select [Researcher] :from [SampleAreas]). (This notation depends on hacking the syntax of the character '[' as well as using S-expression notation.) In other languages you have to send SQL expressions as (brace yourself) strings. To construct what is obviously a hierarchical expression involving data objects, one must discard the hierarchy and the objects, convert both to strings, and send the result to the database, which will then invert the process, recovering the hierarchical structure. S-expressions are an obviously better idea.

Now for macros. Let me first observe that macros are not the only useful consequence of the fact that Lisp programs are S-expressions. Another is quote. It's easy to write Lisp as a package or class in Java, until you come to a point when you need to translate, say, (setq x 'foo). You suddenly have to allocate a new static variable Q_foo somewhere and bind it at static-variable initialization time to the symbol foo. This is a real buzz-kill.

But the existence of macros is the really good reason why programs should be S-expressions. The debate about whether this power is a good thing is not important. We are not debating here. We are explaining what Lisp is. Macros are a non-negotiable item. For one thing, if they were repealed, anyone could reintroduce them. (All you need to do is introduce a preprocessor before the compiler; introduce your own version of 'defun' that runs the preprocessor.)

Even Scheme has macros, although they are of the "hygienic" variety. I don't think I've ever written a hygienic macro; by chance, all the interesting ones seem to be un-hygienic. Nonetheless, I still have all my fingers and toes. The desire to restrict the syntactic transformations a macro can perform is in keeping with the general atmosphere around Scheme, that there is one right way to do any given thing, and until the Central Committee announces what it is, we should avoid doing that thing. Yegge recommends that the Lisp community elect (or suffer the ascension of?) a dictator, but anyone can decide to live under the benevolent rule of the Revisedⁿ Report on Scheme. I gather this particular dictator is not appealing to him, and I wonder what the odds are that the next one will be either.

Finally, the read-eval-print loop. Some languages besides Lisp, or their IDEs, approximate it, but for various reasons they can't quite get to the right idea: a playground where lots of pieces of code are lying around, and you can work on whichever one catches your fancy next. It's like the shell in Unix, except the shell is more of an oil refinery than a playground. But once you're in Lisp, there's no reason to go back to the shell. It's like Emacs; correction: Emacs is like it; correction: Emacs is it, because Emacs is basically a special-purpose Lisp.

Obviously, the Lisp idea is not for everyone. But one must acknowledge that there is such a thing as the Lisp idea. John McCarthy once said that Lisp seemed to be a local maximum in the space of programming languages. Most successful languages are. It's a characteristic of local maxima in high-dimensional spaces that small steps in any direction can plunge you into the depths. Mr. Yegge's casual suggestions about Lisp reforms are actually more likely to be high dives into the lower rings of hell.