Scheme vs Common Lisp: Which characteristics made a difference in your project? [closed]

Question

There are no shortage of vague "Scheme vs Common Lisp" questions on both StackOverflow and on this site, so I want to make this one more focused. The question is for people who have coded in both languages:

While coding in Scheme, what specific elements of your Common Lisp coding experience did you miss most? Or, inversely, while coding in Common Lisp, what did you miss from coding in Scheme?

I don't necessarily mean just language features. The following are all valid things to miss, as far as the question is concerned:

• Specific libraries.
• Specific features of development environments like SLIME, DrRacket, etc.
• Features of particular implementations, like Gambit's ability to write blocks of C code directly into your Scheme source.
• And of course, language features.

Examples of the sort of answers I'm hoping for:

• "I was trying to implement X in Common Lisp, and if I had Scheme's first-class continuations, I totally would've just done Y, but instead I had to do Z, which was more of a pain."
• "Scripting the build process in my Scheme project got increasingly painful as my source tree grew and I linked in more and more C libraries. For my next project, I moved back to Common Lisp."
• "I have a large existing C++ codebase, and for me, being able to embed C++ calls directly in my Gambit Scheme code was totally worth any shortcomings that Scheme may have vs Common Lisp, even including lack of SWIG support."

So, I'm hoping for war stories, rather than general sentiments like "Scheme is a simpler language" etc.

Answer 1

My undergrad degree was in Cognitive Science and Artificial Intelligence. From that I had a one-course intro to Lisp. I thought the language was interesting (as in "elegant") but didn't really think much of it until I came across Greenspun's Tenth Rule much later:

Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.

Greenspun's point was (in part) that many complex programs have built-in interpreters. Rather than building an interpreter into a language he suggested it might make more sense to use a language like Lisp that already has an interpreter (or compiler) built-in.

At the time I had been working on a rather big app that performed user-defined calculations using a custom interpreter for a custom language. I decided to try re-writing its core in Lisp as a large-scale experiment.

It took roughly six weeks. The original code was ~100,000 lines of Delphi (a Pascal variant). In Lisp that was reduced to ~10,000 lines. Even more surprising, though, was the fact that the Lisp engine was 3-6 times faster. And keep in mind that this was the work of a Lisp neophyte! That whole experience was quite an eye-opener for me; for the first time I saw the possibility of combining performance and expressiveness in a single language.

Some time later when I started working on a web-based project I auditioned a number of languages. I included Lisp and Scheme in the mix. In the end I selected a Scheme implementation--Chez Scheme. I've been very happy with the results.

The web-based project is a high-performance "selection engine". We use Scheme in a number of different ways, from processing data to querying data to page generation. In many spots we actually started off with a different language but ended up migrating to Scheme for reasons I'll describe briefly below.

Now I can answer your question (at least in part).

During the audition we looked at a variety of Lisp and Scheme implementations. On the Lisp side we looked at (I believe) Allegro CL, CMUCL, SBCL and LispWorks. On the Scheme side we looked at (I believe) Bigloo, Chicken, Chez, Gambit. (The language selection was a long time ago; that's why I'm a bit hazy. I can dig up some notes if it's important.)

Right off the bat we were looking for a) native threads and b) Linux, Mac and Windows support. Those two conditions combined knocked everyone but (I think) Allegro and Chez out--so in order to continue the evaluation we had to loosen the multi-threading requirement.

We put together a suite of small programs and used them for evaluation and testing. That revealed a number of issues. For example: some implementations had defects that prevented some tests from running to completion; some implementations couldn't compile code at run-time; some implementations couldn't easily integrate run-time compiled code with pre-compiled code; some implementations had garbage collectors which were clearly better (or clearly worse) than the others'; etc.

For our needs only the three commercial implementations--Allegro, Chez and Lispworks--passed our primary tests. Of the three only Chez passed all tests with flying colors. At the time I think Lispworks didn't have native threads on any platform (I think they do now) and I think Allegro only had native threads on some platforms. Furthermore, Allegro had a "call us" run-time licensing fee which I didn't like very much. I believe Lispworks had no run-time fee and Chez had a straightforward (and very reasonable) arrangement (and it only kicked in if you used the compiler at run-time).

Having produced somewhat significant chunks of code in both Lisp and Scheme here are some compare and contrast points:

• The Lisp environments are far more mature. You get a lot more bang for the buck. (Having said that, more code also equates to more bugs.)

• The Lisp environments are far more difficult to learn. You need a lot more time to become proficient; Common Lisp is a huge language--and that's before you get to the libraries that the commercial implementations add on top of it. (Having said that, Scheme's syntax-case is far more subtle and complicated than any one thing in Lisp.)

• The Lisp environments can be somewhat more difficult to produce binaries in. You need to "shake" your image to remove unneeded bits, and if you don't exercise your program correctly during that process you could end up with run-time errors later on. By contrast, with Chez we compile a top-level file that includes all of the other files it needs and we're done.

I said before that we ended up using Scheme in a number of places we didn't originally intend to. Why? I can think of three reasons off the top of my head.

First, we learned to trust Chez (and its developer, Cadence). We asked a lot from the tool, and it consistently delivered. For example, Chez has historically had a trivially small number of defects, and its memory manager has been very, very good.

Second, we learned to love the performance we got from Chez. We were using something that felt like a scripting language--and we were getting native-code speed from it. For some things that didn't matter--but it never hurt, and sometimes it helped an awful lot.

Third, we learned to love the abstraction Scheme could provide. I don't just mean macros, by the way; I mean things like closures, lambdas, tail-calls, etc. Once you start thinking in those terms other languages seem rather limited by comparison.

Is Scheme perfect? No; it's a trade-off. First, it allows individual developers to be more effective--but it's more difficult for developers to grok each other's code because the signposts that most languages have (e.g., for loops) are missing in Scheme (e.g., there are a million ways to do a for loop). Second, there's a much smaller pool of developers to talk to, hire from, borrow from, etc.

To sum it up, I think I'd say: Lisp and Scheme offer some capabilities not widely available anywhere else. That capability is a trade-off, so it had better be one that makes sense in your particular case. In our case the determining factors between whether to go with Lisp or Scheme had more to do with very fundamental features (platform support, platform threads, run-time compilation, run-time licensing) than they did with language or library features. Again, in our case that too was a trade-off: with Chez we got the core features we wanted but we lost the extensive libraries the commercial Lisp environments had.

Also, just to reiterate: we looked at the various Lisps and Schemes a long time ago; they've all evolved and improved since.

Answer 2

I usually don't like pasting a link as an answer, but I wrote a blog article on this very thing. It is not exhaustive, but it gets some of the major points through.

http://symbo1ics.com/blog/?p=729

Edit: Here are the principle points:

1. EXISTENCE: Both lisps came after a bunch of other lisps. Scheme took the minimal, axiomatic route. CL took the baroque route.
2. CASE: Typically Scheme is case sensitive. CL is not (though it can be). This is sometimes missed, but its practicality is debated (by me).
3. NAMES: Names of symbols in CL are many times odd and confusing. TERPRI, PROGN, etc. Scheme usually has very sensible names. This is something missed in CL.
4. FUNCTIONS: CL has a separate function namespace. This is not missed in Scheme. Having a single namespace usually allows for very clean functional programming, which is often hard or awkward in CL. But it comes at a cost---you sometimes have to obfuscate names like "list" to "lst" in Scheme.
5. MACROS: I miss low-level dirty macros the most in Scheme. Yeah, syntax-rules is all fine and dandy until you want to really hack some things out. On the other hand, hygienic macros are sometimes missed in CL. Having no standard way to do them means re-inventing the wheel.
6. PORTABILITY: It's often the case that CL is more portable, despite both languages being standardized. CL is bigger, and therefore there are more standard features to use without external libraries. It also means more implementation-dependent things can be done portably. Also, Scheme suffers from having a trillion implementations, most of which are somewhat incompatible. This makes CL very desirable.
7. LIBRARIES: Very related to my last point. Scheme has SRFIs but are not universally acknowledged. There's no portable way to work with libraries. CL on the other hand does have ways. And Quicklisp is a gift from god (Xach) --- a sort of repository of libraries for use.
8. IMPLEMENTATIONS: Scheme suffers from having so many implementations. There is no real canonical implementation. CL on the other hand has some very nice high performance or specific-use implementations (high performace: SBCL, commercial: Allegro, embedded: ECL, portable: CLISP, Java: ABCL, ...).

While I only talked in first person a bit above, it should be clear what I miss and what I don't.

[I do apologize if these are too general. It seems you may want a lot more specific details. There are some specifics in the post.]

Answer 3

I recently started a home project using a library that has a C version and a Java version. I wanted to use Lisp for the project, and I spent about a month vacillating between using Common Lisp, Scheme or Clojure. I have some experience with all three, but only toy projects. I'll tell you a bit about my experience with each of them before telling you which one I ended up choosing.

PLT Racket has a nice IDE that not only lets you evaluate expressions from the editor, but also lets you type brackets instead of parens, switching them back to parens where appropriate. Racket also has a large set of libraries with the install and even more available for download. The visual debugger is also helpful.

My Common Lisp implementation (SBCL) doesn't have an IDE, but it's customary with open-source CL implementations to use Emacs and SLIME. This combination can be very efficient. Along with the ability to evaluate expressions as you type them into the source file, there's also a REPL that has all of the editing commands of emacs available, so code copying can go efficiently both ways. Even objects displayed in the REPL buffer can be copied and pasted. Alt+( and Alt+) are efficient for dealing with matched parentheses and indentation.

All of the above Emacs features are also available for Clojure. My editing experience with Clojure is similar to that of Lisp. The Java interop worked fine, and I'd like to do a Clojure project once it matures.

I was able to access the library using all three (Common Lisp, Racket, and Clojure), but I ended up choosing Common Lisp for the project. The deciding factor was that the FFI was much easier to use in Common Lisp. CFFI has a very good manual with example code and detailed explanations of each method. I was able to wrap 20 C functions in an afternoon and haven't had to touch the code since.

The other factor was that I'm more familiar with Common Lisp than with Clojure or R6RS Scheme. I've read most of Practical Common Lisp and Graham's books, and I'm comfortable with the Hyperspec. It's not very "lispy" code yet, but I'm sure that will change as I gain more experience.

Answer 4

I program in both CL and Racket.

I'm developing a Web site now in Common Lisp, and I wrote a suite of in-house programs for my previous employer in Racket.

For the in-house code, I chose Racket (then known as PLT Scheme) because the employer was a Windows shop, and I couldn't get them to pay for LispWorks. The only good open-source CL implementation for Windows was (and still is) CCL, which requires SSE support in the processor. The employer, being cheap, was using Stone Age hardware. Even if the employer did have decent hardware, the only GUI library of consequence in Common Lisp is McCLIM, which only works on Unix. Racket has a good GUI library that works on both Unix and Windows, which was critical for my project's success.

I spent over a year putting up with the primitive DrRacket editor. EMACS couldn't turn the GUI version of Racket, then known as MrEd, into an inferior-lisp on Windows. I had to do without being able to evaluate the expression at the cursor with a single keystroke. Instead, I had to manually select the S-expression, copy it, click on the REPL window (because there's no keystroke to switch to it), and then paste the S-expression. I also had to do without an editor that could show me the expected arguments of the function or macro I was using. DrRacket is no substitute for SLIME.

The employer was using a proprietary database with a complicated XML API that required loads of seemingly unnecessary information to be able to respond to its version of a SELECT query. I decided to use HTMLPrag both to emit XML to this API, and to parse the responses. It worked great.

I had to learn Racket's overcomplicated "syntax-case" macro system in order to write a macro that would allow me to interact with the overcomplicated XML API by typing forms that looked like SQL. This part would have been much easier if I had DEFMACRO at my disposal. However, the end result was still seamless even though it took more effort to achieve.

Furthermore, I had to do without Common Lisp's LOOP macro. Racket started providing an alternative only after I had written most of the code, and the alternative still sucks compared to LOOP (even though Racket's development team insists it's better-- they're simply wrong). I ended up writing a lot of named LET forms that used "car" and "cdr" to iterate over lists.

Speaking of car and cdr, nothing is more frustrating than Scheme's interpretation of (car '()) as being an error. I took advantage of Racket's case-sensitivity and implemented CAR and CDR, which have Common Lisp's semantics. However, the separation of '() and #f makes it far less useful to return '() as a default value.

I also ended up re-implementing UNWIND-PROTECT, and invented my own restart system to fill the gap left by Racket. The Racket community needs to learn that restarts are very useful, and easy to implement.

Racket's let-values form was overly verbose, so I implemented MULTIPLE-VALUE-BIND. This was absolutely necessary, because Racket requires you to receive all the values that are generated, whether you use them or not.

Later, I attempted to write an eBay XML API client in Common Lisp, only to find that it doesn't have anything like HTMLPrag. HTMLPrag is frickin' useful. I ended up doing that project in Racket. I experimented with Racket's Literate Programming facilities, only to discover that I'm the only programmer on Earth who finds properly-written literate code harder to edit than ordinary code, or improperly-written "excessive comments" literate code.

My new project is being done in Common Lisp, which was the right choice because the Racket community just doesn't believe in parallelism, which is essential for this project. The only thing I thought I might have missed from Racket was continuations. However, I was able to do what I needed by using restarts, and, in retrospect, probably could have done it with a simple closure.

Answer 5

Scheme is designed with a separate compilation in mind. As a consequence, a power of its macros is often limited severely, even with the extensions that allow a Common Lisp-style defmacro instead of a poor, limiting hygienic macro system. It is not always possible to define a macro which defines another macro, intended for an immediate use in a next line of code. And such a possibility is essential for implementing efficient eDSL compilers.

Needless to mention that Scheme implementations with only R5RS hygienic macros are barely useful for me, as my metaprogramming style can not be adequately translated onto hygiene.

Fortunately, there are Scheme implementations (e.g., Racket) that do not have that limitation.