Is LISP still useful in today's world? Which version is most used?

Question

I try to teach myself a new programming language in regular intervals of time. Recently, I've read how Lisp and its dialects are at the complete opposite end of the spectrum from languages like C/C++, which made me curious enough to know more about it. However, two things are unclear to me, and I'm looking for guidance on them :

1. Is LISP still practiced/used in todays world, or is it a legacy language like FORTRAN/COBOL ? I mean, apart from maintaining existing code, is it used on new projects at all ?

2. What is the most widely used dialect ? I came across Scheme and Common Lisp as the 2 most prevalent dialects, and wanted your opinion as to which is the most favored/useful one to learn - and would be immensely gratified if you can suggest any resources for a rank beginner to start from.

While eager to learn a language which is fundamentally different from the procedural languages I'm used to, I don't want to invest undue effort in something if its totally obsolete - I'd still learn it if it was professionally "dead", but only with an academic perspective...

Answer 1

I rather like Scheme, if you want to work with the JVM you should check out Clojure, which is a lisp that is designed to work in the JVM. And yes Lisp is still worth learning to see how powerful such a minimal design can be! The folks who created lisp got some things really right. Its amazing how many of the cool new features of modern languages lisp had in the 1960's!

For an embeded scheme try guile: http://www.gnu.org/s/guile/

Answer 2

In answer to question 2, "Most widely used dialect":

Depends on how you interpret "Most widely used." Here are some napkin math for lower bounds on how much use each of a few chosen dialects get.

Most raw code written

• Possibly Scheme (all dialects) because of educational applications
  • 270 schools... Let's guess 50 students/year, 500 lines of code/student, 10 years running. 1,000,000+ lines based on napkin math.
  • 500,000 lines of Racket in Racket itself and its tools.
• Possibly Common Lisp because of AI and a whole lot of open source code and production code (See production code).
  • Cocreate Modelling (now from PTC): several million lines of Common Lisp
  • 200,000+ Lines from Orbitz alone. Could be a fluke, but probably not.
• Possibly Emacs Lisp because of...
  • Emacs itself: over 1,000,000 lines
  • The amount of emacs lisp written publicly - 237 modules, about 200 lines/modules is 40,000+ lines just on EmacsWiki, probably much more in ELPA
  • Plus applications written in emacs Lisp, .emacs files, etc.
• Clojure
  • Hard to estimate. Mostly smaller dotcoms. Could be a lot.

Winner: Scheme or Common Lisp. Too many unknowns.

Most 'production' code

• Scheme - Some, mostly open source.
  • 500,000 lines of Racket in Racket itself and its tools.
  • Game scripting, e.g. in game development at Naughty Dog
• Common Lisp - Hard to say. Probably a lot of internal use.
  • Cocreate Modelling (now from PTC): several million lines of Common Lisp
  • "200,000 lines of Lisp in our base search engine"
  • Many others commercial
  • And probably some open source.
  • And, of course, the Common Lisp in the implementations themselves.
• Emacs Lisp - At least 1,000,000 lines counting Emacs itself. Other than that, not a lot (but some, apparently).
• Clojure - Hard to say, definitely some use, e.g. Etsy. The presence of books on shelves begins to suggest it.

Winner: I'm guessing Common Lisp. I think we can answer "Yes" to question 1 based on the use of Common Lisp and Clojure.

Most end users

• Common Lisp
  • ITA Software customers, including Orbitz, Bing Travel and Kayak.com - Probably millions.
• Clojure
  • Citigroup (no details), Akamai (no details), simple.com, many others. Probably millions.
• Scheme
  • CS students, researchers and hobbyists. Less than a bazillion.
  • The Uncharted series - Over 5 million
• Emacs Lisp - Mostly programmers, mostly on unix platforms. Could be a million.

Winner: Scheme has the best lower bound here, since there are sales numbers on Uncharted, but it's quite debatable.

Conclusion

In conclusion, I find that large apples.

Answer 3

Is LISP still practiced/used in todays world, or is it a legacy language

Yes, it is, but you have to know where to look. People who use LISP don't tend to shout too loudly about it but there's a handful of examples of a few high-profile startups having used it to great effect over the last 20 years. It is also very popular with small companies in Europe.

What is the most widely used dialect ?

This is a valid question, but it's not an easy one to answer. It may also not be a particularly useful one to answer: many implementations have a specific focus so are best chosen if they fit your particular problem rather than based on how widely it is otherwise used. Instead, I'll tell you a little bit about your options and you can decide for yourself.

LISP is a family of languages and each of those languages has a family of dialects and implementations. Broadly dialects fall into two camps "LISPs" and "Schemes".

LISPs: Until relatively recently, Common LISP was king. It was an attempt to unify all the disparate LISPs and, without being unkind, was the "C++" of LISP. That is to say, it was a HUGE language. It had everything. Over the last few years, Clojure has appeared. Clojure is a LISP that runs on the Java Virtual Machine and tries to root itself in a functional programming philosophy. Traditionally, other LISPs have been strictly multi-paradigm. Clojure is interesting because it gets both the very best and very worst of both LISP and the JVM. There's still a lot of the verbosity of Java-based languages and they've been fairly free and easy with the syntax so that has lots of knobs and buttons for different things, but they've got some really interesting ideas around datatypes, especially some of the practical ways they have come up with to apply ideas from functional programming.

Schemes: Schemes are a strict subset of LISPs. Scheme was invented by Steele and Sussman and in early life was notable for being used in the MIT Computing 101 Lecture Course. Scheme is defined in the "Revised^n Report on the Algorithmic Language Scheme (RnRS)". Yes: they have a maths joke in there. Scheme is a standardised language in a way that other LISPs are not. This helps greatly with portability between implementations, but it's not a silver bullet. The standardisation efforts have tended to be conservative and innovations in the implementations, especially around things like modules, have tended to be disparate. There is also a series of SRFIs (Scheme Requests For Implementation) which is similar to the IETF's RFC process. People use it to standardise small things as needed.

Schemes are different from LISPs that they have a set of hard requirements that they must satisfy, one of which is "tail call optimisation" which helps make recursion efficient. Therefore, recursive styles of programming are much more popular in Scheme than in LISP. Scheme is, also without being unkind, like the "C" of LISP. That is to say, it's a small languages and you're supposed to be able to keep all of it in your head at once.

There are currently two Scheme families: Those based on the 5th version (R5RS) and those based on the 6th version (R6RS). The complexity of R6RS was wildly greater than that of any of its predecessors and so many R5RS implementations have opted to skip it, hoping that R7RS will be more similar to R5RS than R6RS. The R7RS standardisation process is currently ongoing and has tried to include both the R5RS implementers' desires as well as those of the R6RS folk by standardising a small base language in their first working group and then commissioning a second working group to standardise the larger features. This will allow the language to have efficient and useful implementations on both tiny embedded hardware as well as more capable machines.

Now I'll be more specific:

PicoLisp is a really, really cool LISP. It's tiny! It's author wrote it for himself and, as I understand it, he's been making a living off of it since the 1980s. If you ever get the opportunity to attend a talk by him then you should do it: he's really interesting and really knows his stuff and you won't get even the smallest sniff of anything mainstream or boring.

I'm not familiar with Common Lisp implementations so I won't comment further on them.

Guile is the official GNU Scheme.

Racket is an R6RS scheme but lately it seems to have widened the net and is trying "to serve as a platform for language creation, design, and implementation.".

Chicken aims to be a practical Scheme. It's based on R5RS and compiles down to C. This turns out to be a really, really, important advantage as it makes it absolutely trivial to use existing C libraries. Therefore Chicken is probably the most useful scheme for replacing Perl, Python, Ruby, etc, as your day-to-day scripting language. There are several people who have used it exclusively for all their needs for several years. It has an interactive REPL as well as a compiler. The community (both on the mailing list and IRC) are knowledgeable, friendly and helpful.

Look for an implementation with lots of modules: this shows that it's widely usable and means that it's likely to have something that helps with the task at hand.

Look for an implementation with a compiler or, at the very least, something that isn't strictly IDE or REPL based. A lot of the implementations designed for teaching are very difficult to use for general purpose scripting.

I'd recommend Chicken as that's what I use. I've used it in my personal projects and I've used it (and am currently using it) professionally.

I don't want to invest undue effort in something if its totally obsolete - I'd still learn it if it was professionally "dead", but only with an academic perspective...

Scheme isn't professionally dead, but you might have to go to some lengths to use it in that context. Something like Chicken is far more than an academic pursuit and it can easily cover almost all the bases of whatever high-level languages you currently use.

Answer 4

Can't really speak for all Lisps but Clojure is definitely a hot and relevant language at present. A London Clojure User Group I went to earlier this week had over 100 attendees....

I've found it to be a very enlightening experience to learn Lisp in the form of Clojure over the past year (after a lot of experience with Java and C#). Main reasons for this are:

• It has quite a strong emphasis on functional programming (more so than most other Lisps). The author and BDFL Rich Hickey has frequently cited Haskell as one of his inspirations for the language design which means you get things like fully immutable data structures and lazy infinite sequences etc.
• Macro metaprogramming - the Lisp "code is data" philosophy is hard to understand unless you've actually experienced it, but it's one of the reasons Lisps are so expressive and productive
• Fantastic support for multi-core concurrency - I actually think Clojure is the best language for concurrent programming right now. See http://www.infoq.com/presentations/Value-Identity-State-Rich-Hickey for an enlightening presentation about this

It also seems to be a practical choice for real production use for the following reasons:

• Running on the JVM with very easy Java interoperabilitiy gives you access to all the libraries and tools in the Java ecosystem
• It's a dynamic language by default, which makes it very convenient for development and rapid prototyping with hardly any boilerplate. However you can add static type hints to get pretty good performance where you need it.

I personally know of people using Clojure in a couple of investment banks and startups. I've also chosen Clojure as the primary development language for my own startup, so I'm willing to put my money where my mouth is :-)

Answer 5

I'm currently learning Lisp as well (and I love it). I use Common Lisp but I have also messed around with SBCL, Arc (note, that is Paul Graham's version of Lisp who, as mentioned by Vitor Braga created Yahoo store/Viaweb) and Allegro CL which is similar to Visual Studio for Lisp (can create GUIs, however, I've never used it).

As for uses, Lisp has been used a lot in Artificial Intelligence but I'll be honest, I'm not sure how many other "general" uses for Lisp. A lot of the websites that were built in Lisp originally have been re-written in other languages so it is hard to say that it is used in web development (not saying it isn't, but the larger sites that use it no longer do). After doing a quick search (very quick) here is a list of software written in Common Lisp from Wikipedia.

[EDIT]As for using Lisp professionally, there are jobs out there that would have you use Lisp. They aren't as numerous as, say Java or C# jobs, but they exist. I think Lisp is one of those languages that is used for internal applications and may provide a competitive advantage which companies don't want to give up by advertising that they use Lisp. I remember seeing a post on P.SE that stated Smalltalk was similar in the financial arena.

Also, being able to show you are able to learn different paradigms can open up more doors even if you don't use Lisp in the job.

"Lisp is worth learning for the profound enlightenment experience you will have when you finally get it; that experience will make you a better programmer for the rest of your days, even if you never actually use Lisp itself a lot." - Eric Raymond, "How to Become a Hacker"

[/EDIT]

Answer 6

If you want to learn Lisp today, I'd have a look at either racket, which is a fast scheme implementation (well, it actually departed a little from scheme, so it is its own dialect now) or clojure, which benefits from the JVM it runs on (so gazillions of libraries are available, plus you can make it interact with your own Java code).

Even if you don't learn it to actually use it, learning it is always beneficial : you learn new ways to think and deal with problems, even in other languages, once you've wrapped your mind around Lisp for a while.

Answer 7

A significant part of Amazon backend used to be in Lisp but now was rewritten in C++, or so I have heard (I must admit I don't have a reliable source for this).

Yahoo! Stores is one of classic examples of Lisp web apps. From the Yahoo! Stores fame you may have heard of Paul Graham. Graham is one of the best known Lisp advocates and writes extensively about this subject. You might want to read his site to known his points.

AutoCAD used to have a Lisp dialect for it's macro system. I don't know if it still does.

Scheme is a clean language and very elegant. It's probably my favorite programming language so I may be biased. If I was going to write right now a major application I'd probably write a skeleton application in C, extend and define business rules in Scheme. This allows me use Scheme and to leverage C - for speed and for the sheer availability of libraries for almost everything.

Answer 8

Is LISP still practiced/used in todays world, or is it a legacy language like FORTRAN/COBOL ? I mean, apart from maintaining existing code, is it used on new projects at all ?

I know several guys who do Lisps in some startups in Silicon Valley, and I know that Amazon.com has been using Lisp since the beginning (though I've heard they're replacing it with C++ for some reason?)

But one company to keep an eye on is Naughty Dog. All of their games are written with a Lisp dialect. Originally, they rolled their own, but they use MZScheme in the Uncharted series.

What is the most widely used dialect ? I came across Scheme and Common Lisp as the 2 most prevalent dialects, and wanted your opinion as to which is the most favored/useful one to learn - and would be immensely gratified if you can suggest any resources for a rank beginner to start from.

I'd wager that Common Lisp, Clojure, Scheme, and Emacs Lisp are the four most widely used dialects, and of those, I'd suspect that Scheme is the most commonly-deployed. I don't have anything to back this up with, of course. :)

I rather like SICP and the Little Schemer, as suggested by others, but I'd also suggest Land of Lisp, which is a rather entertaining read. :)

Answer 9

I think it partially depends on what you want to do with it - if you are looking at furthering your insights into the various concepts of programming and make yourself a better programmer then I would say it is worth learning at least a modicum of Lisp. If you're looking for yet another language to add to your resume with a view to getting a job working with this language, you probably want to look elsewhere. There aren't that many Lisp jobs out there.

I personally try using either SBCL or more recently, Clojure (and some Emacs Lisp, but htat's because I'm a long term Emacs user - I would try to learn Emacs Lisp when I try learning Lisp). Now all I need to do is find the time to actually play with those languages...

Answer 10

I don't know that there are a ton of jobs in LISP, I certainly cannot see them. But I do remember a long time reading about some NASA probe where they were running LISP and were able to insert new code from earth.

Also in New York the clojure meetup group is huge. I'm guessing that if you are interested and go to your clojure meetup group you may find opportunities to network and find jobs (not clojure jobs but things like Java/C++/etc.). It seems to be absolutely enormous in the New York area, other areas may vary.

Also LISP is a different way of thinking. Additionally LISP and SQL both result in using tons of nested expressions. I used SQL a ton and then notice LISP made more sense. But if you have trouble with SQL getting used to super nested parentheses and expressions while using LISP will probably make SQL expressions much easier to figure out.

A classic example is how to implement MAX(a, b, c). You could make a convoluted function with a bunch of if statements. Or you could just say

MAX( MAX( a, b), c)

and use two simple nested calls of the two item MAX, which to me is easier to read. Although if performance is an issue you might want to do it the other way too, I did not bother to count the number of comparisons when using each method... Also if you are implementing MAX via a C macro or other method that evaluates the expressions multiple times you may not get the expected result as the expression may be evaluated multiple times so watch out with side effects...

Answer 11

Two additional points:

First, Lisp is a great language for writing code in which interactions between functions or data are often complex. In many popular languages, if you're confused about why your program is doing what it's doing, you have to write special functions to let you examine the program's internal state as it's going along. In some languages, you then have to wait for your code to recompile. Of course there may be tools such as debuggers that help with this. In Lisp, though, all you need is a way to stop the program, and then, typically, you have access to everything in Lisp. In Common Lisp, sometimes I just type ^C, or throw a call to error into a single function (which is the only thing that must get recompiled, and you don't have to do anything to recompile it). Once I stop the program, I'm instantly thrown into a debugger--but the debugger gives me all of the power of Lisp, and I can exit the debugger, call specific functions, etc. I can easily examine any of the data structures in my program. Most Scheme's should allow similar practices. Lisp is not unique in this regard, but what it offers goes beyond what many popular languages offer.

Second: If you're going to experiment with any dialect of Lisp, I would not do it without (a) using an editor that matches parentheses, and (b) learning proper Lisp indentation. It would be a good idea to use an editor that will approximate proper indentation for you, too. C/C++/Java-style code formatting is designed for languages with fewer parentheses/braces/brackets than Lisp. Lisp pretty printing style works well once you're familiar with it, and you don't end up being confused by parentheses or wasting half of your space putting one closing parenthesis per line.

Answer 12

Right now I get the impression that Lisp is mostly used in consulting shops (not that consulting shops use it mostly).

It's considered a bit outre for practical software. Mostly because people aren't used to it, I believe.

Traditionally, Scheme is a fairly academic dialect of Lisp, and Common Lisp was the industry dialect.

Lisp is particularly useful for symbolic manipulation and reflection-esque capabilities.

As an example, the code I wrote to learn Lisp was an program that constructed random lambda functions, evaluated them, and then operated on them, in an attempt to minimize the function's difference with a target function. All of that was direct manipulation. In a language like C++ or Java, I would have had to invent a representation for the functions that the computer would manipulate.

Answer 13

I'd like to add some perspective about Common Lisp and Scheme. I've avoided clojure due to the JVM, so I don't have a lot of perspective on it.

Scheme is a beautiful concise, well defined language. It took a lot of decisions that CL had to make due to legacy, and took the more pure approach. For instance, variables and functions are in the same namespace. The default language is smaller and more concise, which works well for teaching you the sorts of things that lisp is good at, for instance it pressures usage of recursion instead of iteration, since it by default has no iterative loops, and requires tail call optimization. Scheme has a very interesting hygienic macro system, which there is much value to learning. But IMO, I would recommend learning CL-style Unhygienic macros first, of which most(?) scheme dialects offer at least one implementation of, even if it's not a part of the spec. Since it seems very much that you want to learn a language for learning's sake I'd recommend Scheme, since additionally, many of it's concepts are easily transferred to other lisps, and dynamic languages in general.

Having said that, Scheme's community is fragmented, and other than in a few small pockets, seems to be mostly dedicated to research over getting real work done... For instance every dialect has their own package manager, and packages are most often not portable across dialects, which is a huge problem in the scheme community.

Common Lisp on the other hand, seems to be the much more pragmatic approach to developing a language. The standard is such that much lisp code is portable between implementations, many compilers are fast and well-optimized. There is a large set of packages, many of which are portable between implementations. And seemingly a (comparatively) large amount of real products are getting created in the language. And with say quicklisp, it's package management feels reasonably close to what you'd get in a modern community like say Ruby, or Node. CL, is however not without flaws, it has a very large spec at this point compared to scheme, and even it's object system CLOS, something that is traditionally written as a library in Lisps, is part of the spec IIRC. And you can feel legacy entrenched through a large amount of the system. The standard build system approach of extracting an image for compilation feels particular arcane, for example.

Otherwise, I said I couldn't speak on clojure, and Emacs Lisp is clearly the appropriate choice if your goal is to write Emacs extensions, and pretty clearly not the appropriate choice for other software.

TLDR; If your goal is to learn, I recommend Scheme. However, if you want to create software, I'd recommend Common Lisp of the two major lisp variants.

Answer 14

I learned a lot from the answers. Thank you to all who contributed to this enlightening conversation.

I should mention newLISP. Although it differs a bit from both CL and Scheme, it implements a lot of very useful functions. I find the http server, multi-processing functionality (Cilk) and remote evaluation functions very clean and easy to use.

The single executable is small, fast and includes a lot of cool batteries.

Answer 15

Another factor to consider in choosing which dialect of Lisp to learn may be the number and quality of implementations. If a dialect has only one or two implementations, then you're stuck with their flaws until those flaws get fixed, and there's always a chance that people will stop refining the implementation. In itself that's not a reason not to use a single-implementation dialect of Lisp, but it's something to take into account. An advantage of Scheme and Common Lisp is that there are many implementations of these dialects. Both languages have published standards, so relatively recent implementations are likely to run the same code, mostly. Some of these implementations have been around a long time, but are still under active development: They're now high-quality implementations, and they are still being refined. (For example, Steel Bank Common Lisp--SBCL, mentioned above--is extremely fast. I wonder how easy it would be to make JVM-based Lisp as fast.)