50

I am starting to learn Scheme by the SICP videos, and I would like to move to Common Lisp next.

The language seems very interesting, and most of the people writings books on it advocate that it has unequaled expressive power. CL seems to have a decent standard library.

Why is not Lisp more widespread? If it is really that powerful, people should be using it all over, but instead it is nearly impossible to find, say, Lisp job advertisements.

I hope it is not just the parenthesis, as they are not a great problem after a little while.

  • 3
    xkcd2.heroku.com/297 – Michelle Tilley Mar 19 '11 at 22:31
  • 5
    Steel Bank (common) LISP is actually more popular than you might imagine. LISP like macro processors (e.g M4) are also widely used. You might not see it in your domain of choice, but it's still out there (for better or for worse). – Tim Post Mar 21 '11 at 2:50
  • 8
    The recent earthquake and tsunami wiped out the parenthesis factory in Japan. :-( – oosterwal Mar 22 '11 at 13:45
  • 1
    Which version of Lisp should we be using? Remember Lisp dialects are more or less incompatible. – user1249 Apr 7 '11 at 6:19
  • 2
    LISP (or dialects of ) are used all over the place eg AutoLISP is a dialect of LISP used by Autocad en.wikipedia.org/wiki/AutoLISP or EMACS en.wikipedia.org/wiki/Emacs_Lisp – Jaydee Feb 21 '12 at 16:57

11 Answers 11

69

Expressiveness isn't always a positive language trait in a corporate environment. Java is extremely popular partly because its easy to learn, easy to write, and easy to read. Mediocre programmers can still be very productive in Java, even if their code is wordy and inelegant.

Furthermore, it's easy to abuse expressive languages. An expert java programmer can refactor poorly written code quickly. The more expressive the language, the more difficult understanding and refactoring horrible code becomes. LISP macros are a good example. Macros are powerful tools in the right hands. In the wrong hands, they can result in confusing and hard to debug code.

LISP is a risky choice for senior management. If things go wrong, no one is going to blame management for picking a popular object oriented language backed by a major corporation like Oracle or Microsoft. Its much easier to hire programmers with experience in popular, easy to learn languages.

Even progressive companies willing to use a more powerful language usually don't choose LISP. This is because many of the newer languages try and compromise by borrowing powerful features from LISP, while staying easy to learn for the masses. Scala and Ruby follow this model. Bad programmers can pick them up quickly and keep writing the same mediocre code that they did in Java. Good programmers can take advantage of the more advanced features to write beautiful code.

Parentheses are not the problem. Haskell is an incredibly powerful and expressive language with a syntax similar to Python or Ruby and it hasn't been widely adopted for many of the same reasons as LISP.

Despite all this, I am hoping...

Clojure has a chance of becoming popular. It runs on the JVM, has great interop with Java, and makes concurrent programming much simpler. These are all important things to many companies.

*This is my perspective as a professional JVM programmer with experience in Java, Clojure, JRuby, and Scala.

  • 26
    The objection about the difficulty of finding qualified programmers is a bit of a dog chasing its tail. If Lisp was more widespread, it would be easier to find good programmers. – Andrea Mar 19 '11 at 23:17
  • 3
    @Andrea That is definitely true. Its harder to learn lisp though, which contributes to the problem as well. I realize that might be a controversial opinion, as many professors teach scheme as a first language. – dbyrne Mar 19 '11 at 23:24
  • 9
    Yes, APL is an excellent example of an expressive language. Also, a good example of why expressiveness isn't the most important thing ;) – dbyrne Mar 19 '11 at 23:54
  • 9
    I don't think this definition actually conveys what expressive means. Personally, before I call a language expressive, I would make sure that I am able to read what I have written. For example, using nontrivial logic in the loop initializer in C may save you some lines of code, but it is not readily understandable. On the other hand a list comprehension in Python can save a couple of line and be more readable. So you actually have found a way to express what you meant more concisely. If it is not readable, you have not find a way to express anything at all. – Andrea Mar 20 '11 at 0:17
  • 3
    I think maybe I am coming off as someone who doesn't like lisp. In truth, I love it. Clojure is an incredible language. I think its a very viable choice for a certain kind of company to use. I am just pointing out there are a lot of companies where it wouldn't make sense, and using something like Scala is a much more practical choice. – dbyrne Mar 20 '11 at 4:00
17

Why is not Lisp more widespread? If it is really that powerful, people should be using it all over,

If you believe that languages are chosen for their technical merits, you are in for a soul-crushing disappointment.

Such decisions are made based on Strippers And Steaks. Microsoft can afford them. Oracle can. Sun spent so much money hyping Java that they went bankrupt. Twice. A decentralized, heterogenous, volunteer community cannot compete with that.

but instead it is nearly impossible to find, say, Lisp job advertisements.

Funnily enough, the Lisp companies say exactly the opposite: they constantly have job openings but can't find enough people to fill them. (The same applies to Haskell, ML, O'Caml, Forth, Smalltalk, ...)

  • 3
    Where I live (Italy) I doubt even a single Lisp company exists... – Andrea Mar 19 '11 at 23:58
  • 1
    Which large companies are pushing Ruby, Python & JavaScript? JS is an admittedly special case, but the other two seem plenty popular without massive corporate/vendor backing – Ben Hughes Mar 20 '11 at 5:05
  • Yes, that's true for other languages as well. Most good COBOL coders already have jobs, and are not reading the ads anyway. Why bother advertising? – Bo Persson Mar 20 '11 at 13:14
  • What really? I'll code in any language thats not mainstream, although Haskell sounds way over my head. The small amount of code I've coded in it is extremely beatiful, but without a good mentor I have no idea when to use each Monad and the Applicative Functors. – aoeu256 Jul 1 at 19:24
9

I have no experience in working for a real company, but I know why LISP has been hard for me to use.

First of all, this reminds me of this blog post: http://steve-yegge.blogspot.com/2006/04/lisp-is-not-acceptable-lisp.html

The main problem that I have with Lisp is the "which Lisp" question. I usually work on Linux as my main platform, but the things I make need to be compatible with Windows. That means that when I'm evaluating a technology to use, it must make my life easy when working on two radically different operating systems. I don't like this requirement, but to use it on a real project that's a requirement. Now I'll use languages that don't have very good support on Windows for my own personal side projects, but because I never get a chance to write a large software project in them, I don't have the necessary experience.

Now when I was trying to learn a functional language, I really wanted to learn Common Lisp. It seemed like the right thing to do. I started reading Practical Common Lisp as a jumping off point since I really didn't know the built-in functions and needed a project to work on in Lisp. S-expressions were beautiful and easy. All those parenthesis were incredibly beatiful to me as it was clear as day exactly what was happening in the code.

So I try to write my first program in Lisp outside of the book. I wanted a command line tool that would count lines of code and remove trivial lines from the code count. Not the most useful tool, but fun to do. It involves file access, a bit of parsing, and counting. I had implemented the same tool in Python about a week earlier.

I need to access command line arguments. Then I learn there is no standard way to get command line arguments. They're all non-standard features. It's not cross-platform at all. It mostly just gets worse from there as the language doesn't have a lot of libraries built into it. I ended up switching to Haskell and didn't get very far into Common Lisp (so my complaints may not even be valid).

This kind of non-standard thing has always been a pain for me in the past. C++ has this same problem, but with libraries like Boost you can get around those weaknesses.

It also doesn't help that the Lisp syntax for everything other than S-expressions is a little ugly.

  • 1
    PLT Racket (formerly PLT Scheme) runs well on both Linux and Windows. – Larry Coleman Mar 22 '11 at 13:47
  • Interesting, I would expect Common Lisp to be much more standard and usable for real applications than Scheme is. (I am reading Practical Common Lisp right now.) – Giorgio Dec 12 '12 at 13:27
  • I commend you for choosing Haskell (which is a better language in my opinion), but what about Clojure? It runs on the JVM, so it should be portable and have access to all the libraries you need. – Andres F. Nov 25 '16 at 13:35
  • Command line arguments are difficult to use in C++? Really? – Nick Keighley Apr 10 '17 at 10:50
  • Isn't it trivial to build a cross-complier that changes Lisp to Scheme to Clojure? Why don't people use them? – aoeu256 Jul 1 at 19:26
8

IMO, it's mostly due to:

  • Poor library support. Sure, there's Quicklisp now, which makes it easy to install libraries, but it doesn't compensate for them being still fairly few, and quite a few are poorly documented or not very well maintained. Compared to Python, there's a good chance that writing non-trivial Lisp application (regardless of the particular dialect) will probably still involve reinventing at least part of a wheel or two.
  • Lack of familiarity with the model adopted by development tools. Most of the folks I know are fairly intimidated by having to use SLIME and Emacs, which is understandable for people used to Eclipse and Visual Studio. There are two Eclipse plugins I think, I only tried one of them a few years ago and it was pretty buggy. The whole Lisp ecosystem looks like it's stuck halfway between the days when it was part of a full-fledged Lisp-running system and the modern standard of oh-it's-another-language. Learning Lisp is not limited to learning a new language -- you also need to learn an entirely different way of working and thinking, and while this is okay if you're doing it as a hobby, it's questionable whether it's worth doing it in a business.

Things are starting to look slightly better though, especially with Clojure being around.

  • 1
    "Most of the folks I know are fairly intimidated by having to use SLIME and Emacs, which is understandable for people used to Eclipse and Visual Studio.": Again and again I get the feeling that Lisp is for elite. – Giorgio Oct 21 '16 at 19:25
  • 2
    "You also need to learn an entirely different way of working and thinking, and while this is okay if you're doing it as a hobby, it's questionable whether it's worth doing it in a business.": Isn't the increased productivity a good enough reason? – Giorgio Oct 21 '16 at 19:26
  • Has this "increased productivity" been demonstrated? It certainly isn't clear to me (yes I've programmed in a Lisp dialect). There Are No Silver Bullets. – Nick Keighley Apr 10 '17 at 10:46
  • @nickkeighley there was a study that showed that Lisp is at least less verbose when compared to Java: norvig.com/java-lisp.html . Whether or not that makes someone more productive, I don't know, but typically the difficulty of maintaining a system is the length of the program. There are also many things that you cannot do in a language such as Java without essentially creating a Lisp or Python interpreter without realizing you have done so. I think that's extremely relevant in web development back-end programming. – Simeon Ikudabo Nov 27 at 7:55
  • Proving something is less verbose than Java is setting the bar kind of low... – Nick Keighley Nov 29 at 13:40
5

I learned LISP a billion years ago in college.

LISP, like FORTH, is great for logic. But most programming is not about logic, it's about manipulating data in boring mechanical ways. For example, at the time there as no way to right-justify numeric output.

LISP is about nested functionality, and people just don't think that way. They think in terms of DO A, B, C, D, then E. Not do A, which involves doing B and C, then D and E. THis involves a kind of concurrency which is confusing. Except for predefined tasks like "file an income tax return", people don't think concurrently, they think sequentially. This is why procedural languages are dominant today.

As a result, produral code liks Java and C can be translated into English easily. LISP code can not; no human language is structured in that way.

So it's great for problem solving, but problem solving is not very important in programming. Data entry, validation, output formatting, all of which LISP was terribly weak at.

  • 8
    Problem solving IS all we do. Manipulating data is not easy in Java or C. Manipulating data in cons cells is much easier in Lisp and other functional languages. Most operations on data are the exact same and don't care what order you process them in. These are easily done with a call to map and a function pointer. I would also say it's easier to think in nested functionality than in procedural functionality (as one details your goal, and the other details how to perform said goal), but I have no proof for that. Either way, I would not say this is the reason LISP isn't used. – jsternberg Mar 22 '11 at 6:35
  • 5
    Programming is not about problem solving? I don't think so. And by the way I don't think that you can learn a language in college either. – Adam Arold Aug 2 '11 at 15:24
  • +1, sounds very plausible for me who don't know any Lisp. I would state the same reason C/Java is better than Python for enterprise-size solutions. – Jonas Byström Jul 8 '12 at 11:13
  • This is the only answer so far that raised the huge topic of functional vs procedural, but let's not forget procedural thinking tends to like tinkering with globs of data in vars everywhere you can touch from many places and threads creating synchronization problems. Functional doesn't suffer as quickly from this, well-written functional doesn't suffer at all. – maxpolk Nov 25 '16 at 15:32
  • 1
    A programmer once asked me what the best way to express a for loop using a a data-flow diagram. For such people there is no point to non-procedural languages. The think in FORTRAN. – Nick Keighley Apr 10 '17 at 10:53
5

I think one problem with Lisp not yet mentioned is that for a mediocre or novice programmer (like myself, I freely admit), it can be difficult to see how you turn Lisp code into a big program. It's easy to write but hard to architect. I don't think any of the concepts are particularly difficult, but the DIY mentality is so strong that I often feel at a loss where to even begin.

With an OOP language like Java or C#, you can use the type system to boost yourself towards a working model, and build off that. With Lisp (or Lua, or Javascript, for that matter) there's this notion that you can go out and do it any way you want. If you want OOP, just make your own OOP system! Except making your own OOP, or learning someone else's, is a new barrier on top of the language before you get usable programs. Plus I always feel like the OOP in Lisp or Lua isn't really there, like I could just ignore it if I really wanted, so what's the point?

In short, I think programming in Lisp requires a great deal of discipline, and that's very hard to come by. Strongly-typed languages and OOP languages both offer a kind of built in discipline, so the programmer can focus their limited reserves on finishing projects instead of tweaking the language.

EDIT: As an analogy that just struck me, it's like if you need to do some wood work and two people offer you their tool boxes. One person's tools are kind of crummy, but would basically get the job done with a bit of effort. The other person has a big bin of parts, but is promising that you can combine those parts to make the best tools you'll ever used, perfectly suited to your grip and of the best quality. You just have to build them first.

  • 2
    Common Lisp, at least, is explicitly an OO language: the Common Lisp Object System is part of the standard. – Frank Shearar Mar 22 '11 at 8:45
  • True, and as I understand it's a very powerful one, but it comes across to me as a side-feature of the language. It isn't like Java where you need to understand objects from day 1. Practical Common Lisp doesn't touch on CLOS until chapter 16. I may also just be biased towards static-typing, though I don't dislike dynamically-typed languages. – CodexArcanum Mar 22 '11 at 13:58
  • Lisp allows you to use closures (let-over-lambda) instead of objects for light-weight OOP, but I believe its easy to upgrade to use OOP via CLOS. – aoeu256 Jul 22 at 21:39
2

I've been wondering the same for long, and I even went to Lisp conferences to try to understand what is this Lisp "dark side" that keeps everyone from adopting it.

I've found no complete decent answer.

Ignorance may be the reason for the missing popularity, but what puzzles me more is that even who for sure knows about Lisp (for example Google - Peter Norvig works for them) is not using it.

The only partial explanation I come up with is that most of Lisp great ideas are now commonplace, the only really important missing one (a hugely important one IMO) is ease of metaprogramming.

Unfortunately I see no easy way to adsorb this concept for other languages because metaprogramming to be nice requires an homoiconic and regular language (I'm talking about general metaprogramming, not the dumbed-down template-only version). In other words it basically requires the Lisp approach to syntax: code is data and data is code. Writing code in a syntax-rich language that manipulates an AST is more difficult because you need to know two languages: how to write the code and how to write the AST. This is especially hard if your AST is fixed and also complex and irregular with a lot of different node types. Lisp instead has a reasonably regular (and extensible!) AST and you already normally code by directly writing the AST.

Metaprogramming is also inherently more difficult (and meta-metaprogramming even more and so on) and most programmers and managers apparently just prefer the "no one would need that" answer.

I find particularly sad that "new" languages like go ended up using text-based metaprogramming when is needed (externals code generators writing text files) and "magic" (i.e. the compiler can do what the programmers cannot do).

I think the solution to the complexity problem are powerful tools and education. The trend is apparently instead blunt tools and pretending the problem is not present.

  • +1 for "I think the solution to the complexity problem are powerful tools and education." – Giorgio Apr 9 '17 at 14:09
  • after 50 years the 'ignorance' excuse is wearing pretty thin – Nick Keighley Apr 10 '17 at 11:01
  • 1
    @NickKeighley: depends. Even today most programmers really don't know anything about Lisp (for example most of the times you hear Lisp described as a functional language). Even among who "knows" Lisp almost no one ever saw the macro idea with sufficient detail (I'm not talking about the dumbed-down version of scheme, but full algorithmic power with convenience of quasi-quoting when that fits better). – 6502 Apr 10 '17 at 13:46
  • @6502: While not being purely functional, IMO Lisp does support functional programming much better than many mainstream languages like C#, Java, C++, and so on. For many programmers, FP means just using a lambda expression from time to time: these programmers won't miss Lisp or Clojure or Haskell. So I would add: (1) Lisp does support FP pretty well and functional programmers would profit from it but (2) ignorance about FP and its benefits is one reason why Lisp is not used more often. – Giorgio Apr 19 '17 at 10:24
  • 1
    @6502: Having lists as basic data structure and the usual high-order functions on lists is also a feature that is missing in Javascript. And, as far as I can remember, Javascript also has the duality statement / expression. I would definitely place Lisp (Common Lisp) a few steps further in the direction of FP than Javascript or Python. By this I do not mean that Common Lisp is a purely functional language, I agree that it is multi-paradigm, but it supports FP much better than other multi-paradigm languages. – Giorgio Apr 19 '17 at 11:21
1

It seems that even CL does not have very good library support. At least according to people who switched from Lisp to Python:

http://www.redmountainsw.com/wordpress/archives/reddit-switches-from-lisp-to-python

Personally, I know some Scheme and enjoy playing with it, but can't imagine doing a non-trivial project in that language.

  • 2
    This is no more true. Quicklisp solved that problem. – user20645 Mar 19 '11 at 23:04
  • 2
    It's also worth mentioning that with CFFI, any C library can be used from Common Lisp. CFFI is well-documented and works well. On the other hand, if spending a few hours writing wrappers for C functions has a major impact on your project, Lisp probably isn't the right choice for it anyway. – Larry Coleman Mar 21 '11 at 13:06
  • I did a non-trivial project in Scheme and know a company that uses Scheme (Chicken Scheme) for several products. – Giorgio Apr 20 '17 at 7:40
1

Being powerful doesn't necessarily imply widespread use. Have you heard of the term 'Optimize for the common case'? Unfortunately as many have told before mediocrity if assured consistently is a lot better for people than big hits with many failures in between them.

This isn't just the case with lisp, but with a lot of technologies. A good touch over Unix text processing tools, awk, sed, perl can save you days of programming. Unfortunately I have seen people take days do these sort of tasks in other tools badly what could have done with these tools more efficiently in minutes. But if one spends all his life in eclipse he will never come to appreciate the value of these things. You may write a huge program with readability and maintainability and all that, but whats the whole point of writing such a program while not writing it could have easily done the job.

Another aspect while designing tools these days how much useful out of the box they are to use them directly to solve a problem. You can't build something too generic and then then say you will hedge all of that through libraries and frameworks. Its a little difficult to work things out that way.A good tool gels well with the environment and problem around it. That's why tools like php,perl and awk continue to remain relevant despite endless trolling and bashing, because they are too useful to throw them away and often they do a lot of work than a general language with many libraries and frameworks bolted on.

Similarly you will see languages like Java/Python are very good and I would say best for certain tasks. Python especially is very good and easy to learn and write. Also these languages do really good if your data end points are standard. Some sort of a database or a XML or data of that sort. Basically structured data.

Lisp will be there around for long time, but not necessarily widespread. As you will see each tool has its niche. And they solve certain set of problems very well out of the box. I think and I'm sure lisp is doing good in areas and problems for which its designed to solve well.

  • +1 for citing the "Optimize for the common case" principle. – Giorgio Dec 12 '12 at 13:30
  • Yeah but LISP's Closures are an optimization for the common case of Objects. Also I was wondering if anybody makes changes to their LISP code base via treating it as a tree and running queries on it like JQuery for HTML. – aoeu256 Jul 2 at 13:01
1

For web development with Lisp dialects, there can be a bit of a chicken-and-egg problem - because few people use Lisp, hosts either don't allow it or don't make it easy, and because it's not easy, few people use it. But actually, getting Lisp running on a host can be easier than you might think, even if it requires a bit more work than their out-of-the-box PHP service does. I recently got guile Scheme apps working here with just a little effort.

0

IMO, it's mostly a matter of poor timing: Lisp was old (and almost by definition no longer exciting) long before it became practical for most people or uses. Clojure (for one example) stands a much better chance. Its success will depend far more on being perceived as new, fashionable and cool than anything as practical as interoperation with Java (and everything else that runs on the JVM) though.

Not the answer you're looking for? Browse other questions tagged or ask your own question.