Implicit parallelism^ can take a big burden away from many programmers, placing it on the computer. So... why is it is not more widespread at present?

^ Implicit parallelism is to make a computer be able to figure out itself how to do more than one thing at a time, instead of a programmer needing to do this job using threads and the like


5 Answers 5


Because with a few exceptions (Haskell) there is no way that the compiler can unwrap a loop. The problem is that each iteration through the loop can modify global state. So doing it in a different order may cause things to break. In haskell you can count on a function being pure, which is to say it does not read or change global state, so they can be executed in any order.

The real issue is that with a few exceptions how to do concurrency well is still very much an open problem. The Erlang and Haskell communities seem to be doing pretty well but its still a long way to go before we really understand how to program a N-core system for large N.

  • 1
    In Scheme, there are some loops that explicitly choose not to guarantee order.
    – Javier
    May 9, 2011 at 14:11
  • Cool I did not not know that about scheme
    – Zachary K
    May 9, 2011 at 16:41

Most of the programming languages which we are using now came at the time where single threaded programming and single user interaction is the most used for many applications(ex: stand alone desktop applications). With the raise of web applications, cloud computing and multi user applications now we need more of multi threaded applications.

The legacy programming languages are trying to support multi threaded features from language itself slowly (Like the java added java.util.concurrent).

New languages which will come in future will have better in built threading and concurrency support.


Apart from the points mentioned in the other answers (hard to prove that operations are independent, and programmers think serially), there is a third factor that needs to be considered: the cost of parallelization.

The truth is, that thread parallelism has very significant costs associated with it:

  • Thread creation is very expensive: To the Kernel, starting a thread is just about the same as starting a process. I'm not sure about the precise costs, but I believe it's in the order of ten microseconds.

  • Thread communication via mutexes is expensive: Usually, this requires a system call on each side, possibly putting a thread to sleep and waking it up again, which produces latency as well as cold caches and flushed TLBs. On average, taking and releasing a mutex costs around one microsecond.

So far, so good. Why is this a problem for implicit parallelism? Because implicit parallelism is easiest to prove on the small scales. It is one thing to prove that two iterations of a simple loop are independent of each other, it is a whole different thing to prove that printing something to stdout and sending a query to a database are independent of each other and can be executed in parallel (the database process could be on the other side of the pipe!).

That is, the implicit parallelism that a computer program can prove is likely unexploitable because the costs of parallelization are greater than the advantage of parallel processing. On the other hand, the big scale parallelism that can really accelerate an application is not provable for a compiler. Just think about how much work a CPU can do within a microsecond. Now, if the parallelization is supposed to be faster than the serial program, the parallel program must be able to keep all CPUs busy for several microseconds between two mutex calls. That requires really coarse grained parallelism, which is almost impossible to prove automatically.

Finally, no rule without an exception: Exploitation of implicit parallelism works where no threads are involved, which is the case with vectorization of the code (using SIMD instruction sets like AVX, Altivec, etc.). That indeed works best for the small scale parallelism that's relatively easy to prove.


Programmers think serially, and current languages are built to support that model. With the exception of the fringe languages such as Haskell Erlang etc, languages (I refrain from using the adjective "modern") are essentially high level assembly where we still tell the computer what to do, when to do it and how to do it. Until we have an echo-system where we tell the computer what result we want is avalible, we don't have the mental capacity, as programmers, to make full use of multithreading capablilty.

i.e. it's not natural......

  • Programmers think how they were taught to think, just as they program in the way their programming language encourages them to program. If a programmer doesn't expose themselves to your so-called fringe languages, they will never learn that there are other ways of reasoning about problems. I think this is why Seven Languages in Seven Weeks is high on many peoples recommended list.
    – Mark Booth
    May 9, 2011 at 16:16
  • Perhap fringe was the wrong word - I meant languages not widely used in commercial applications (i.e. not C++ or Java).
    – mattnz
    May 9, 2011 at 21:04
  • 1
    However I stand by my assertion (with nothing but my own opinion to back it it), that programmers, being people, do not, have the metal cpacaity to really "get" multithreading and massive parrallisum. It's not human nature to do more than one task at the same time. Every book on time management essential promotes the concepts of starting something, finish it then do the next thing, because thats how we are wired. To efficently and effectively use these paradigms, we need massive tool support, which is currently not avalible. The few do "get" it, and need to develop these tools.
    – mattnz
    May 9, 2011 at 21:12
  • 1
    I think don't have the patience is a more accurate assessment than don't have the mental capacity though. Over my career, I've seen many more lazy programmers than I've seen stupid ones. I was lucky though, I was taught functional programming and fine-grained parallel programming along side procedural and O-O, in my first year at university. I suspect many programmers weren't so lucky and their thought processes have been straight-jacketed as a result.
    – Mark Booth
    May 11, 2011 at 15:16

Transactions must be ACID, so, programmer mainly tends to think about one thread.

Languages and platforms must protect programmer from concurrency as much as they can afford

And concurrency is not as easy to test as funcionality itself, so programmers tends to leave besides these issues and, even, not thinking about commit concurrency handling, what is a mistake

Not the answer you're looking for? Browse other questions tagged or ask your own question.