Concurrent programming is quite difficult to me: even looking at a basic slide seems challenging to me. It seems so abstract.
What are the benefits to knowing Concurrent programming concepts well? Will it help me in regular, sequential programming? I know there is a satisfaction to understanding how our programs work, but what else?
Here's a quick and easy motivation: If you want to code for anything but the smallest, weakest systems, you will be writing concurrent code.
Want to write for the cloud? Compute instances in the cloud are small. You don't get big ones, you get lots of small ones. Suddenly your little web app is a concurrent app. If you designed it well, you can just toss in more servers as you gain customers. Else you have to learn how while your instance has its load average pegged.
OK, you want to write desktop apps? Everything has a dual-or-more-core-CPU. Except the least expensive machines. And people with the least expensive machines probably aren't going to fork over for your expensive software, are they?
Maybe you want to do mobile development? Hey, the iPhone 4S has a dual-core CPU. The rest won't be far behind.
Video games? Xbox 360 is a multi-CPU system, and Sony's PS3 is essentially a multi-core system.
You just can't get away from concurrent programming unless you are working on tiny, simple problems.
2016 update: The current iteration of the $35 Raspberry Pi is built around a quad-core system on a chip intended for cell phones. Dramatic advances in AI have been made in part due to the availability of high-end graphics cards as parallel compute engines.
Everything has a dual-or-more-core-CPU. Except the least expensive machines. seems a little preposterous. Lots of people have single-core machines, not because it was cheap, but because they are happy with what they have and see no need to upgrade. That said, thinking in terms of concurrency will help the scheduler on a single-core system as well, so it is not wasted effort anywhere you can assume preemptive multitasking, either (which is about every multitasking environment most developers will come in contact with, these days).
From 1970 to about 2002 processors doubled in speed about every 18 months. So as a programmer all you had to do was wait and your program would go faster. The problem is that around 2002 the rules changed. Now they are not making bigger fast processors they are making smaller slower processors but putting them out in groups. The computer I am working on now has 4 cores, and Chips with up to 8 cores (and 4 threads per core) exist. Soon enough we will have chips with a lot more cores.
So if you write a program that is not at all concurrent you will find that you are using 1 core or thread, but the rest of the CPU is sitting there doing nothing. So if you have 16 cores 1 will be running your program and the other 15 are sitting there!
The problem with concurrency is that it is non deterministic. Which is to say that you don't know exactly what order different threads will do things in. Traditionally programmers have tried to solve this by using locks and the like. This has lead to a LOT of pain. Having some form of mutable state that more than one thread can access freely is often a formula for pain and heisnebugs!
Of late the trend has be to moving to functional languages which tightly control mutable state. There are two basic ways that functional languages handle concurrency. The first is by using message passing. This is best shown by Erlang. In Erlang there is in general no Shared state between processes. They communicate not by sharing memory but my passing messages. This should make sense to you as we are doing it right now. I am sending this information to you by sending you a message, not by you remembering it out of my brain! By switching to message passing most of the locking bugs simply go away. In addition messages can be passed over the network as well as within one node.
The other method is STM, which stands for Software Transcriptional Memory, This is present in clojure and Haskell (and others). In STM memory is shared but changes can only be made via a transaction. As the Database folks figured all this stuff out in the 1970's it is pretty easy to ensure that we get it right.
Actually I over simplified a bit, Clojure and Haskell can both do message passing, and Erlang can do STM.
Disclaimer I am the author of Programming Web Services with Erlang, which will be out in Early release in the next few weeks.
Because concurrency can blow up in your face when you expect it the least...
The first rule of concurrent programming is "It's difficult". The second rule of concurrent programming is "It. Is. Difficult"..!!
Seriously though, there are two common approaches to concurrent programming, multi-threading and multi-processing. Multi-processing is the easiest to comprehend since it just means having multiple instances of a process running to accomplish a task. The is pretty easy to do on Unix based systems via calls to fork/join, but not so easy on Windows systems.
Multi-threading is probably the approach most people think of when talking about concurrency. It's not difficult to start multiple threads within an application, but the devil is in the details. You need to co-ordinate data sharing between thread (usually using locks) which can lead to deadlock or data in an invalid state. You also need to understand how to communicate between thread using concepts like semaphores, conditional variables etc etc.
The advantage to all this is that once you understand it you're able to more effectively utilize the underlying hardware. These days it's pretty much the norm for a processor to have multiple cores. By utilizing concurrent programming you can make these cores work for you, and your application will get a speed improvement.
The disadvantage is that you have to start thinking about how you'll split your application up into small parts that can be run on different threads. This is a lot harder than it sounds. Also, highly concurrent solutions can be awkward to unit test as the order of execution is less deterministic.
These days most languages ship with an abstraction over most concurrent primitives to make life a bit easier. For example, .NET 4 ships with the Task Parallel Library which make life a bit easier. In Java land they've got the Concurrency package.
I recently had a very interesting task to do in which multiprocessing saved me. I basically had to do a lot of requests to a few separate servers, dealing with very small amounts of data, but many requests.
Working with PHP, I did things the old fashioned way, and the best time I obtained after a few hours of work resulted in ~120 seconds to run a certain test (many requests + network delay + no async)
But that wasn't nearly enough compared to what I needed, and after failing miserably with PHPs multiprocessing, I switched to Python.
After a few hours, I had a running Python multiprocessing script that ran in 20 seconds, and after a bit of fiddling with the timeouts and no. of threads to be used, I got it down to ~10 seconds.
This was for a website written 100% in PHP, except a single, 100 line Python script. And the whole thing is working perfect.
My conclusion would be that even if it won't help you on a day to day basis, you may encounter situations where knowing at least the basics of concurrent programming will help you greatly.
Good luck, and happy coding!
PS: I'm not trying to bash PHP, but PHP simply wasn't the right tool for the job at hand.
PS2: Knowing a new technology, or a new way of doing things can open the door to a whole new world of possibilities.
If you do any sort of web development, concurrency comes into play, at least with most languages. For example, I use spring for web development and each new request comes in as its own thread. Therefore, if any request ends up accessing a shared object, where the state can be changed of a variable, concurrency is very much a factor and has to be taken into consideration. If its not, then the data can be edited in unpredictable ways and data corruption can result. Its not critical to know every last detail about concurrency but learning pieces at a time is important to better understand web app programming, if you are working on desktop apps, maybe its not so important unless you need to run multiple threads
Learn the insight of operating systems. Reading the source code of schedulers and device drivers will help; they are definitely concurrent.
