Since my SO thread was closed i guess i can ask it here.

What practice or practices are good 90% of the time when working with threading with multiple cores? Personally all i have done was share immutable classes and pass (copy) data to a queue to the destine thread.

Note: This is for research and when i say 90% of the time i dont mean it is allowed to fail 10% of the time (thats ridiculous!) i mean 90% it is a good solution while the other 10% it is not so desirable due to implementation or efficiently reasons (or plainly another technique fits the problem domain a lot better).

6 Answers 6


The 90% solution I use is multi-processing. Seriously. Shared Nothing. Pass "immutable" objects from stdout on one process to stdin on the other process.

Make the OS handle it.

Since that API has been around for decades, it's very, very simple to implement.

  • Which API is that? I am aware of cases when A launches B, but not A and B hooking up their stdin and stdout voluntarily. How can this be done in Java/c# or Python?
    – Job
    Jan 26, 2011 at 21:08
  • @Job. OS pipes. python a.py | pyhon b.py
    – S.Lott
    Jan 26, 2011 at 21:15
  • Interestingly, the D programming language type system is designed to prevent sharing unless the programmer explicitly asks for it, without the overhead of multiple address spaces.
    – dsimcha
    Jan 26, 2011 at 21:16
  • @Job: This type of "API" is completely transparent to the programmer. I think he means the level of system calls, and the internal scheduling mechanism for the CPU of the OS.
    – sova
    Jan 26, 2011 at 21:17
  • @Job: "*nix platform". False. "one-way communication". True. "shell script must launch". False.
    – S.Lott
    Jan 26, 2011 at 21:26

What you are doing is already pretty good.

I note you did not mention a language or platform; but in general, a shared-nothing, message-passing environment will scale pretty well, so long as you don't make it too chatty.

If you can avoid synchronization other than perhaps on the head and tail of your queues, then you avoid 90% of problems that will occur.

Good Luck

  • I agree. Efficiently using many CPUS is, first and foremost, about removing coupling from your design. Or as you say, chattiness.
    – sova
    Jan 26, 2011 at 21:21

Devil's advocate: Ignore multi-threading until you have no alternative. I'm serious in this being the "90%" solution because of the effort to design, write, verify, and maintain multi-threaded functionality. Its not worth it in all but the most performance-hungry applications.

I know it doesn't provide the answer you like, but I felt it should be mentioned.


Let a thread only hold a single lock at a time. This will avoid all deadlock problems, but you just can't accomplish some tasks by doing so. Graceful shutdown seems like one place you must write code where a thread holds a lock, and wants to lock something else.

The rule of thumb for doing this would be to hold a lock for only the smallest number of lines of code possible. But you still have to think about it.


I do scientific computing and use multithreading primarily for parallelism rather than concurrency, so my 90% solution is to use extremely "local" parallelism constructs so that the multithreaded parts are small and easy to reason about. In other words, most of the program can be reasoned about serially as if it was single threaded, and multithreading only gets brought by, for example, using a parallel foreach in one middle loop in some performance critical part of my code.

Or, at the other extreme, multithreading can only be brought in at the very outermost loop, where the loop iterations are basically completely independent.


For business applications most of the time depend on the higher level framework to achieve effective use of multiple processors/core servers. E.g. Using ASP.Net you should not have to code explicit multithreading (except possibly sharing static caches) but rely on the ASP.Net framework to effectively use the available cores to serve the requests. Same can be applied to WCF services as well.
I would assume same for Java application servers etc.