I know a little about how to use message passing and shared memory, but I still don't know when to choose one or another.
Can you tell me about their traits (not the implementation)?
Software Engineering Stack Exchange is a question and answer site for professionals, academics, and students working within the systems development life cycle. It only takes a minute to sign up.Sign up to join this community
The quick summary:
Shared memory parallelism (e.g., OpenMP) is strictly more powerful than message passing (e.g., MPI) as you can simulate message passing with a shared memory system but not vice versa. However, it's also much more difficult to scale up the shared memory model (it needs very elaborate and expensive hardware once you go to more than one system board) and it is also much more difficult to analyze due to the number of possible interactions. The message passing model is considerably simpler to understand — you just have a stream of messages coming in that you want to process — and it maps much more nicely to what is actually possible with networks so it obviously can scale up massively.
From the point of view of how you actually design systems with them, they are in some sense mathematical duals, as this adage by Joe Armstrong, one of the original designers of Erlang, nicely demonstrates:
With shared memory, you communicate by sharing data, with messaging, you share data by communicating.