7

What thought process should one follow to convert a sequential algorithm to a parallel one ? Are there any specific code pattern that can be parallelized. Some patterns that I normally use are:

  • Looking for loops that can parallelized.
  • Try to break algo in some form of Map-reduce format.
  • Looking for producer-consumer pattern.
Improve this question
3
  • 2
    This is a more general question, and better suited for programmers.se.
    – David Thornley
    Commented Dec 7, 2011 at 18:58
  • 2
    Look for divide-and-conquer, or something like quick-sort where it breaks the problem in half recursively. In that case, every time it splits it in two, and each half isn't affected by the other half, you could parallalize there.
    – Andrew Rasmussen
    Commented Dec 7, 2011 at 19:02
  • 5
    Look for dependencies. Dependencies imply serialization. Non-dependencies imply parallelization.
    – Jim Clay
    Commented Dec 7, 2011 at 19:08

2 Answers 2

Reset to default
7

Ian Foster describes four basic steps in his book "Designing and Building Parallel Programs":

  1. Partitioning - find possible ways to split the data among the workers as fine-grain as possible.
  2. Communication - identify the communication patterns.
  3. Agglomeration - reduce the initial partitions to coarse-grain tasks according to the resources available.
  4. Mapping - map the tasks onto the processing units.

The book is free online, you can find it here: http://www.mcs.anl.gov/~itf/dbpp/text/book.html

Improve this answer
1

To somewhat formalize what you have already said about looking for loops that can be parallelized, the key concept being exploited is flat data parallelism. A majority of the parallel libraries (that I know of) seem to exhibit this (e.g. Microsoft Task Parallel library for .NET, Microsoft Parallel Patterns Library for C++)

Flat data parallelism happens when you have a chunk of (flat) data (e.g. an array) that you want to perform work on, you then divide up the data into chunks of how many processors you have, does independent work in parallel for each of those chunk, and finally potentially aggregate/combine the result of each chunk to get your final result.

A sample article on this: http://goose.ycp.edu/~dhovemey/spring2011/cs365/lecture/lecture18.html

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.