1

I am starting to learn parallel programming in c++, and I have a program like this:

for i = 1:N
    do something time and memory intensive
end

If I program this in parallel, and my processors have shared memory access, will the parallel programming help? I thought maybe I am going to overflow the memory, since I am letting 15 processors run a memory intensive task at the same time, which may make it slower, but I could not really find out if this was indeed the case.

  • 1
    I guess you will have an I/O bottleneck due to use of paged memory. Also the benefit of parallelisation will generally decrease due to limited memory bandwidth. – Den Sep 8 '16 at 10:09
  • When you say "shared memory access", that throws up a red flag right there. If your iterative tasks are sharing the same memory and any of them are writing to it, trying to parallelize this is a very bad idea. – Mason Wheeler Sep 8 '16 at 12:11
  • 1
    To get a useful answer, please show us what things you have tried so far. Have you used a memory usage profiler? Do you know how much memory your program uses, for each value of N (i.e. average and peak memory usage as a function of N)? How about total execution time as a function of N? – rwong Sep 8 '16 at 12:29
1

None of us can tell you whether it will overflow memory, because we don't know what your software is, or what platform it will run on. On most modern operating systems, exceding the available RAM will cause swapping/paging, which causes a massive performance hit.

But even if you don't run out of memory, then excessive multithreading can still cause performance issues.

  • If you have more active threads than you have CPU cores, then the operating system will keep switching between them. The time spent context switching won't be spent doing useful work.
  • Modern processors use cache memory on chip to work around the slowness of RAM. If your threads use a lot of memory, then every time you context switch to a different thread, the processor will find that its cache is useless - it's full of data that was being processed by the previous thread. This will require more reading of data from RAM, and will slow things down.
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.