5

This is a general question but I work in the .NET world so I'd like to know if there any specific quirks about the .NET Framework / Core platforms that I should be concerned about here.

I think it's safe to say that as a general rule, breaking long-running jobs in to threads could be slightly faster than breaking jobs in long-running processes. However, the amount of performance difference would be negligible. I haven't done any benchmarks on this, but this answer seems to indicate that there is an consensus on the matter at least as far as .NET is concerned.

However, I still hear the argument "It was broken up in to separate processes for performance reasons" often. I'm puzzled by this and I wonder if this argument ever holds weight. Please remember, this discussion has nothing to do with maintainability. The question I am asking: are multiple jobs running on a single process each necessarily faster than multiple jobs running on a single thread each inside one process?

Intuitively, I would have to guess that the answer is no. For example, 10 long-running jobs running on 10 threads should run with roughly the same performance as 10 long-running jobs on 10 different processes. But, I see people making the design choice of breaking services up in to smaller parts purely for the purpose of performance.

What I suspect is going on is that shoddy code has created scenarios where starvation is occurring. I think that what is happening is that parallelism is being overused and availability of CPUs is not being honoured so that the benefits of multiple cores are being eroded because the jobs are fighting for CPU power. Still, this doesn't really explain why breaking a job out in to separate processes would improve performance.

So, how can I prove that either a) performance is not improved by breaking jobs out in to separate processes, or b) the improvements made by breaking the code out in to separate processes is only because of poor code design in the first place?

13
  • Aren't they breaking the services into small processes because of some vague promise of and independent release life cycle? Oh, and containers. Edit: perhaps I should link 10 Tips for failing badly at Microservices by David Schmitz. – Theraot Feb 20 '20 at 3:35
  • No. They claim it's for performance reasons. – Christian Findlay Feb 20 '20 at 3:59
  • 1
    Does this answer your question? Multiprocessing vs multithreading – Christophe Feb 20 '20 at 7:54
  • 1
    Nope. That question doesn't have an answer, but again, I'm not asking about what is best. I'm just asking of it's possible that it's possible that breaking jobs out in to separate processes could actually be faster than breaking them in to separate threads. – Christian Findlay Feb 20 '20 at 8:02
  • 1
    If you are only focussing on performance with this question, and "this discussion has nothing to do with maintainability", then maybe the title should reflect that. It's always weird if the answer to the title is "yes" but to the body "no" – Mark Feb 21 '20 at 9:56
6

as a general rule, breaking long-running jobs in to threads could be slightly faster than breaking jobs in long-running processes

There was a time when I believed this as well. But I have first-hand, real-world experience that this can be very wrong - specifically with .NET. I have observed several scenarios where the extra isolation by processes gained a speed-up by a factor almost equivalent to the number of CPUs, whilst thread parallelization only gained a speedup of a factor 2 at maximum. Though I did not always understood the reasons behind this in full, I can tell you two (possible) causes for such behaviour:

  • Shared memory pool between all threads: not sure how well the garbage collector handles the objects created by different threads, but I am pretty sure this was one of the reasons where the speedup was far below the factor we expected

  • Usage of third-party libs which support multithreading, but do not support it very efficiently. An example for this is ADODB for accessing MS Access databases. If one uses thread parallelization to open one connection per thread, each one connecting to a different database file, it seems the different threads disturb each other - still working correctly, but way slower than one could expect.

Hence my recommendation is to try it out: if there is a task to solve where you would expect thread parallelization to bring you a speed factor of "N", where N = "no of CPUs", but you only observe a factor much smaller than N, then try "process parallelization". Of course, this makes only sense if the extra effort for implementing this is acceptable.

I would not be astonished if you make the same observation as me.

7
  • 1
    This is a fair call and the reason why I ask this question. I'm more or less trying to avoid literally benchmarking the difference between threads vs. processes. I could do that, but it would be a huge time sucker. This gives me food for thought, because no matter the principle says, empirical evidence is the only way to prove something true or false. It doesn't leave me in a good spot though. – Christian Findlay Feb 21 '20 at 2:54
  • @MelbourneDeveloper: You can always start to measure the speed improvement by multiple threads compared to a single thread and find out if it scales linearly, or not. If it scales as expected, then using processes cannot bring any benefit. If not, then you have to estimate the effort of creating a "by-process" solution and to analyse if it is actually worth the effort. I think there is no easy way around this. – Doc Brown Feb 21 '20 at 5:31
  • The only way I'd be able to do that would be to start merging existing projects together. Even then, I'd be benchmarking in an unrealistic scenario. – Christian Findlay Feb 21 '20 at 5:33
  • The second anecdote reminds me of COM STA (COM apartment threading model), MTA, COM marshaling... good old Microsoft technology (sarcasm) – rwong Feb 21 '20 at 15:10
  • While your answer could imply that there's a fundamental flaw in how .NET handles threading, I think one possibility is that developers over-communicate between threads and create a lot of contention. Inter-process communication tends to be more structured and the costs more obvious. – JimmyJames Feb 21 '20 at 17:07
5

I think you are misconstruing people's motivation for using processes. In general, people don't choose processes for speed, they choose processes for the extra isolation, then measure the speed to make sure they aren't sacrificing too much speed.

There are some languages where you usually need processes to gain true parallelism. JavaScript is famously single-threaded, and Python has the Global interpreter lock that will cause serialization within a single process. Also, separate processes are obviously required to be able to scale across multiple physical nodes. In general, though, threads are preferred for speed, and even beyond that, lightweight threads/fibers/greenlets/goroutines/whatever are going to be faster than threads for many use cases.

2
  • I had cases in the past where the extra isolation of processes helped to scale up the performance almost linear with the number of CPUs, which was not possible by using just threads. – Doc Brown Feb 20 '20 at 22:15
  • I don't know how many times I have to repeat what I've been told. I haven't misconstrued anything. I have been told that the applications were broken up for performance reasons - not isolation. But, I do understand your point. – Christian Findlay Feb 20 '20 at 23:32
3

There is not a real performance benefit for using processes over threads.

The real reason why you hear this is a tiny “white lie”. Thing is: writing multithreaded software is hard and it can get real complex real fast. You have to deal with race conditions, locking, asynchronous calls and a lot more, which make it very error-prone.

So you do the poor man’s multithreading: multiple processes of sequentially coded programs. You still have to deal with some race conditions but all in all, it becomes magnitudes easier to code.

But then when someone ask you why you chose processes over threads, are you going to say: “I’m too bad of a programmer to use threading” or mutter something about performance?

4
  • 5
    Using processes because they are harder to get wrong isn't a bad thing. Falling into the pit of success – Caleth Feb 20 '20 at 13:58
  • @Caleth true that. – Pieter B Feb 20 '20 at 15:43
  • 1
    @Caleth Actually it's also easy to get wrong too. There are a few famous examples (well known big corps, both Seattle and Bay Area) where when big corps implement process isolation, they got so many deadlocks (or what looks like so) in shipped products that the instability haunted their customer base for several years, tanking their reputations. After that they learned how to do this correctly. It was a workforce-wide educational issue, and inadequately covered by the CS education. Now these products look very stable. But they also became very memory hungry too... – rwong Feb 21 '20 at 2:19
  • I think you may have hit the nail on the head here @Pieter-B – Christian Findlay Feb 21 '20 at 2:48
0

As far as I remember it depends on the OS. If the OS supports only ULT (user level threads) and if your process will do a lot of blocking I/O on different ULTs then it will probably be faster to split it into multiple processes and let the OS schedule them so that every time one of the processes block the OS will schedule one of the other processes and the overall performance will be better, especially in a multiprocessor environment.

-1

There is positive proof there is no difference: Windows only runs threads. Windows does not "run" processes. Processes are a security boundary and memory is managed per-process.

3
  • Even if this would be true (which is not, Windows runs several service host processes in parallel, and other processes as well), it would not proof anything for .Net processes. – Doc Brown Feb 20 '20 at 22:10
  • Processes are not a runnable unit in Windows. I didn't invent this, you can check the Windows Internals book. – Martin K Feb 20 '20 at 22:15
  • That's not my point. – Doc Brown Feb 20 '20 at 22:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.