This is a general question but I work in the .NET world so I'd like to know if there any specific quirks about the .NET Framework / Core platforms that I should be concerned about here.
I think it's safe to say that as a general rule, breaking long-running jobs in to threads could be slightly faster than breaking jobs in long-running processes. However, the amount of performance difference would be negligible. I haven't done any benchmarks on this, but this answer seems to indicate that there is an consensus on the matter at least as far as .NET is concerned.
However, I still hear the argument "It was broken up in to separate processes for performance reasons" often. I'm puzzled by this and I wonder if this argument ever holds weight. Please remember, this discussion has nothing to do with maintainability. The question I am asking: are multiple jobs running on a single process each necessarily faster than multiple jobs running on a single thread each inside one process?
Intuitively, I would have to guess that the answer is no. For example, 10 long-running jobs running on 10 threads should run with roughly the same performance as 10 long-running jobs on 10 different processes. But, I see people making the design choice of breaking services up in to smaller parts purely for the purpose of performance.
What I suspect is going on is that shoddy code has created scenarios where starvation is occurring. I think that what is happening is that parallelism is being overused and availability of CPUs is not being honoured so that the benefits of multiple cores are being eroded because the jobs are fighting for CPU power. Still, this doesn't really explain why breaking a job out in to separate processes would improve performance.
So, how can I prove that either a) performance is not improved by breaking jobs out in to separate processes, or b) the improvements made by breaking the code out in to separate processes is only because of poor code design in the first place?