1

Recently I was reviewing my team leader pull request in our Web API hosted by Kestrel. There was a place in our code which is sort of hot path and we are processing frequently over hundred of items sequentially on each request. So I suggested him to process this items in parallel instead of sequentially to speed up the response time of our API, but then he replied with:

In web apps in general we shouldn't do computational stuff in parallel, because the web app is inherently parallel (handles more than one request at a time). If this was a desktop app, or the work was IO - yes for sure.

I know I haven't written a lot of web services and such I am more interested in runtimes and operating systems, but is he right?

His main argument was that if we do parallel processing we will take resources that might otherwise be used by Kestrel for serving requests, although I am not sure if Kestrel is using user level threads or kernel level threads (if that matters.)

  • Unless you have a lot of compute cores to spread the work around, parallel processing may even be slower than sequential due to the overhead of thread swapping. You might look into offloading the computation to another machine, or if it is something vector math can handle a GPU. – Berin Loritsch Jul 5 '18 at 16:21
  • Kestral is not using kernel threads: docs.microsoft.com/en-us/aspnet/core/fundamentals/servers/… – Berin Loritsch Jul 5 '18 at 16:24
2

He's right.

Presumably, all of the processor cores on the web server are already devoted to handling user requests. If you process your hot path in parallel on the web server, the most likely outcome is that performance will not improve unless the web server is lightly loaded. You might even see a slight performance degradation under heavy load.

One way to solve the problem is to offload the parallel processing onto another computer.

  • Depending on the nature of processing, you might be able to offload it to a GPU which is able to apply the same math in parallel on a chunk of data. In either case it's no longer on the CPU that is handling requests. – Berin Loritsch Jul 5 '18 at 16:25
  • I see, however I dont see the CPU being loaded even under heavy load of my service. Requests start to fail before I see CPU loaded to up to 90-100% in fact it only goes to 30-40% but I guess thats question to Kestrel guys. – kuskmen Jul 5 '18 at 16:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.