1

Recently I was reviewing my team leader pull request in our Web API hosted by Kestrel. There was a place in our code which is sort of hot path and we are processing frequently over hundred of items sequentially on each request. So I suggested him to process this items in parallel instead of sequentially to speed up the response time of our API, but then he replied with:

In web apps in general we shouldn't do computational stuff in parallel, because the web app is inherently parallel (handles more than one request at a time). If this was a desktop app, or the work was IO - yes for sure.

I know I haven't written a lot of web services and such I am more interested in runtimes and operating systems, but is he right?

His main argument was that if we do parallel processing we will take resources that might otherwise be used by Kestrel for serving requests, although I am not sure if Kestrel is using user level threads or kernel level threads (if that matters.)

Improve this question
2
  • Unless you have a lot of compute cores to spread the work around, parallel processing may even be slower than sequential due to the overhead of thread swapping. You might look into offloading the computation to another machine, or if it is something vector math can handle a GPU.
    – Berin Loritsch
    Commented Jul 5, 2018 at 16:21
  • Kestral is not using kernel threads: docs.microsoft.com/en-us/aspnet/core/fundamentals/servers/…
    – Berin Loritsch
    Commented Jul 5, 2018 at 16:24

1 Answer 1

Reset to default
2

He's right.

Presumably, all of the processor cores on the web server are already devoted to handling user requests. If you process your hot path in parallel on the web server, the most likely outcome is that performance will not improve unless the web server is lightly loaded. You might even see a slight performance degradation under heavy load.

One way to solve the problem is to offload the parallel processing onto another computer.

Improve this answer
2
  • Depending on the nature of processing, you might be able to offload it to a GPU which is able to apply the same math in parallel on a chunk of data. In either case it's no longer on the CPU that is handling requests.
    – Berin Loritsch
    Commented Jul 5, 2018 at 16:25
  • I see, however I dont see the CPU being loaded even under heavy load of my service. Requests start to fail before I see CPU loaded to up to 90-100% in fact it only goes to 30-40% but I guess thats question to Kestrel guys.
    – kuskmen
    Commented Jul 5, 2018 at 16:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.