I am trying to create a very basic tool which would execute a script, retrieve the data and then write it into CSV. This needs to be done across 5000+ database servers. This can definitely be improved if it can be done in parallel. I was reading about Threads, ThreadPools and TPL and still confused which one should I be using. Any help would be much appreciated.

P.S. The number of parallel threads has to be configured within the tool

closed as too broad by Doc Brown, gnat, BobDalgleish, Jörg W Mittag, Bart van Ingen Schenau Feb 11 at 12:53

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 3
    Hi Theepan, questions on performance are seldom really answerable in a sensible manner by outsiders like us who do not operate the real system. I recommend you try out one of the approaches you mentioned with a small subset of the servers, measure how well it works, and when you have a real issue, come back and ask again (maybe on stackoverflow, if it is a coding issue). In C#, I would start with a simple Parallel.ForEach. Use Google to find plenty of examples, and how to control the number of parallel threads. – Doc Brown Feb 5 at 7:36
  • Since you have tagged this as C#, use async await and leave it to the compiler and runtime to optimise it around threads for you. Your use case is exactly what async await is for. – David Arno Feb 5 at 8:22