How to scale a C# server application

Question

I'm writing a pretty simple TCP server/client application for the first time. It is a personal project for education, but I really like my applications to be extendable and to scale in case the code is good enough to use somewhere else.

I am using Microsofts async TCP server/client as my starting point https://msdn.microsoft.com/en-us/library/fx6588te(v=vs.110).aspx

My end goal is to be able to have some kind of ServerManager application that the client first connects to, and is then redirected based on that servers capacity.

My question is: Do the clients get redirected to a different physical server, or does the ServerManager spawn another instance of TCPServer on the same machine and connect them there (not even sure how that would work)?

Answer 1

Well it kind of depends on what resources you already have. For instance if the application uses all the cores on the computer, simply spinning up another instance of the application won't necessarily mean that requests get handled faster because the two applications would be sharing the resources on the server.

The important thing to remember here is that generally it doesn't make much sense to spin up another instance of the application which would end up using the same resources as the first instance (for example if they try to access the same hard drive at the same time, the operating system has to keep switching access to the hard drive between the two applications).

Most multi-server environments use load balancers to automatically distribute requests between physical machines (see the link in JeffO's answer or https://azure.microsoft.com/en-gb/documentation/articles/load-balancer-overview/) so one of the things you have to keep in mind when programming for scalability, is that one client might not always have every request handled by the same server.