I'm currently working on a project which will have client applications (desktop & android & web) communicate with varies processes running on a server(s).

My question is how much should I be thinking about load balancing and the like?

All projects which I have worked on before consisted of creating applications for internal use within companies so the number of connections was alwyas very low (probably no more than 50 at a time).

However in my own time I decided to start creating a piece of accounting software, just because. So what I have ended up with is a WPF desktop application, and ASP.NET MVC web site and an Android App which all connect to various processes/data running on my server.

On the server there is an SQL database (MS), various WCF services for communication from client to server and a few services running cleanup stuff.

This works fine when I'm testing and its just me connected. However I have no idea how it will scale if a few people start connecting at once.

I'm not even sure if I will release the software, let alone anyone use it! However as part of a learning exercise how should I be developing the system so that it could take on a large load and not crash/delay.

I'm aware this could be a very open question and could be subjective to a lot of things, but as I say I have no experience in this department and it's something I would like to learn.

4 Answers 4


However I have no idea how it will scale if a few people start connecting at once.

Premature optimization is the root of all evil. If it works, don't fix it, and if you are curious if it scales then don't lose sleep over it and just test it.

It doesn't matter if you'll ever release the software, there are numerous tools that can help you run automated load and stress tests, simulating thousands or even millions of connections. It really is as simple as that, if your tests show there's a problem, fix it, if not, move on.

  • Yeah I have started running some "tests" (though I think it would be a sham to call them that) of just launching a number of instance of the desktop application on a few pc's so I suppose I can do this in an automated sense to increase the load. I just got to thinking what must people like Google do for apps like Google Docs. Commented Apr 4, 2013 at 12:41
  • @PetePetersen I occasionally use loadimpact.com (but it's not free). I've heard great things for JMeter if you'd like to experiment with an open source solution at first.
    – yannis
    Commented Apr 4, 2013 at 12:45
  • Thanks Yannis, I'll look into this. If, for arguments sake, there is a problem under load my initial thought was to have all clients hit a web server which then routes the calls to a particular server. So essentially the services will be running in many places at once. Is this a legitimate strategy? (I'm aware it may not be needed but from a purely educational perspective) Commented Apr 4, 2013 at 12:51
  • @PetePetersen If your application has a comparable number of users to Google Docs then that is what is known as a Good Problem To Have. :)
    – maple_shaft
    Commented Apr 4, 2013 at 12:51
  • @PetePetersen Sounds like a solid strategy, but it also highly depends on the exact bottlenecks you've identified in your app, so again don't look for solutions until you have a very clear idea of what the problem is. For educational purposes you can take a look at Apache's mod_proxy_balancer, a commonly used load balancer. Even if you don't use Apache the docs should give you a good general idea of how everything fits together.
    – yannis
    Commented Apr 4, 2013 at 13:05

As Yannis said, premature optimization is the root of all evil.

However, some things like security and scalability can be really hard to fix 'later'.

In your case I think a good compromise is to not build scalability in, but keep it in mind and try to avoid fundamental design decisions which will make it impossible to add later:

  • statelessness, REST flavors are good
  • disposable clients and servers are good
  • long lived job threads can be problematic
  • anything relying on global synchronization/serialization or only one actor processing something can be problematic

Also generally speaking, app tiers are easier to scale horizontally than a database - so when balancing work between them more work on app server is good.

  • +1 for app servers being easier to scale horizontally than database servers.
    – maple_shaft
    Commented Apr 5, 2013 at 11:24

Scaling a web application is typically a deployment concern, assuming that proper software development practices have been followed in development of the web application (Eg. Not storing user data in statics or application/server contexts.)

Most web servers, like Apache, can be configured to act as a load balancer to multiple application servers by delegating requests for specific contexts. Web servers can themselves be load balanced by sitting behind a content switch that is configured to delegate incoming requests.

A number of application servers in many different technologies can also be configured to have session sharing, automatic server farm deployment and more. One hiccup that you might run into could be scheduled tasks as unless you have implemented task scheduling with concurrency in mind then each server will run and do its own thing which could be bad. Again many application servers have task scheduling features built in so that you won't have to implement that inside of your application directly.

Most major databases have a number of load balancing options and possible configurations. It could be as simple as configuring one server for a cold failover, or something complex like multiple database server nodes, each handling their own user requests, query plans and database connections in a concurrent way against a common fileshare.

The file share containing the data can also be concurrent and have failover by mounting against something as simple as a RAID array of disk drives or something expensive and complex like a SAN (Storage Area Network).

And if all this isn't enough and you don't have money or access to a data center for the best toys... there is always a cloud service provider that basically gives you infinite scalability.

  • Thanks maple_shaft. This gives me a lot to think about. I am expecting it to be a deployment issue as I have developed quite stringently to certain guidelines. I think you're right that the cleanup stuff I'm doing at the moment would have to go/be changed quite a bit, but that's not really a big issue. Thanks Commented Apr 4, 2013 at 13:22

As far as "scaling by load-balancer" goes, make sure you know what properties your load-balancing will need. If your session-management relies on hitting the same application server each time, you need sticky sessions.

If you scale your front-ends, how do your back-ends (storage, databases, ...) scale?

It's hard to give a hard and fast answer, though.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.