Scaling Up applications

Question

What should I consider when taking an application (web) from a few users to hundreds or thousands? Basically, what should I be considering when thinking about escalating?

Answer 1

In general, when we talk about scalability, it's more about going from few thousands to few millions or billions of requests. In such case, scalability requires an application to work on several servers, which has some very specific issues which scalability tends to solve.

In your case on the other hand, it seems that you just has to face the increase of users, without being forced to use several servers instead of one (unless the current server is pretty slow, in which case the upgrade must be considered, or the website has an impact on the resources of the server, in which case the bottlenecks have to be found and removed).

There are several points to consider in your case:

• Do you use cache enough? Don't forget that cache can be implemented on several levels, from caching database results to avoid doing the same queries again and again, to output caching to avoid using more CPU than needed, to client caching, to reduce the number of requests (for example avoiding the client to request the same CSS file again and again every time just to receive again and again the 304 response).

• Do you use SQL profiler? If not, it is a good idea to start using one to be sure that there are no repetitive queries, and to find what queries must be optimized.

• Do you use a profiler? Profiling your application may show where does it use too much memory or too much CPU. Working on those bottlenecks can strongly reduce the load.

• What is the weakest part of your server? If you see that RAM is used up to 20% while the CPU is used up to 95-99% all the time, you should probably upgrade your CPU, but don't need to buy more RAM.

• Do you use your server to the full extent? If you see that RAM is used up to 20%, why not implement more caching and use it up to 75 - 85%? If your application spends lots of time querying data from hard disk, why not buying a second hard disk and use it in RAID1 (if in your case, given your hardware and your disk usage, RAID1 can provide faster access)?

• How the server is used outside your web application? In other words, are there other processes which may use the resources, and if yes, do you need them and do you know what they're doing? It also applies to the scheduled processes. Running a backup at 3 A.M. when there are at most two-three users on your website is fine. Running the same backup at 6 P.M. when you observe a peak of visits on your website is clearly not a good idea.

Answer 2

The most important piece of your infrastructure to consider when scaling up (assuming you have a decently functioning software stack) is the physical layout. That is do you have multiple web servers, sitting behind load balancers? Do you have clustered app and db servers? Are you sitting on a dedicated network isolated from dev and qa? These are some of the things to focus on.

Answer 3

You can decrease the load of your database if you denormalize it, but you should benchmark it.

For Java you can decrease the stacksize using parameters to the JVM, because the amout of available memory may be a bottleneck if you have many concurrent users. It can also be interesting to change to an event-driven server e.g. Netty if your server uses much memory for servlets. Play Framework is an interesting Java web framework for Netty.

When using PHP you should use APC or Memcached for speedup.

You should also consider using Varnish for caching your requests.