I keep seeing in job postings that the applicant must have experience writing "scalable" applications. What makes an application scalable, and how do I know that my code can scale to millions of users?
I guess a better way of phrasing this question is: How can I write my code with scalability in mind? So that the code is scalable from the get-go as opposed to an afterthought. Are there certain design methodologies? Or is it simply a matter of picking the correct algorithms for the job?
There are two directions of scalability:
For the first one, you just have to take care that you do not have any arbitrary limitations. These either because of too small integer sizes or fixed/limited length structures. These structures might be related to underlying OS. For example if you try to scale up using more threads or processes, at some point you're going to reach OS's limits. That's why currently servers build for high-scalability are doing concurrency based on asynchronous events. This problem is described in famous C10K document.
Second one is more difficult. It requires programming with two things in mind: data will be processed in parallel, and data might be physically distributed. The communication between the nodes should be limited. In practice that usually means sacrificing some parts of ACID (it is proven that you cannot have full ACID and ability to scale-out at same time). The most known solution for data storage in that paradigm are NoSQL solutions. They range from very simple key-value stores, to systems RDBMS-like, only stripped of ability to do joins. The key-value stores are ultra-scalable, but that comes as a price. You can basically query only on primary key. There is however solution to that, it's map reduce. It might seem very suboptimal if you look at cumulative complexity point of view, but you have to keep in mind, that it's running massively parallel.
If you want to read more about scalability with real life examples, check out HighScalability.com blog.
Scalability is measured in terms of throughput based on some variable. For example, number of requests/second with X users. The simplest way to describe scalability is:
A measure of efficiency as load increases.
The first thing you need to understand in designing for scalability is what measurement is most important for your application? There are several ways of measuring efficiency which is a key component of scalability:
There are more efficiency measurements that can be used, but these are common for web based systems or batch processing systems.
The next aspect of scalability is measuring what happens to your efficiency as load is increased. Common ways for load to increase are:
The goal for a scalable application is to either maintain or improve efficiency as we deal with the load problem. In short, if the response time is taking too long, can we add another server to distribute the load evenly? This approach reduces the amount of work for one server to do, and keeps the servers operating in that "sweet spot" for efficiency.
You're application will need to be designed specifically to scale. That means you have to be careful with session data, routing requests to the right server, reducing bottlenecks that limit the ability for the application to scale.
You basically want to avoid performance bottlenecks when you increase the number of users, and/or process a larger data set, and/or offer your interface in more languages, etc.
You basically take a look at your database schema, your algorithms, and your software development process and try to predict future problems. You also want to setup performance monitoring to identify problems when they start building up.
I picked up these tips when I read Building Scalable Web Sites (link to amazon).
Hope this helps!
The only way that applications can be truly scalable, is by not having any restrictions which cannot be passed (or only very expensively).
A typical example is what happens when you run out of available cpu-cycles? If your program is multi-treaded you can run on a box with multiple cores but what happens when you cannot buy a bigger box anymore? Your application simply cannot grow anymore, and hence is not scalable.
Any truly scalable application must be able to spread over multiple computers in a transparent fashion and do so without any noticeable bumps. THis is not easy, and it is one of the reasons why Google has been so successful.
There are unique problems that come with supporting large scaled applications. The job posting is looking for applicants that have worked in that environment and had to solve such problems.
From a high level application's are made scalable by constantly asking the question what would happen if this piece of code was requested to be run thousands of times in a very small period. This means managing your memory footprints, making use of caching of totals and data, using data sources that are scalable themselves,etc.
If you were building a search feature that performed well when it has 100 rows in the DB to search and 10 users using it at a time. How well would it perform when 100 users were using it at the same time and there is 100K rows to look up.
If it performs the same no matter what then its very good. it if performs proportional to the amount of users/data (meaning 10x more data == 10x longer to process) thats good. If it performs much lower the more data it has (10x mode data == 10x^10 longer to process) then it does not scale well.
My examples should really be shown in Big O notation but I currently do not know it well enough to write out the examples in Big O.
You can simulate more data by dumping dummy data into your DB, and there are tools to simulate more users such as Apache AB.
