What should I do to scale out a high-traffic website?

Question

What best practices should be undertaken for a website that needs to "scale out" to handle capacity? This is especially relevant now that people are considering the cloud, but may be missing out on the fundamentals.

I'm interested in hearing about anything you consider a best practice from development-level tasks, to infrastructure, to management.

Answer 1

Design for Concurrency

That is, as you're coding, plan around having multiple threads going. Plan the shared state (often just the db). Plan for multiple processes. Plan for physical distribution.

This allows you to distribute your system across multiple machines, and across multiple processes with load balancing. It allows you to have redundant processes running in case of failure, and in case you need to modify the system in-place, you don't have to kill all service to do so.

Answer 2

A few things you might consider:

• Separating read- and write-sides of your data storage.
  
  • CQRS/Event Sourcing
  • CQS
  • Message-passing/Actors
• Avoiding shared process and thread state
  • Hence avoiding locking
  • You can avoid this through the type system by creating your classes, structs and other data types to be immutable, i.e. non-changing after construction. Especially for complex abstract data types, it works surprisingly well (e.g. jQuery's implementation)
• Not blocking web server threads on IO. If you are using ASP.Net use asynchronous pages/actions with the APM pattern/task-parallel library (TPL)
• Not saving loads of state in the user-session dictionary
  • This has to be moved across threads when thread migrations occur in IIS.
  • Having intelligent routing, such that non-secured/static resources aren't served with the same application framework (e.g. ASP.Net) that add overhead. Look at having different web servers, for example.
• Writing continuation-passing code with an asynchronous workflow-pattern (e.g. bind (haskell)/callcc/Tasks.ContinueWith/F#'s async)
• Use queueing theory to calculate where your bottlenecks may happen
• Use push- rather than pull-based updates to read-models and other application state. E.g. through RabbitMQ/nServiceBus
• Use the least-features applicable 'http handler'
• For static files, serve e-tags and cache expiry policies to enable the web infrastructure to work as it should (e.g. with squid proxy)
• (Hire me to solve your scaling issues and get on-site tutorials ;))

Answer 3

Share Nothing architecture.

With that in mind, and contrary to what you might think, don't jump to a scale-out solution right away. The off-system overhead vs. an in-system call should not be under-weighed. For instance, it takes a LOT longer to make an DB connection across any network interface than it does to make a local call. Budget how much time in management, power, and tuning effort is needed in scale-out vs. the extra $ for a true large system.

Regardless, I there is still great value in "share nothing" architectures and you can layer and scale-out your systems when the time comes.

Answer 4

Parallelize requests over several hostnames

Part of the HTTP standard is a section that says webclients will request a maximum of 2 sessions per DNS Host. Here is a solution where you and alias out your www.domain.com and get a higher request concurrency, making your page load faster:

https://stackoverflow.com/questions/3653609/how-do-i-code-my-asp-net-page-to-parallelize-downloads-across-hostnames

Basically it involves editing your ASP.NET HTTP Handler to alternate the target hosts you send clients to, where each host is a CNAME to "www".

Answer 5

Secure, Fast, Reliable DNS

I found a few high-capacity websites using the registrar's DNS server, which had no SLA for uptime or performance. In addition, their servers were located in India and the latency alone increases the chance that a DNS spoofer could poison your customer's, or intermediate ISP's cache. This would cause even your SSL-protected traffic to be redirected without anyone knowing.

DNS speed also affects the initial load time of your server, before the records are cached.

I use DynDNS or Neustar to most of my customers since they have a pretty solid DNS infrastructure (though it's expensive and I have no other affiliation to those companies).

Answer 6

I think that the key is going to be simple:

Have simple code. That means something that you look at and understand. As you expand and change servers you need to know what is going on. You also might need to add coders who need to quickly understand. Hooks and XML files that call random code that is not obvious is very bad.

Then you can test and find the problems.

Look here: http://blog.servint.net/2013/08/27/going-big-how-to-scale-a-website-part-1-infrastructure-that-scales/

We at stellarbuild try to make sure our websites scale without down time. That means you need to be able to know what your code does and where it does it. Even if you are testing a different machine you can't take too long to scale. Most people only start when it is almost too late, sadly. You can optimize only once you do that in my opinion.