I've been dabbling in code for many years now but it's always been personal projects. I don't have experience with much traffic. I was recently reading through a "Hello, World." tutorial for Dropwizard and at the bottom it states "Well, congratulations. You’ve got a Hello World application ready for production (except for the lack of tests) that’s capable of doing 30,000-50,000 requests per second."

Although what I can only imagine as being a good problem to have, what do you have to start doing at that point? If my application was built upon Dropwizard, would I have to start migrating to something different, or is that where deploying to multiple "nodes" and load balancing comes in?

  • 3
    In production you don't want any server to be anywhere near the throughput that it is capable under "normal" load. You also most likely want multiple servers behind a load balancer to deal with equipment failures and maintenance. – Gort the Robot Mar 28 '15 at 20:05
  • 1
    And in the unlikely event that multiple servers plus a load balancer isn't enough, you set up a full-blown content delivery network. – Ixrec Mar 28 '15 at 20:40

That's generally where horizontal scalability (distributed computing/multiple nodes and load balancing) comes into play. Every system is going to have some kind of upper limit on vertical scalability.

One critical key to horizontal scalability is state management. You want to design your software so that individual sessions don't require affinity to a specific node. The less work the load balancer has to do to keep utilization even across the nodes, the better. And if you're putting the burden of state management onto a back-end server (maybe a database), you need to be aware of that resource becoming a bottleneck despite your load balancer. So perhaps you look into database federation or table sharding (best to use a framework instead of manual sharding, in my opinion. For instance, how do you decide the criteria for key splitting? How do you adjust that when you discover that your assumptions about the key distribution were wrong?)

  • 1
    The good news is that modern enterprise-class databases are very good at this. When properly installed and configured they can perform about as well as theoretically possible for a given hardware configuration, and support clustering for situations where a single server is not sufficient for the load. – user22815 Mar 29 '15 at 2:02
  • @Snowman Exactly... ;-) I've encountered situations before where people went to great lengths, burning lots of development hours, to develop their own custom sharding mechanism before they even needed that sort of thing. They of course split their keys on the first character of the primary key. So they had giant "E" and "N" tables and a little itty bitty "X" table, plus it forced developers to craft their data access code to the custom sharding implementation, so it crept all the way into the middle tier. They put the cart before the horse, and re-invented the wheel, in mediocre fashion. – Craig Mar 29 '15 at 3:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.