-2

I was once asked a system design question during an interview. I don't remember the details, but it was a fairly typical web app, so I described what I thought was a pretty common architecture (users making REST-style requests to the server, servers talking to the database, the database is partitioned and replicated for scalability, there are multiple databases around the world for faster access, there're multiple servers as well, with a load balancer in front of them, which does round robin or something smarter).

I failed that interview. My mistake was that I way overestimated the scale of the application. After multiple hints I was given, we agreed that all this complexity is unnecessary and (for the number of users I was given) the app would work just fine on just a single server with a local database. (Btw, I know that for a long time stackoverflow itself was hosted on a single machine).

My question is, how am I supposed to properly estimate the number of servers / database nodes for the solution?

My naive understanding is that I need to estimate the parameters of the application and then roughly divide the workload by the bandwidth of a typical server.

A "typical" server probably has:

  • 256 GB of RAM
  • 1-10 TB of disk space
  • 10 Mb/S of network bandwidth
  • The request time between the server and the database is 100-500ms

Does it sound reasonable?

As a concrete example, let's look on this article about designing a URL shortener.

After estimating the scale of the app, the decide that they need 15TB of storage for the database, 170GB of RAM for the cache, and the network bandwidth is about 10Mb/s.

Based on my estimates, this app could also be hosted on a single machine (maybe with a little extra vertical scaling). And yet they proceed to describing a "complex" architecture with a partitioned database, multiple servers, load balancers, etc.

If asked this question during the interview, would it be reasonable to suggest hosting this server on a single machine, since it's much cheaper and removes a lot of complexity from the solution? If not, how would you estimate the number of machines you need?

closed as primarily opinion-based by amon, Greg Burghardt, gnat, BobDalgleish, Doc Brown Jan 16 at 21:41

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 2
    You made no mention of the application, or how many users would conceivably use it. You're approaching the problem from the wrong direction. Different applications have different reliability requirements, different usage expectations, etc. – whatsisname Dec 30 '18 at 22:56
  • @whatsisname I linked a very concrete application as an example (url shortener), along with its expected number of users and usage patterns – Valentin Dec 30 '18 at 23:28
  • 1
    An url shortener is a terrible example application. – whatsisname Dec 30 '18 at 23:32
  • Ten Megabit per second sounds completely unreasonable. – gnasher729 Dec 31 '18 at 9:11
  • @gnasher729 what number would you consider reasonable? – Valentin Dec 31 '18 at 19:38
3

Particularly in an interview, they aren't looking for solutions, but how you observe, and how you solve problems.

They probably wanted you to demonstrate that you are keeping the business interests in mind, and are willing to engage in that discussion to find out the businesses real needs, and help them manage real risks.

If they presented you with some data about the systems performance, perhaps they would like to see how you do back of the envelop calculations to narrow down on a suitable solution.

So the first question should be: what are the operational expectations?

These need to be precise statements about how the system operates within a given period of time. Something like:

  • between the hours of 8pm and 2am 95% of requests for links created in the last 2 weeks are responded to within 100ms.
  • The first 1000 requests every 10 minutes for links created more than two weeks ago take no longer than 500ms to complete.
  • The system will be available for querying 99% of the time.
  • Should the system fail, all Links that were acknowledged as created must be available when the system is restored.
  • The system should be restored within 1 hour of failing.

Each of these statements helps to clarify exactly what you need in order to run this system. At the very least they can be setup as reports/tests that verify that the system is operating within expectation.

The second question to ask is: what are the limitations?

Not every business is Google, or Face Book. Not every business needs or can afford shiny gadgets. As such they will usually already have infrastructure that they will prefer to leverage, be it in house or through a provider. Some of those limitations will be hard, others will be negotiable if the outcome is worth it.

  • xyz database engine must be used
  • Backups must be made daily
  • Backups must be moved offsite within 1 hour of being made
  • only the xyz machines can be used.
  • data must be stored in a Raid

Solving

Now you have a set of Requirements and of Limitations, time to brain storm possibilities and check them for relevance. Rank each possible implementation by its cost (material + time) in both implementation and operation along with how likely it will satisfy the requirements, and whether it violates the limitations.

  • Pick the cheapest to create
  • Pick the cheapest to maintain
  • Pick the easiest to expand
  • Pick the easiest to shrink
  • Pick the easiest to change
  • Pick the one that manages to Fit within all the limitations regardless of how difficult it is to maintain.
  • Pick the cleanest solution that only breaks a few of the (softer) limitations.

If these are the same solution, keep thinking. If not great, go forth and discuss. Let the business know whether the solution limits their future options or not, whether it costs more to operate, or if there is some other form of risk such as lack of technical expertise.

Handwavium Specs

As to how to spec out each solution, that does indeed depend a lot on the specific technologies at play. You can do some handwavium projections based on some of the limitations and requirements to get a feel for the spectrum. Like perhaps there is a rule that the software, database, and backups must be on separate machines. Or a Geographically redundant system needs at least 4 machines in two data-centres.

Only go in for exact requirements when you have some data around the software's actual response times. For widely used systems like data stores and caches, the publically available reference data + a margin should be sufficient for back of the envelope style calculations. Even better if you can trial an actual setup, even if using under-powered hardware.

Be clear what the base values for your calculations are, and show a projection for +-10%/20% to demonstrate how the requirements might change due to a change in the referenced rates. The point here is to address the risk of under/over estimation and see how that impacts the end solution.

Not the answer you're looking for? Browse other questions tagged or ask your own question.