I was once asked a system design question during an interview. I don't remember the details, but it was a fairly typical web app, so I described what I thought was a pretty common architecture (users making REST-style requests to the server, servers talking to the database, the database is partitioned and replicated for scalability, there are multiple databases around the world for faster access, there're multiple servers as well, with a load balancer in front of them, which does round robin or something smarter).
I failed that interview. My mistake was that I way overestimated the scale of the application. After multiple hints I was given, we agreed that all this complexity is unnecessary and (for the number of users I was given) the app would work just fine on just a single server with a local database. (Btw, I know that for a long time stackoverflow itself was hosted on a single machine).
My question is, how am I supposed to properly estimate the number of servers / database nodes for the solution?
My naive understanding is that I need to estimate the parameters of the application and then roughly divide the workload by the bandwidth of a typical server.
A "typical" server probably has:
- 256 GB of RAM
- 1-10 TB of disk space
- 10 Mb/S of network bandwidth
- The request time between the server and the database is 100-500ms
Does it sound reasonable?
As a concrete example, let's look on this article about designing a URL shortener.
After estimating the scale of the app, the decide that they need 15TB of storage for the database, 170GB of RAM for the cache, and the network bandwidth is about 10Mb/s.
Based on my estimates, this app could also be hosted on a single machine (maybe with a little extra vertical scaling). And yet they proceed to describing a "complex" architecture with a partitioned database, multiple servers, load balancers, etc.
If asked this question during the interview, would it be reasonable to suggest hosting this server on a single machine, since it's much cheaper and removes a lot of complexity from the solution? If not, how would you estimate the number of machines you need?