I'm reading a design suggestion for facebook/instagram social network in grokking the system interview (closed content :( so I'll describe the relevant part in my question).
They are speculating over how the sharding of the data is done, and according to that selection, how is the allocation of
story_id/content_id/whatever_id) being done:
- Shard by
user_id, and then use a simple auto-incrementing sequence for the photos of that user
- Shard by
photo_id. this requires some key generating service, as there's no single database for the user's photos.
They say that the reasons to prefer the more complex alternative (2), could be because option (1) can create
- Non-uniform distribution of storage
- Running out of space for a single user
- Unavailability of all user's photos if a shard is down
- High latency for a user if there's a high load
My question (finally): are these all real problems? Isn't the scale of facebook/instagram enough so that working in a granularity of a single user good enough to spread the load?