Facebook sharding by user_id or by photo_id?

Question

I'm reading a design suggestion for facebook/instagram social network in grokking the system interview (closed content :( so I'll describe the relevant part in my question).

They are speculating over how the sharding of the data is done, and according to that selection, how is the allocation of photo_id (story_id/content_id/whatever_id) being done:

1. Shard by user_id, and then use a simple auto-incrementing sequence for the photos of that user
2. Shard by photo_id. this requires some key generating service, as there's no single database for the user's photos.

They say that the reasons to prefer the more complex alternative (2), could be because option (1) can create

1. Non-uniform distribution of storage
2. Running out of space for a single user
3. Unavailability of all user's photos if a shard is down
4. High latency for a user if there's a high load

My question (finally): are these all real problems? Isn't the scale of facebook/instagram enough so that working in a granularity of a single user good enough to spread the load?

Answer 1

Engineering doesn't scale evenly. The problems faced when designing/building a single family home are different from those for a typical office building, and are different again for record breaking skyscrapers.

For "most" applications, you're right - shard by user is probably fine, and the problems you list would be rare.

However, Facebook operates on a completely different level where one-in-a-million errors occur often.

They would definitely not shard by user and probably not by photo (but I'm guessing here). If they sharded by user, how would they support celebrities with millions of followers?

Likewise for photos, a popular image (meme?) can be shared heavily enough to render 'sharding by photo' problematic.

To answer your question, the really big companies have to correctly solve these and other problems but smaller companies can safely ignore them.

On a side note - since this is an interview question, remember that the real question is "can you think about and communicate complex technical issues clearly?" - the actual details are of lesser importance.