Start small - but design in such way that sharding is possible - how?

Question

Following question is more about best-practices than a real problem - nevertheless, I'd like to know how to do it in best way.

Given a service, that can operate in multiple countries/geo-areas, one probably start simple before even scaling is needed. A design would contain a single DB and single piece of infrastructure. An API endpoint(s) would look like this:

/.../v1/items?geo_area=xyz&page=1&size=100

Now imagine that service grew a lot, and there is a need to create separate piece of infrastructure per each country/geo-area where our service operates. Would you do:

Option 1)

Stay with above API format, and route to shards based on queryString param from api url?

Option 2)

Create new API endpoints that have country/geo-area in url, e.g. /.../xyz/v1/items&page=1&size=100

Option 3)

Put country/geo-area to server part of url, e.g. https://xyz.mydomain.com/api/v1/items&page=1&size=100

I see that Option 1) has a pros of not breaking contracts. But I'm not sure that routing based on queryString is good idea at all.

Option 2) a 3) breaks previous API contracts (clients that uses it must update) and it forces clients to react on server's infrastructure changes, which is also a design-smell in my opinion.

Designing for sharding from very beginning is also not an option, as you don't know if you ever need it.

Answer 1

Option 3 lets you do something the others can't, which I'm going to call "hard sharding": requests are distributed to different servers in different datacenters based entirely on the client's request. You can be sure that requests to abc.example.com aren't going to xyz.example.com. This may be useful if for legal reasons you have to keep the shards separate.

It also lets you market the different URLs more easily, just as Amazon have "Amazon.de" vs "Amazon.co.uk" and so on. Or you can "white label" it and have a "geo area" that in fact represents a different company rather than country.

The other two need a proxy in the middle to route the requests to the right part of the infrastructure.

(Conversely, if you're using HTTPs, then it's completely hidden which geo-area the client is requesting. If you're not, then it's possible for third parties to observe it going to different IP addresses. This usually doesn't matter but may be worth mentioning.)

Answer 2

It doesn't matter. All of these solutions will allow you to scale in the future. Start with a simple design, and maybe later add sharding behind the scenes, without requiring API clients to do anything.

The important insight is that your API servers can be scaled separately from your databases. E.g. you can have separate databases for different geographic regions, and your API servers just connects to the appropriate database depending on the parameters in the request. For this, it doesn't matter where the region is indicated, whether in the domain, path, query, or even in the request body.

If you want to scale in the sense that requests from different geographic regions are handled by different API servers, then it's usually not a good idea to force the API client to select a close server, such as by having to connect to a particular domain name. This way, a client could easily circumvent your sharding, whether for malicious or accidental reasons. Instead, you would likely use a CDN that routes request to nearby servers of yours. Using network-based routing (e.g. to provide different content to clients in different countries) gets especially important when the reasons for such separation is not technical but done for legal or marketing reasons.