1

If I have a system where my distributed service is sending live scores of 1000 s of football games from some hypothetical event to millions of clients. The service subscribes to the games from a source service that publishes the scores. The source publish happens over the ID of the game, i.e. each game is published on a different topic that my service subscribes to.

In this situation, I need my service to have instances where each instance handles a set of games, i.e. I do not want all instances to handle all games. How could this be accomplished? If I use a microservice based architecture, it is supposed to be stateless so that won't work, right (unless I separate them out as different microservices per set of games they handle)? What alternatives are there?

3
  • 1
    What is the meaning of "stateless" here? And why microservices are supposed to be stateless? Either way just group those ids and run an instance per group.
    – freakish
    Commented Jul 4 at 17:54
  • I think you're conflating conceptual (micro) services (which may or may not have state) with physical (micro) service instances that involve duplication of instance for scale out, which require some cooperation, a strategy for partitioning (here the game id space), increasing/decreasing scale dynamically, etc.. You could diagram the whole scale out set of services as a single micro service at a conceptual level.
    – Erik Eidt
    Commented Jul 4 at 19:34
  • @ErikEidt you are right. I was thinking from a Kubernetes deployment point of view.
    – Ufder
    Commented Jul 6 at 0:47

2 Answers 2

1

This setup does not sound like it is well architected.

each game is published on a different topic

Consider revisiting that design decision.

But let's go with premise. Introduce the notion of a shard. Choose some convenient hash, such as SHA3. I will illustrate with crc32(). We will have a handful of servers picking up the load. Select a convenient NUM_SHARDS such as 10; it should be greater than max number of servers.

The list of all games is available. Define a game --> shard function:
shard = crc32(game_id) % NUM_SHARDS

Server events like scale-out and crashes will change the number of active servers. Each time there is such an event, rebalance their shard assignments. So when you bring up the very first server, it claims all shards, and issues thousands of Kafka consumer.subscribe(game_id) calls. When the second server comes up, it claims a portion of the shards, perhaps 5 .. 9, and then the first server unsubscribes from the same set of games. Notice that this is a racy situation, so you get to work out locking or more likely de-dup details. When third or fourth server comes up, they each assume responsibility for one or more shards, with a previous server relinquishing them. In general you might not see exactly equal load balancing across servers as the deployed system evolves.


Now, how could we make such handoffs go more smoothly?

Teach the publisher about the notion of shards, and have it create just MAX_SHARDS topics, with multiple games muxed onto each topic. Now, instead of thousands of .subscribe() calls, each server can make just a handful of such calls.

Or if you retain the current very large number of topics, at least use a topic prefix of the shard number, so consumers can take advantage of pattern matching.

3
  • I see what you mean: if the publisher can use a {gameid}_{shard_num} to publish (where shard_num goes from 0 to MAX_SHARDS), then each shard that comes up will only consume from the resp. topic it is responsible for. Is my understanding right? This would be better than using zookeeper or something to coordinate the config between different servers. So anytime something changes, and I want scale up or down, I will have to update the topic names accordingly.
    – Ufder
    Commented Jul 6 at 22:49
  • Also, in practice how would a service where the service instances that cater to specific topics only handled? Would I use a something like statefulset where I can put logic based on the ordinal number? Or would be more like multiple microservices?
    – Ufder
    Commented Jul 6 at 22:54
  • 1
    "Also, in practice how would a service where the service instances that cater to specific topics only handled? Would I use a something like statefulset where I can put logic based on the ordinal number? Or would be more like multiple microservices?" I have no idea how to even begin to parse these questions. The OP described a terrible architectural setup which I attempted to salvage. The publisher should prefer topic names of {shard}_{game_id}, for the convenience of lexical ordering. The "hard part" has been pushed down from "game_id -> server" to "shard -> server", but it's still hard.
    – J_H
    Commented Jul 6 at 23:04
-2

Make each set of games resulting data a different API Endpoint. example endpoints: api/westcoast/gamedata vs api/eastcoast/gamedata. Use a proxy to consume the endpoint from the appropriate server handling those games. Could be done with HAProxy or NGINX. They would just point to different microservices based on the URI https://domain.tld/api/eastcoast/ vs https://domain.tld/api/westcoast/ for example.

1
  • 1
    As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.
    – Community Bot
    Commented Jul 6 at 6:27

Not the answer you're looking for? Browse other questions tagged or ask your own question.