Suppose you were asked to design a scalable chat server with the following requirements:

  1. The main use case is: player A sees B online, A sends a message to B, B receives it.
  2. The secondary use case is: player A sees B offline, A sends a message to B. When B comes back online, B receives the message. (No push notifications).
  3. The goal is to minimize latency. Speed matters.
  4. The messages should arrive in order. We cannot lose messages but receiving duplicates once in a while is fine.
  5. Just text data, no binary data.
  6. No need to store chat history. Once sent, the messages can be destroyed.

I've been reading this article: How League Of Legends Scaled Chat To 70 Million Players and I think I missed the core architecture they used in the game. But anyway here is my "thought process". Can someone have a look at it?

  • If the secondary use case didn't exist, I wouldn't have to store anything. I think I could use a p2p network, wherein a user regularly sends a ping message "i'm online" to all his friends to notify of presence.
  • But since I have to store messages to be able to deliver them later, I need my own servers that store user presence, user friendship lists, and messages.
  • The goal of minimized latency can be achieved by placing servers close to the users. This means that there will be more than one server so they need to stay in sync. Also, we need to load balance them so that one server does not store everything.
  • I've read somewhere on the Internet that a way to load balance the servers is to assign a server to each user. So for example server 1 gets assigned everything related to user A, and server 2 gets assigned everything related to user B. We could decide this by proximity.
  • When A sends something to B, there has to be a way to dispatch the message to server 2. Maybe use a service bus to communicate servers.
  • So the flow would be something like this:

    1. A writes message "Hi B!"
    2. Server 1 receives message and B. Since it does not find B in his user base, he forwards the message to the service bus. He stores a copy of the message.
    3. The service bus requests all servers to look for user B.
    4. Server 2 replies that he has B in his user base.
    5. Server 2 receives the message and stores it.
    6. Server 2 sends message to user B.
    7. Server 2 signals to the service bus that the message was sent. He destroys the message.
    8. Server 1 destroys his copy of the message.
  • If B were offline, everything up to step 5 would stay the same. The difference is that server 1 can destroy his copy of the message but server 2 cannot.

  • Now, storage... My guess is that each server should have their own persistent storage, but I've no idea what should be optimized here (speed of reads? speed of writes?). Also I'm not sure if a MySQL store or a NoSQL store would be better. Since NoSQL is optimized to be partitioned and there's no need here I guess MySQL would be enough.
  • If a server crashes we need a way to failover quickly. I suppose we could place like a "primary" and "secondary" server in each location, the primary would be connected to primary storage and the secondary to replicated data.

So the overall architecture would look like this:


I realize I am missing many many things here, did I miss something obvious? Is there any part of my thought process just plain wrong?

  • One of the most scalability-affecting factor is the service bus implementation. I'd say even more than the rest of each server implementation. In your referenced article it is mentioned, for example, that they had to tweak the XMPP implementation. Jan 10, 2017 at 19:03
  • Yeah, the service bus could be a potential bottleneck, but how would you make servers talk to each other? (Do they even need to talk to each other? Not sure now...) Jan 10, 2017 at 19:06
  • I just mentioned the service bus as its details were missing in your post :) Personally I'd design the whole system as a distributed cloud app - easier to scale than a discrete solution. Jan 10, 2017 at 19:52
  • @MariaInesParnisari did you come to any conclusion or final architecture of your design ? I need to design a scaleable chat server also, I just landed on this page through google, any help ?
    – Ahmed
    Nov 25, 2018 at 6:10
  • Your question has many possible answers. So, I'm not comfortable answering it definitively. That said, you make an incorrect assumption in thinking that you'll need centralized servers to store messages, friends, and presence. All of that can be done via P2P. A should just queue the messages locally, if B doesn't respond. Then when B comes back online, it should ask A for messages in FIFO order. Jan 3, 2019 at 2:04

2 Answers 2


You can use a P2P network, but it's architecturally interesting.

Using something like Kademlia as a DHT for peer discovery means talking to a limited number of nodes before reaching your target. If you stored your message at each of these hops, you'd have redundancy for your message store that may be reliable enough for your requirements. Offline delivery would mean periodic forwarding attempts for each buffered message. That would guarantee fairly low latency in terms of peer discovery, which is probably the most costly part of the problem.

Once a direct P2P connection is established, you're clearly in online mode and can skip offline storage (or not).

You can also run nodes that persistently store messages, but otherwise act as regular DHT participants. That would give more reliability, at the cost of only running a handful of nodes.

As @aridlehoover writes, though, there are so many possible answers that you can't really provide a final one.


I'd design this in the cloud and use something like MongoDB or Azure Cosmos DB. The upside is that the database handles the scaling on the data side, so you don't have to worry about it.

All that would be left is to write a Web API on top of the datastore you chose and as that could also be cloud-based, your cloud provider could handle scaling of the requests automatically.

If the cloud is not an option for whatever reason (can't think of any right now), then realize that both of these database platforms are hostable on-premise as well. That would leave you with scaling the Web API, which can be done easily with containers and something like Kubernetes, which BTW would also be my recommendation to hosting the database platform itself.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.