0

I have a hobby project, for simplicity imagine a poker game, consisting of a backend (Spring) and a client application (Flutter) where the client can join rooms in which he can take a seat. The client makes requests via REST and gets updates sent via WebSocket (user took a seat, game progress events etc.). The client also receives updates meant only for himself (personal message, cash update, etc.).

I initially used WebSocket because it was handy to do so but I didn't think about the scaling. At some point it would be too many connections (in case app usage explodes) and I could scale by deploying more backend instances on more servers (number of WebSocket connections are limited by the system, correct?).

I then thought about using a Message Broker like RabbitMQ but I somehow don't get if this doesn't stumble into the same problems if user numbers increase. Additionally I don't see how I would connect the clients to the broker (directly? Flutter -> RabbitMQ?) and limit the access for each user for their own channel (personal updates) (Authentication in RabbitMQ, so it must know about every user?) and no limit in access for the room channels/everything that's public and everyone can receive.

I feel like I should know this given I have some professional years as a developer now but I also feel like I'm missing some point so help on this would be greatly appreciated since I'm stuck with these thoughts for a while now. Thank you.

6
  • if you want the push messages you need an open connection regardless of tech. websockets is the way to go for websites
    – Ewan
    May 14 at 20:16
  • The client can be built for web and desktop (and mobile) target which is why I use Flutter.
    – mvi
    May 14 at 20:29
  • Websockets are a connection technology, message queues are a way to deal with ephemeral data (such as data sent over a websocket) when encountering interruptions in said connection. They solve different things but this question makes it seem like they are equivalent solutions to the same goal.
    – Flater
    May 15 at 0:26
  • I cannot help with actual experience, but with some search-engine skills: read germano.dev/sse-websockets and see what you can do with it.
    – mtj
    May 15 at 5:52
  • @mtj Didn't know about Server-Sent Events yet, read like it is what I search for. Thanks.
    – mvi
    May 15 at 7:04

3 Answers 3

2

WebSocket is a communication protocol. RabbitMQ is a message broker. Two different things.

Now, whether the client connects to your WebSocket server, or RabbitMQ server directly, is of no difference in terms of numbers of used sockets. You will hit limitations on both.

But there are other, major differences. First of all, WebSockets are supported by browsers. Browsers are very limited in terms of networking, in particular they won't let you open an arbitrary tcp connection. So if you want to support browsers, then WebSockets is a good choice. I don't know if RabbitMQ works over WebSockets (other answer claim it does). But lets say it does. Then you now still use WebSockets, but connect to RabbitMQ directly, instead of your server. If it doesn't support WebSockets, then you have no choice but to put a WebSocket server in front of it.

So we are in a situation where transport protocol isn't really relevant (which is fine, there's nothing wrong with WebSockets), and the question becomes: should I connect to my own server or RabbitMQ directly? And I do believe it is the true question here.

First of all, however RabbitMQ scales, you can of course implement your custom server at least as well. But if RabbitMQ scales badly, then you won't be able to improve it, unless you put a custom server. This is of course a tradeoff: you need to sacrifice time and have knowledge to do that correctly. Secondly, RabbitMQ won't be able to handle more sockets than your custom server. Sockets, as in file descriptors, aren't bottlenecks anyway, you can always modify your OS limits to handle millions if you want to. The true bottleneck is due to what those servers do with the data that sockets transmit. And how big the traffic is. In which case transport protocol is rarely a bottleneck.

And thirdly, most importantly: connecting to RabbitMQ directly will completely break your application. What about authentication? What about validation? What about additional custom logic? What about security (e.g. ddos)? Etc. etc. Don't ever do such thing. Resources like message brokers or databases have to be hidden behind servers. No production ready application can work like that.

So how to scale such game application? There are many variants. One of the more widely accepted way is that when a new game starts, the application choses a server for it to live on. So you need two kinds of servers: one that hosts a game, and one that choses the first one (technically can be the same server, but its not necessarily a good decision - a story for another time). Then the game, and all participants are redirected to that particular server, and communicate with that server only.

This works well for any scenario where we have isolated pieces, like online games. But even MMO games use similar strategy: these divide game world into regions and try to ensure that no more than couple of thousands of players are at the same time in one region. And they put a server per region.

Depending on your actual requirements and how "real" time your game has to be (poker probably does not have to be correct up to say milliseconds) then there is a different, global approach possible. You have say 10 web servers that accept (websocket) connections. In a round robin fashion or something, arbitrarily. And then those web servers communicate between each other. And here you can put a RabbitMQ instance behind them as an internal message broker. But other pub/sub servers (e.g. Redis) will probably work better.

1

WebSocket connections, while convenient for real-time communication, can pose scalability challenges as the number of connections increases. Deploying more backend instances on additional servers can alleviate some of the load, but it's not a scalable long-term solution due to resource limitations on individual servers.

Introducing a message broker like RabbitMQ into your architecture can help offload processing from your backend servers and provide a more scalable solution. However, you need to consider how to connect clients to the broker and manage authentication and authorization.

Clients typically don't directly connect to the message broker. Instead, they communicate with your backend servers over HTTP or WebSocket, and your backend servers handle communication with the message broker on behalf of the clients. This approach centralizes authentication and authorization logic within your backend servers.

Authentication and authorization can be implemented in your backend servers, which act as intermediaries between clients and the message broker. When a client connects to your backend server, you authenticate the client and establish a session. Then, your backend server subscribes the client to relevant topics or queues in RabbitMQ based on the client's permissions.

Therefore, leveraging a message broker like RabbitMQ can improve the scalability and reliability of your system. By centralizing communication through your backend servers and managing authentication and authorization at that level, you can control access to different channels and ensure that clients receive the updates they're entitled to. Additionally, message brokers provide built-in mechanisms for scaling out your infrastructure to handle increased load.

3
  • Pretty much my thoughts but being unsure about the authentication part as I wrote in the question. I'm failing to see though how I connect the client with the broker through the backend...is this really a direct connection then or is it something like client -> (client-backend-queue) -> backend -> (backend-broker-queue) -> broker?
    – mvi
    May 15 at 7:13
  • "Clients typically don't directly connect to the message broker. Instead, they communicate with your backend servers over HTTP or WebSocket" adding rabbitMQ doesn't remove websocket from your solution. so has no effect on the scaleablity limit
    – Ewan
    May 15 at 7:46
  • @Ewan message processing happens asynchronously in the backend. So, leveraging a message broker does enhance scalability. May 15 at 22:02
1

Both RabbitMQ and websockets connections work over TCP/IP and have similar limitations to the maximum concurrent connections between any two machines.

Getting a web browser to connect to a rabbitMQ server is possible... via websockets!

https://www.rabbitmq.com/blog/2012/05/14/introducing-rabbitmq-web-stomp

Essentially the problem is getting a rabbitMQ client written in javascript or webasm that works within the browsers sandbox limitations.

However, by tweaking your networking settings though you can get impressive results using websocket

https://blog.caustik.com/2012/08/19/node-js-w1m-concurrent-connections/

If you are worried about scalability though, you need to be able to split the users or the games amongst multiple servers. Using your example of a poker game you have a max of maybe 10 players? well under the limits of websocket. So if you can have a different machine hosting each game you will have a scalable solution.

In practice this might mean a light networking layer between your main processing application and the incoming connections. Allowing you to expose multiple IPs and ports for incoming connections to avoid hitting networking limits.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.