How to deal with large data in Websocket message?

Question

I wrote a websocket server in Spring Boot and a client in Javascript. These work fine. I also wrote a second client in Java. When this one attempts to handle a frame after connecting to the host, I get this error:

org.springframework.web.socket.sockjs.client.WebSocketClientSockJsSession - Transport closed with CloseStatus[code=1009,  reason=The decoded text message was too big for the output buffer and the endpoint does not support partial messages]

A quick search showed that you can change the buffer size, but should I go this route right away? Should I try to compress the data in some way (I ask with little knowledge of how to go about that)?

Answer 1

I'll propose an answer since it's been nearly a year.

As @Walfrat has pointed out:

Note that if you're echanging too much data, it might also be a design problem that would lead to poor performance in production environment (where the badwith and stability of the connection is not always here).

With that in mind, it might be better to push a more "notification-like" message in the websocket. A browser client could display this as an alert or badge, and a server-client (a server subscriber) could take this as a trigger to then query the RESTful endpoint. So, essentially:

1. Server pushes a "we have new data in this channel" message.
2. Client receives the message, and based on the small amount of info passed, knows which api endpoint to query to get a more full response.

This allows for a near instant knowledge of new data, and using a RESTful endpoint as designed.

Answer 2

Your buffer needs to be big enough to hold at least the biggest message that you can expect. If the messages are huge, it introduces problems. The more connections you will keep open at the same time, the higher the application's memory footprint can become. That is why Spring's internal buffers are not unbounded.

If it is possible to make the messages smaller (e.g. by splitting them or compressing), it is a useful optimization and will mitigate that problem. If you cannot find ways to make the messages smaller, then increasing the buffer size is the only option that is left. Otherwise, Spring will drop the message if it is unable to buffer it.