Design pattern for socket.io and Express

Question

I have a Node.js Express RESTful HTTP server I will call server A and an Express socket.io server I will call server B.

Server A responds to all HTTP requests by clients and server B listens to the MongoDB oplog and sends database updates currently to all clients.

I don't want to send all DB updates to all the clients -I want to optimize the system so that server B and the connected clients experience less load by partitioning server B so that only certain/relevant DB updates are sent to certain clients...In this way server B could send just a fraction of the messages it would have before, and each client wouldn't have to handle all the updates.

I have read a little bit about socket.io rooms and how clients can join a namespace.

In the past I have appended the user id to all MongoDB collections that pertain to a certain user, so I could have each client join the namespace denoted by their user id.

This might mean that each client only gets the updates corresponding to their namespace on the server + a few universal updates that pertain to all users.

My question is how to best design this so that all users get the updates they need while minimizing the load on the server and the clients.

Does anyone have any advice on how to best do this with MongoDB, the MongoDB oplog and a separate socket.io server?

Answer 1

Although you asked this a year ago, I have a solution! One you may have use for if you are still using socket.io

Hey I came across this precise issue while creating a haiku sharing application (you can check it out at http://haiku.run )

You can see my source code hosted on github at http://github.com/sova/haiku.run

Check out the code for index.js https://github.com/sova/haiku.run/blob/master/src/index.js#L95

on Line 95 there is the following code:

clients[clients.indexOf(socket)].emit('load_haiku_from_cache', a_latest_haiku);

Basically, when a new user connects, I want to load the cache of haiku that are stored in RAM on the node app in a variable called haiku_cache or something like that. There's 7 of them in the cache, and they get emitted when a user freshly connects.

However, it's different because now if a new user, say user 2 connects, then user 1 would (normally) get a double update of the cache in their socket-spurned-feed of stuff.

So, the above code effectively emits information along one particular socket.

In conclusion, I recommend you keep track of all the sockets that you wish to receive a particular set of informations, and then use the above-like line to send to specific sockets. You can emit across the wire / over the socket to one specific connected client, which is really nice, using the above line.

Hope that helps anybody looking around for a good solution to "how to emit to a subset of connected peers" since the group/room socket.io stuff is kinda messy imho.