Bidirectional end-to-end communication between native application and web application

Question

I'm in the planning stages of a project involving communication between users in the browser environment and numerous devices running in a widely distributed manner. The end goal is that users using the web application will be able to send commands to the devices, the devices will receive and act upon these commands, all while sending back an acknowledgement to the user that the command was received. My question is, what are some existing technologies / frameworks that permit this type of communication with scalability in mind? Or, how is this problem solved in existing production applications?

Example Scenario

User X is on a website, served by Server Y. User X has a device, named Device Z, which is anywhere on Earth (including off User X's local network) and connected to the internet. When user X plugs the device into power, it automatically connects to Server Y. User X then logs into the website and can send commands to Device Z, and, upon receiving the commands, sends back a status which is displayed in User X's browser.

Attempted Solutions / My Research on Solutions

Due to the nature of the two way communication, I am heavily in favor of using a WebSocket protocol, i.e. SocketIO. This could work for both the server to web browser link, as well as the device to server link.

• SocketIO does support callbacks, however, not very usefully in clustered environments, which presents scalability issues. The main issue is that if the device websocket is being served by another server (i.e. Server A) and the user is being served by Server X, it seems not possible to use the callback functionality when using message broker/queues like Redis or RabbitMQ to allow cross communication with servers.

Answer 1

If you want a scalable platform, you should hide the server side complexity from clients. No matter it's a device or a user, clients should have zero knowledge about cluster. A client doesn't care which physical server they connected to.

You can implement a register and discover mechanism to let servers in the cluster know which node the client (device or user) connected. Then you need to implement a internal communication mechanism between these two servers who delegate the connected client to talk to each other, it's actually kind of reverse proxy.

User X <----> Server Y <------> Server Z <----> Device Z

Depends on how reliable and fast you need, server to server communication could be broadcast or point-to-point.

If the performance is not the top priority, you can use websocket between servers which can simply proxy input stream to other side. It's not so difficult to develop this kind of platform in Node.js which can integrate with SocketIO very well.

If high performance is very critical for your use case, you may need to touch lower level communication protocols e.g. TCP or UDP. You can have a try on Java Netty project.