We have a ton of GPS devices in vehicles. These devices need to communicate with our system. To achieve this, we need to parse the device's messages before proceeding.
The following describes my scalable and fail resistant architecture idea.
The devices are black boxes that send us buffers of data via TCP every X seconds. A typical message follows this trajectory:
Each device communicates with a RabbitMQ server (R1) that has a persistent queue. This way if something fails, no messages are lost.
R1 sends the messages to the Message Parser Cluster (Hamster Cluster). Fancy name for a bunch of hamsters that receive a data buffer as input and output a JSON object we can understand.
The Hamster Cluster then sends the parsed messages to another Persistent Queue (R2), which has the same properties as R1.
R2 then sends these messages to the Data Processing Cluster (Monkey Cluster) which does the real work. We are inside our system now. The path ends.
The main objectives here are to have an architecture design with the following properties:
Scalable. We must be able to recruit more Hamsters and Monkeys if any of our Clusters is dying ( No one likes dead animals! )
Fail proof. If any given machine in the system fails, the service must not go down.
As designed, this system has 2 critical failure points: R1 and R2. Should this machines fail, the whole system goes down. Is there a way to avoid having these two critical failure points?
My coworkers made the case for NGINX, which also supports TCP/UDP connections. I understand that using it, we would have load balancing between the machines of each cluster. However, Replacing the Rabbit servers for NGINX servers would still have 2 points of critical failure. Could this be avoided?