thinking about the design of an IoT project where the devices don't have a standard application layer but rather a thin custom application layer utilizing TCP sockets. What I mean by this is that the device wakes up every 15 minutes or so opens a socket on the server and sends data in, waits for any incoming data and then goes back to sleep/powers off.
On the software side Node.js is being used with a
net.createServer to create a socket server and handle incoming data using a single-threaded event loop. When the socket is created the bytes are read and translated into numeric values which are then published to a MQTT topic for downstream processing.
What I'm mulling over is the other part of this data exchange in which bytes are written upon a connection. I.e. sending a command to the device. Since it's a TCP connection-oriented device, the software has to queue any asynchronous data to the device and hold it while waiting for the device to connect. Previously I would hold this data in an array (used C++ previously) and then write to the device when it made a connection. This proved to be a poor design since sofware crashes from time to time and loses any outgoing data to the device.
Designing something new, is it a good practice to handle this in a database? Meaning the asynchronous data is written to the database and then the software simply reads the database for the data at connection time. If there are thousands of devices reporting in that's a lot of reads to the database as well as writes to it for confirmation purposes. Was thinking instead to possibly use a subscribe via MQTT topic (retained messages) when device reports in to read outgoing data. However I'm not sure how well this can scale and the speed at which the software can subscribe to a topic which seems like it would be a bottleneck. Any ideas on which one to go with or alternatives to these two designs?