How do I send data to HTML page only when a POST endpoint is hit?

Question

I've looked through similar questions and tried to Google what I'm trying to do, but I haven't found anything concrete.

What I'm trying to do: where I work, we use an HMI/SCADA system that displays tags, machine states, and other standard SCADA functions. Recently, I figured out how to send HTTP requests from the SCADA system and I tested sending tag values and other similar data to an Express server I spun up, where a value is POSTed approximately every second (for monitoring or reporting; think of a temperature gauge reporting temps every second). Even though the system we use is considered "cutting-edge", the user interface is blocky and lacking, especially when compared to modern web applications. I had the idea to send data from the SCADA system to a POST endpoint, which will then be processed by the Node.js / Express application and reported to an internal website. This way, I can use graphs and charts (from D3 or chart.io) and pre-built UI component libraries like Bootstrap or TailwindCSS.

My question: I use the SCADA application to send an HTTP POST request with a data payload. I can verify that the Express server receives the request and I log the message. However, I don't know how to send the POSTed data to the single (for now) HTML page. I have a simple HTML page with a progress bar Bootstrap component that I'm trying to control from the SCADA application. How do I tie the front end components (say, a graph or a Bootstrap progress bar) to update whenever the POST endpoint is hit? I can use the Javascript fetch() but the SCADA system won't necessarily POST at set intervals, so how do I only make changes to the UI when a POST request is made without polling (ie using a fetch() every so often from the frontend, which would be a bad idea, I think).

Also, if you can foresee other issues with what I'm trying to do, I would greatly appreciate any feedback. This is uncharted territory for our company, so if it works, we will likely build out reports and other user content in this manner, and I'd like to create a solid foundational architecture to support many requests per second without lag, with instant changes to the UI.

Answer 1

This is fairly standard. You just need to save the data that is POSTed to the server in a datastructure(variable) in memory in the express server. Then later you can GET it out. If you only care about the most recent data it is probably overkill with databases. If you need data over time you should probably look into saving it in a database.

Regarding polling, it is not as bad as it's reputation. You already get the data at a known interval(1s) so polling data out from the frontend at a similar interval or a fraction(5s) is reasonable. If you have limited bandwith at frontend, you can implement some change logic on the server so it doesn't send data data if it is the same, but I don't know if it even possible for your data to be the same for lets say 20s. If not, probably not worth the effort. If you decide it's worth the effort look into npm package express-ws and friends.