Best Communication strategy for monitoring dashboard

Question

I have a requirement where I need to display certain stats on Admin Dashboard in my web application (Angular+Java). These stats (transnational data) are being generated on different servers (connected via LAN to my web app host server) by C++ programs.

Since we don't have a centralized database we can't just store statistics into DB and retrieve it in Java services and display on UI. As a workaround I have created a shell script which SSH on different servers, collect stats and prints them on console one by one. This script can also be fired on regular intervals from my java web service.

I can parse output of script and send it to UI but this is not an efficient solution. The constant SHH after every few seconds generates lots of interrupts.

Few thoughts which initially crossed my mind are below

1. Create a Java TCP Client which sends requests to different servers and get the data periodically.
2. Create a Java TCP server which accepts json data from different servers.

I have never worked on something like this. Can somebody suggest a better strategy or preferably point me to online resources in order to learn and understand these sort of architectures.

Answer 1

This answer will fall into the category of "suggest a better strategy".

Gathering stats on infrastructure assets (not just servers) is what I am currently doing for a living in a global financial services leader. I am trying to understand why one would avoid using a database to hold the bits that are gathered. If cost is a concern there are open source solutions that are fantastic. If size of the app is a concern use a lightweight or embedded DB. While what you describe can be done without a DB, it will require re-creating of a lot of wheels. Even if you justify and do that, and release your product it will not take long for your customers to want to see the stats in a historical context, or want to make these stats available for consumption by other means/apps. If you are unable to do that, it will severely limit the business usefulness of what you are building.

In terms of the client/server architecture for transferring the stats from the collector into your receiver, by coding that using native TCP (using sockets I presume) at a minimum you will need to devise a protocol for session setup and transfer of the data, and you will need to code a connection manager that listens for inbound connections and starts up a handler thread (so your app can support concurrent connections from the collector clients).

All of this is taken care of by leveraging a webapp container like tomcat or wildfly. By building as a webapp on a container you can also leverage frameworks/libraries so that what you develop also comes with a SOAP API without a lot of additional development work on your part.

If this is an academic exercise then the approach of using low level TCP programming makes sense.

Answer 2

I would use one of our successful projects as example. Sentilo

Sentilo is (in bottom lines) a data concentrator that stores milion of entries from any sort of sensor, gadtget or client. It store data into noSql db and the way to do this is by Restful and stateless webservices. Which accepts json as data representation.

Then offers also webservices to serve this data.

What do you have in common with Sentilo?

You are connected to all those servers you need to monitorize. Good. Your server is going to be the gadget. You also have made a shell script that will perform the task of to get the data.

Here comes the new part.

Instead of remote calls, schedule these calls. Every 2,5,10 seconds. Script will get the response and it will redirect it to a URL vía curl.

This URL is going to be your concentrator. Concentrator is quite easy to build with NodeJs and MongoDB (or Redis). This concentrator have also a URL to serve all this Info. But it does serve data from db not raw data from the script.

With NodeJs to parse script raw data is going to be easy too.

Finally your current Java webservices consumes data from the concentrator (or directly from the noSql db).

I will go even further. Kill your Java webserver and let Angular consume data from your concentrator. The availabilty of NodeJs is quite higher than the Java app servers. And faster. And it supports way more requests and concurrency than any Java app server.

If you don't like shell scripts. From NodeJs you can execute shell commands, to do pings, http/s requests, socket connections, soap webservices clients, ssh connections, etc...

This would be my approach. Here some links that may help to investigate

1. NodeJs: The core of my approach
2. Restify: NodeJs module to set webserver and rest URIs
3. MongoDb: Data repository noSql
4. NPM: Repository of NodeJs modules and other Javascript libs. It's to NodeJs what Nexus is to Java
5. Bower: Javascript dependencies Manager. It's to any Javascript project what Maven is to Java.

Some modules my interest to you:

• socket.io
• node-schedule

Answer 3

Honestly, this sounds an awful lot like a log aggregator, of which there are many existing options. Possibly the best open source version is the ELK stack (Elastic Search - Logstash - Kibana), which gives you the full boat of collection, indexing, and high quality visualizations (which you can customize).

In your case, it'd be a pretty easy solution. Either have each of the servers you're monitoring just put all their state data in their normal logs (with logrotate configured, etc), and then use Filebeat to periodically push that to the ELK stack, or come up with a custom connector to push the data into the ELK infrastructure, your choice.

Either way, everything you described can be done with nearly no code, just setup and config. Worst case, you could write your Java + Angular app to pull the info out of ElasticSearch and do whatever you want with it, if yo discover that Kibana can't provide the dashboard you want.