4

I have a server which is exposing API through REST. I need to implement a simple Heartbeat service to monitor this server status and availability of API. The interval of Heartbeat check will be every few seconds, like every 3 seconds.

Is it good enough to just define a REST end-point (say ping) which will return the status of the server and API? The client side will call this REST API every few seconds.

Or should I really think of using something like WebSocket just to add this Heartbeat service?

  • 1
    How many clients do you expect to do "ping" to the server every 3 seconds? – Laiv Oct 31 '18 at 8:53
  • 1
    Why does every client need to actively make sure every 3 seconds that the API is up?? – Darkhogg Oct 31 '18 at 14:25
  • I've used Jenkins in the past to ping the heartbeat endpoint and notify the support/development team in the event the response exceeded a given time. The question here is really what you're trying to do with the heartbeat. Are you trying to notify the user or notify a support team or development team that they need to take some action? – neal Nov 4 '18 at 11:51
7

Is it good enough to just define a REST end-point (say ping) which will return the status of the server and API? The client side will call this REST API every few seconds.

Depends. It's important matching frequency vs concurrency because a handful of clients doing pings to the server might not cause a noticeable load but hundreds or thousands of clients could. Imagine 1000, 10000, 100000 clients sending pings every 3 seconds. The peak is likely to make your server sweat.

HTTP connections are costly to perform on both sides, reliable but slow and overall they are blocking I/O communications. Something you need to mitigate if your system is frequently exposed to a high concurrency. However, if the concurrency is low a RESTful solution is more than enough. It's reliable, secure and it scales well.

On the other hand, communications through WebSockets are not blocking, they perform quite well with high concurrency where client and server exchange little information and the computing time is short on both sides. Such is the case of heartbeats. Moreover, WebSockets implements a "Ping-Pong" protocol within its "Control frame" through which you could achieve the expected behaviour. For more information, check out the RFC.

In my opinion, whether to use WebSockets or HTTP REST endpoints will depend on the concurrency and its frequency.

I would also consider a couple of things:

  • Whether clients can (or can not) adopt new communication protocols. If they can not there's no room for discussion.

  • Whether the infrastructure allows you to expose WebSockets. I have worked for customers with very strict network rules where almost anything (but SMTP, SFTP, HTTPS) were unallowed.

Let's say we still expect high concurrency but, for some reason, we don't implement WebSockets. If I were requested to do it this way, I would rather make the server to "push" its state somewhere at a fixed delay. Then I would move the responsibility to expose the state to another service capable of Async I/O (as for instance NodeJS). Then, I would redirect requests to this new service. In other words, redirect the stress of the pings to a dedicated and performant service.

I would tell clients to hold the last state retrieved. If the state doesn't change in further requests, it's likely the service is not reporting its state and something is wrong on the server-side. This approach scales well since you hold the state of several services without affecting their respective performance and you can track them, as a whole or independently, from a single endpoint.

  • The main problem I see with using websockets for this is that the connections are unstable over long periods of time. If the websocket stops working, it doesn't mean that the server is down. It just means that websocket is defunct. If you could combine this with a reconnect routine, that could work. – JimmyJames Nov 1 '18 at 14:43
  • Agreed. Both approaches have weak points. Depending on the prupose, we will have to implement something else to make the Heartbeat secure and reliable. REST is also cheaper, simpler and could be easier to secure. However, the problem concurrency-frequency outweigth anything else. IMO – Laiv Nov 1 '18 at 15:09
  • I'm not convinced that the overhead of HTTP is terribly significant. Creating a new TLS session is costly but once connected, this should survive (in an typical setup) over many calls. – JimmyJames Nov 1 '18 at 15:16
  • But it's blocking and thousands of blocking calls degrade the service performance. Heartbeats should not erode our capacity to perform our business. That's why, In the case of REST, I suggested decoupling. – Laiv Nov 1 '18 at 15:23
  • One call every few seconds is not going to create an issue on any decent webserver. NIO is an option but for such a low volume call, I doubt you'll notice. The client would probably have a dedicated thread for this. I'm reading this as there being a small number of monitoring clients. If you have many clients doing this, then yes, that might be problematic. I'm not sure the websockets overhead is better, though. It's really going to depend on the implementation. That means keeping track of all the clients essentially the whole time. – JimmyJames Nov 1 '18 at 15:33
1

My opinion is that you should just stick to REST. No need to add another protocol/technology with all the potential problems that come with it, while you don’t even need any functionality from WebSockets. Also this solution stays as close to the actual API as possible. There is (depending on implementation) a tiny possibility that the WebSocket would remain open while the REST api wouldn’t accept any new requests.

Also a quite battle-tested piece of technology called Spring Actuator also does this over REST endpoints, though this does more than just check for heartbeats.

0

I agree with @Laiv and gave him a +1, but would like to add a bit of a thought exercise for you.

What is the heartbeat supposed to tell you and who is supposed to be checking it?

I know you think you answered the first question, but not really.

You said “to monitor service and API availability”. Does that include just the actual API processes? Or do you mean from a consuming application perspective? If your API aid up and running but a client can’t hit it because the firewall or load balancer are misconfigured, is that something that should interrupt the heartbeat? What if you’re load balanced across multiple servers do you care that one is down but two are up? What if your API is dependent on upstream stuff like a database or a third party service, does the heartbeat check all that as well?

As for the “who” question, do you want all your clients hitting the heartbeat service and checking on your availability, or is this something for you to keep an eye on it? How much information do you want the heartbeat to expose to them?

Overall, I disagree with the principle of a heartbeat service specifically being managed and maintained in most cases. I can’t get enough from your question to ascertain if yours is one of the exceptions, but in general I would favor robust logging and a log aggregator like ElasticStack or NewRelic to provide the monitoring and if I really want to test it stuff is working end-to-end from the clients perspective, I’d do it against one or more of the existing GET endpoints, or even just setup clients that can run some set of integration tests regularly to hit multiple key endpoints (with traceability to the “test” account to clean up any artifacts), but that wouldn’t be every 3 minutes (nor need it be if the logging infrastructure is configured right).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.