2

I've been working on implementing a customer-facing REST API at work the last few weeks. Although there's not yet a requirement for it, I think it'd be really nice to have analytics for this.

For example, to track how many requests are being made by which client, etc.

I'm planning on creating an analytics app over the weekends that will have to receive data from our API server. I haven't done anything like this before so I have a few design questions:

What method/protocol would work best for this?

I've been thinking of going with a push model, where the API server would push logs/data to my analytics app such that it could persist it and I can create a nice UI with charts, graphs, etc. I was originally going to just use REST with HTTP since that's what I'm most familiar with. But I was also considering using web sockets. I've done some research and I think I understand the differences, but it's still not very clear to me which the better choice would be.

A design decision that may affect the choice is this: Should the API server be pushing these logs/data every time a request is made? If so, I'm guessing websockets may be better since it's better suited to handle numerous smaller requests.

On the other hand, this API server may be receiving tons of requests. I was also thinking of maybe collecting a bunch of request log/data first, then sending it over as a batch to the analytics app (maybe based on a time interval, or even based on a count). If this is the better approach (bulk vs individual pushes), would REST make more sense here?

I'd love to hear any thoughts and comments on this. I'm here to learn from everyone, thanks so much for your time!

  • 2
    1. ”Although there’s not yet a requirement for it” - then don’t do it. By all means bring it up with your team, and if they agree that it’s valuable it can go into your backlog where it can be scheduled and prioritised appropriately, but don’t just go rogue and do it yourself. Especially if 2. ”I’m planning on creating [it] over the weekends” - this is a disservice to your team (use your rest time to rest, so you can be productive when you’re actually supposed to be working) and an intellectual property issue (who owns it?) 3. Your app shouldn’t route its logs: 12factor.net/logs – jonrsharpe Oct 13 '17 at 6:37
2

There really should be no synchronous requests to another server while the user is waiting on a response unless they are absolutely necessary to form the response. Since the recording of analytics is something that doesn't change the response to the user, the processing of these should be done asynchronously.

This sort of problem would often be solved by pushing the information of each request to some sort of message queue, which can be done very quickly so as not to slow down the request. The analytics system would then pull messages off the queue to process them in its own time. The actual protocol used then just depends which message queue you decide to use.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.