Architecture of mobile backend

Question

I would like some input on some refactoring I am to do on a mobile backend (API).

I have been tossed a mobile API which I need to refactor and improve upon, especially in the area of performance.

One (of the many) place where I can improve the performance while also improve the architecture is how the backend currently deals with push notifications.

Let me briefly describe how it currently works, and then how I intent to restructure it.

This is how it works now. The example is about the user submitting a comment to a feed post:

1. The user clicks send in the mobile app. The app shows a spinner and meanwhile sends a request to the backend.
2. The backend receives the request and starts handling it. It inserts a row in the comments table, does some other bookkeeping stuff, and then for the affected mobile devices it makes a request to either the Apple Push Notification server or the Google Firebase Service (or both if the receiver has both an Android and an iPhone).
3. On success the backend returns a 200 to the mobile app.
4. Upon receiving status code 200 from the backend, the mobile app removes the spinner and updates the UI with the submitted comment.

It is simple but the issue with the above as I see it is

a) Currently this sample endpoint has too many responsibilities. It deals with saving a comment, and also with sending out push notifications to devices. b) The performance is not the best since the mobile app waits for the backend to both save a comment (which is pretty fast) and send a notification which requires a HTTP request (which can be anything from fast to slow).

So my idea is to remove all about notifications from the backend, and host that in a separate backend app (you might call it a microservice).

So what I am thinking is to do it like this:

1. The user clicks "Send" in the mobile app. The app shows a spinner and meanwhile sends a request to the main API backend.
2. The mobile app also sends of another HTTP request, this time to a notification service which is separate from the main API backend. This is kind of a fire and forget request. So the app does not wait for this in anyway, and it can be send in the background (in iOS using e.g. GCD).
3. The main backend receives the request about the comment, and starts handling it. It inserts a row in the comments table, perhaps does some other bookkeeping stuff, and then it returns the response to the mobile app.
4. The notification service receives the request about the comment, and inserts a row in a notification table (this is for historical reasons, e.g. to make an Activity view or something like that), and then puts a message on some queue (or on Redis). A separate job takes whatever is on the queue/Redis and handles it (this is where we actually send a request to Apple Push Notification Server and Googles Firebase Service). By not having the HTTP notification service do the talking with these external services it will be easier to scale the HTTP resources.
5. Upon receiving the 200 from the main backend, the mobile app removes the spinner and updates the UI with the submitted comment. Again note that the mobile app does not wait on the second request it send off (it's not like it can do anything if that fails anyway).

So this is way more complex. But the main API backend is now only concerned actually saving the comment. The mobile app also needs to send two requests instead of just one, but it doesn't need to wait for the second request. So overall it should giver better performance I think.

With regards to the notification service it could be simpler by not using a queue/Redis but just have the notification service call up Apple and Google with the push notifications. But I am thinking that by separating that out into a simple HTTP service that only does some basic bookkeeping stuff and putting stuff on a queue/Redis it can be fast and simple, and the separate job would then do the actual work of calling up Apple and Google.

Does it makes sense? Or have I over complicated things? All comments appreciated.

Answer 1

An API is high level enough to expect it to do multiple things with a single action. If I have an endpoint that does a "Buy Product" then I would expect it to handle all of the transaction handling, logging, and updating balances rather than breaking them up into services the client needs to contact individually.

You have the right idea of making the push notification a fire and forget operation, but it can be done with a single endpoint on your API. If you use a work queue just make two separate jobs and execute both: wait for one and forget the other.

Answer 2

There are several things that could go wrong and mess with the requests.

For example network issues, cpu overload, running out of memory, internet traffic, db server running out of space, inefficient SQL statements, inefficient db mappings, data tables too huge and poorly indexed, bottlenecks somewhere, running out of workers, etc...

Develop a plan first. You have to identify the source and do act accordingly with the issue.

KISS!!! Start by the simplest solution and test. Do implement load tests for a representative measurament of the performance. Get metrics. Later, these test will measure the quality of the solution too.

Regarding the solution you suggest. Hell no! Why don't you try first with parallel executions? Execute the notifications on another thread. Don't wait for it to be finished. As you say, fire and forget.

But still, you will need a way to prove that the solution has fixed the problem and its order of magnitude.