I am designing a AWS web service which is going to get 1000 TPS from devices(Android) and it has dependency on multiple downstream services. The usecase is to hit this service periodically from device, get a piece of data and cache it in the device memory. Since device does not require the data immediately, I designed this way
Device: puts a request in queue (SQS)
Service: polls messages from SQS, process the requests and publish the result to devices via FCM
The Problem is service takes at max 2 seconds to process the request and downstream services would not scale as much as service. In short, I can only process fraction of incoming requests per second (Lets say 200 requests, 20% of actual TPS). This leads to backpressure build up in the request queue. Reading through Internet, I found that general strategies to handle backpressure are
- make queue bounded, throttle the producer when it exceeds its size and make producer retry after some delay
- Increase number of consumers. (In this case, This is not an option due to downstream bottleneck)
How throttling helps to solve backpressure problem? If the TPS is consistent, Wouldn't it create the same problem even when producer retry after delay? At the end some producers will exhaust retries and requests go unprocessed.
Initially I wasn't aware of backpressure and was thinking storing messages in queue will aid asynchronous processing but now I am starting to feel queue is creating more problems than It helps. Is queue even relevant for this usecase ?
What are the real benefits of having a queue in front of service?
Appreciate any help!!