-2

A social network has API, but also it has some limitations like the amount of requests that can be done in one second (let's say API will give an error, if it accepts more than 3 requests per second)

I want to write a web application that allows me to send messages via API. However, I want to automatically do requests to API in order to get some information. Therefore, it's possible that the program will make more than 3 requests per second. In order to fix this problem, the program should handle requests in one place, so my first idea is to create a function that will accept new requests (from different places) via tokio::sync::mpsc in infinite loop and then handle them. However, there are two issues:

  1. The function that starts at the beginning of the program seems to me like a poor solution
  2. It's tricky to return the response from API (the solution is to create tokio::sync::oneshot channel and send data through this channel)

Another approach is to create Context so that we can call ctx.make_request(...).await (we can call this function from different places of the program). And make_request controls not to exceed limitations.

What approach is better, or maybe there are other options? (Maybe is there a design pattern that already exists?)

Improve this question

1 Answer 1

Reset to default
0

In short, you'll want to use exponential jittered backoff to handle rate limits: https://cloud.google.com/architecture/rate-limiting-strategies-techniques#client-side_strategies

You'll get rate limiting errors, especially at first, but eventually your system will enter a state where it submits requests slowly enough that the other system can handle them.

Improve this answer
4
  • This is a good idea, but do you have any recommendations on architecturally how to implement this? Please add enough information to your answer so future readers do not need to refer to an offsite link. Links go dead over time, which would leave this answer without enough information.
    – Greg Burghardt
    Feb 9 at 18:28
  • That's really all there is to it, this is really just submitRequest(); -> while !success { submitRequest(); if success {return} else {sleep();} }. Doesn't require anything fancy in terms of architecture or any of that complicated stuff listed in the OP. The link is there for authority, not as an explanation.
    – user60561
    Feb 9 at 20:48
  • @user60561 to be honest, this approach doesn't look like a solution, because I don't want to get rate limit error anymore (with your approach (jittered back off) it's quite easy to get it). There's another approach called “token bucket”, but all requests should be handled in one place.
    – Roy King
    Feb 10 at 6:48
  • your counterparty is already using the token bucket approach, you don't need to implement it again unless it gives you the warm fuzzies, which is totally reasonable. There's nothing wrong with getting errors, but this is your system and you get to write it how you want. I've had to use the token bucket before, but that was when the counteryparty didn't rate limit but instead allowed latency to increase to crazy amounts. Great solution for that issue.
    – user60561
    Feb 10 at 15:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.