I'm implementing my service on AWS as a set of stateless EC2 nodes controlled by an ASG, backed by an RDS instance. I need to throttle the overall number of outgoing messages my service sends but can't throttle the number of incoming messages.

The specific outgoing messages in this case are SES messages, but the problem seems pretty general to me (my service sends transactional mail, this is not a bulk-mailing problem).

"Use Guava RateLimiter" is not a sufficient solution, the number of instances in my ASG needs to remain variable but the overall number of messages sent across the group of nodes needs to remain under the limit. I think there's going to have to be some shared-state here.

I'm thinking I'm going to have to implement some kind of queue of work items in the database and then some kind of slot-based throttling system to limit the amount of outgoing messages each node processes, so that the overall number of outgoing messages doesn't exceed my limit.

My solution doesn't need to be internet-scale, but I'd like it to be able to deal with a few-thousand outgoing messages per-second. I'm pretty sure the above-mentioned queue/slot-processor system will easily stretch that far on an RDS instance (and it can be re-factored later to work off of Elasticache or something). Another thing I like about the proposed system above is its minimum initial cost at low-scale (well, discounting my cost to write the code).

Are there any AWS services, SaaS solutions or even just libraries that can help me with this? I'm thinking cost-wise, a full-fledged AWS / SaaS solutions will have a cheap introductory tier initially, and as long as the price doesn't get crazy to scale up it should be fine.

put on hold as too broad by Robert Harvey Jun 14 at 18:15

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.


Several options:

  1. Task queue, if you already have the infrastructure establish a single queue which either processes the jobs in order to send emails, or essentially acts as a buffer for pre-generated emails. Use/implement capacity controls for how many of those tasks (and hence emails) are performed within a X time frame.

  2. Create a separate service whose sole job is to control send emails. Have all internal business logic forward their emails through this instance.

  3. Quotas. Each service/node/process has a quota. Every so often additional quota is distributed to the nodes by a quota generator by some logic. A service/node/process subtracts one each time it sends an email, or it must wait for additional quota. Periodically it distributes half of its quota to some K other services/nodes/processes. This ameliorates starving when there is available quota.

Take your pick.

To be honest solution 2 is the least invasive, and probably simplest. Depending on how the emails are dispatched, there might be an off the shelf email proxy which can enforce those limits for you.

Solution 1 leverages any already existing task handling. If your system doesn't need task handling avoid this as a solution.

Solution 3 leverages local knowledge and a global update algorithm. If that global update algorithm is not already implemented and you do not need it otherwise, avoid this solution.

Not the answer you're looking for? Browse other questions tagged or ask your own question.