- TeamA: Adds 5000 IDs to a blacklist per day and notifies other teams by publishing an SNS message per ID added that all other Teams listen to.
- TeamB: Gets 4K TPS to its API, and if a given ID is within that blacklist, it must reject it that request.
- When TeamA updates their blacklist, TeamB and all other Teams, have 30days to start rejected requests for that ID.
- Getting a request for a blacklisted ID will only happen due to a race condition across Teams. I.e. TeamB has blacklisted the ID, but TeamC has not and TeamC calls TeamB with the blacklisted ID. This is an edge case. However it is not guaranteed that it will never happen again, it could totally happen that TeamB gets a request with a blacklisted ID after a year - TeamB must reject that ID.
- ID length will be up to 256chars.
- There is no repository to check whether a given ID is blacklisted or not.
- TeamB has around ~200hosts in the fleet.
- Hosts are deployed to on a daily basis - which causes a restart.
Tenets to follow for design:
- TeamB's availability should not be affected.
- API latency should not increase.
- If possible, avoid a adding a new dependency.
A: TeamB adds all blacklisted IDs to a DB (Dynamo) which is then checked during all of its API requests.
- Con: Adds latency to its API.
- Con: Adds dependency on Dynamo - which effects its availability.
- Con: Expensive to check DB just to handle an edge case.
B: TeamB stores all of the IDs in a local CSV file (or some format) which gets updated every 30days via cron job. Which TeamB then downloads locally every 30days as well, and loads into memory at startup.
- Con: With 5000 IDs per day being added, it might not be scalable in the future (when the file size gets into the GBs).