I am building an open source task management and note taking service that runs on AWS. I'm building it mostly as a hobby in my spare time and will eventually ship a mobile app that consume it into an App Store. Because this is being built nights/weekends as I have time I am trying to use managed services as much as possible - especially ones that are pay-as-you-consume.
My REST services are all serverless using API Gateway, Lambda, SNS and DynamoDB with Cognito used for customer identities. The following diagram is a high-level diagram of my micro-service architecture.
Lambda is used as my compute service here to process the incoming HTTP requests. The data is validated and stored in DynamoDB. Each Microservice is responsible for the resource it owns (such as Projects) and stores the resource data in it's own DynamoDB Table. Each Microservice has it's own SNS Topic as well.
The Task Microservice has a Lambda that is subscribed to Project changes and takes action. For example, if a Project is deleted then the Task Microservice lambda subscriber will go through and delete all Tasks in its DynamoDB table that were associated with the deleted Project.
The Microservices themselves are composed of many Lambdas, one per HTTP Verb + Route.
When a Project resource is created via a POST then a message is published to the project SNS Topic with an attribute of
project-created, allowing for message filtering to take place downstream. HTTP Deletes publish a
project-deleted message while HTTP PUT publishes
As I mentioned above the Task microservice has a Lambda handler listening for
project-deleted messages. I'm about to add a 2nd Lambda subscriber to the Task microservice to listen to
project-created events and store them. The Task microservice will store the
userId of the record associated with the
project-created message payload. This way I can perform validation on HTTP POST requests to my
/task API which includes a
projectId field in the HTTP body.
curl --location --request POST 'https://api.foo.com/task' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer abcdefg \
With how Lambda and the AWS SAM framework works it doesn't matter if I deploy a Lambda that handles all requests with route inspection and have API Gateway proxy all requests to it, or 1 lambda per HTTP verb + route and have API Gateway manage the routing. It's the same amount of effort, albeit less code if I let API Gateway manage it all like I am.
I chose this path to minimize risk to breaking every endpoint on a Microservice. I also chose a distinct Lambda handler for each type of SNS message I want to watch for in the Task Microservice so that I can scale my subscription needs out without having to regression test every subscription code path in 1 large Lambda Project subscription handler.
This has been very cost effective for me and it has allowed me to be pretty agile in my development. Is there anything that I am missing in this design that is going to kill me later on from an operational perspective? I know that with this kind of architecture comes complexity from managing it all but I'm ok with that complexity so I get the horizontal scaleability and lower blast radius if mistakes are made. Being a solo developer on it without QA and lacking a decent test suite (I'll build out) reducing the blast radius was important to me.
One last item of note is that each Lambda also has a dedicated IAM Role with the permissions it needs to do its job. For instance, the HTTP POST Lambda has permissions to publish SNS messages and create new DynamoDb records but not query/scan them. The HTTP GET Lambda can Query DynamoDB but it can not create or modify records nor can it interact with SNS.
Thanks for reviewing and sharing any feedback or insight!