Background: I'm working on a new project at work that will run in AWS. We're trying to use a modern microservice architecture and take advantage of cloud technology, but we don't have much experience with that yet. Due to some library restrictions, we need to run at least some of the services in Python, which my organization has a severe lack of production experience with. Most of our previous applications are in Spring/Java (and some of our other microservices will be running using them).
In our architecture, we want to have a number of microservices running in a sort of processing pipeline that communicates using SQS/SNS for async communication. At the same time, they'll need to be able to accept REST communication from our frontend or other services that need data that the service owns.
My Question: How can I best handle both of these forms of communication in python? Is my solution below viable, or should we be doing something entirely different with our communication strategy?
What I've come up with: My current idea is to have the background/pipeline processing for a microservice happening on an EC2 instance that will listen and process SQS messages, then make a POST request to a storage service specifically for that microservice which sits behind an AWS API Gateway and uses Lambda functions to store the results in a database. In this way, we can have the frontend/other services call to the API Gateway to get data out of the microservice without interrupting the processing of SQS messages. This also lets us take advantage of serverless architecture, making scaling much simpler.
I personally am fairly inexperienced and haven't been able to find satisfactory information or examples of architecture like this through my own searching, so I would appreciate any insights or inputs you can give.
To clarify, what I'm most curious about is the more fine-grained issue of how to open up my microservices to both REST and messaging communication, or if some other pattern would be a better idea.