I have 2 micro-services. One is a Django API, the other is a worker. Each one of these lives in side a docker container.
The API is a public API for the web front end. It handles CRUD operations for users of the web application and it only has an authentication backend for endusers. Some REST actions by the user create SQS messages. The API and worker each have their own AWS IAM roles and permissions. The worker polls the SQS for the messages, and for every message, does some processing.
The issue is, that after the worker is done processing it needs to communicate back to the API with success or failure. The workers are async, and could take a few minutes to finish. I see 2 ways of doing this:
Create an endpoint on the API for the worker to hit. The problem with this is that i think that the django API should only authorize end users of the application. I would have to add another authorization backend just for the one API endpoint or do some sort of token based authentication.
Hit the database directly, but this involves having the database credentials in multiple places.
Is there a better way to go about this? I don't like the idea of having multiple authentication backends in what is meant to be just the end user CRUD api. I also can't have a consumer running in the same docker container as the API because it needs to run as a single process inside of the container.