I have a service running on AWS exposed via a rest API. The service processes files.
Clients of the service post a file to the api and are returned a Job ID.
Clients can then request the status of the job, if it is complete the job result is returned as JSON.
The API essentially consists of a simple gateway service that accepts the job requests and puts them on an SQS queue. Then there are a pool of ec2 workers that process jobs and update a database in RDS with their results.
The actual time taken to process a file can be between 20 seconds and 2 minutes depending on the file size, however depending on the queue size and size of my scaling group it can take several minutes from when a client submits a job and to when it is complete.
I am now building a separate web application that allows humans submit jobs to this service. The user will upload a file to the web application, and will then see some sort of 'Processing...' screen and then when the job is complete will see the results.
I am unsure as to the best way to handle how the web application knows when a job is complete.
At the moment this is my approach:
1) When a user submits a file, my web application posts the file to the processing service and is returned a Job id. That Job Id is stored in my local DB with a state of 'processing'. That Job Id is added to a local 'Jobs in progress' queue in my web application
2) The web application returns the job Id to the browser and the browser polls the web application checking the state of the Job Id in the database
3) A separate thread pulls items from the local 'Jobs in progress' queue and for each one polls the API service checking if the job is complete. This thread pulls say 50 items from the queue and a time, and checks the status of each job against the API every 10 seconds. If the job is complete its status is updated in the local database along with the job results.
I know this will 'work' but it seems cumbersome and I wonder how it will scale, and if I am deploying this in a load balanced environment (say on AWS) how will it cope with the potential failure of one of my web app nodes.
It also seems to me that this is a common use case, and perhaps there are some better patterns.
The web application is being built using Spring.