I have producer / worker queue (1 producer, many workers).
Currently using Redis, but I dont mind switching to RabbitMQ or anything else.
Worker takes a task from queue, runs long running job and confirms that it finished.
Now I have a new type of worker which allows cheaper queue processing, but it may not always be available or there may not be many of them (AWS spot instance)
I need to provide tasks from queue only to new worker type but if the task fails to process after some time (i.e. 30 seconds), it should be provided to other workers instead to meet SLA.
How should I implement this?
In short I need to deliver message to one queue, and it is stays there for more than 10 seconds, move it to another queue.