0

I have producer / worker queue (1 producer, many workers).
Currently using Redis, but I dont mind switching to RabbitMQ or anything else.

Worker takes a task from queue, runs long running job and confirms that it finished.

Now I have a new type of worker which allows cheaper queue processing, but it may not always be available or there may not be many of them (AWS spot instance)
I need to provide tasks from queue only to new worker type but if the task fails to process after some time (i.e. 30 seconds), it should be provided to other workers instead to meet SLA.

How should I implement this?

In short I need to deliver message to one queue, and it is stays there for more than 10 seconds, move it to another queue.

-1

Well, RabbitMQ can do exactly what you need to: https://www.rabbitmq.com/ttl.html#per-message-ttl-in-publishers

After TTL expires, there is an option (see doc) to move a message into another queue.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.