1

I have producer / worker queue (1 producer, many workers).
Currently using Redis, but I dont mind switching to RabbitMQ or anything else.

Worker takes a task from queue, runs long running job and confirms that it finished.

Now I have a new type of worker which allows cheaper queue processing, but it may not always be available or there may not be many of them (AWS spot instance)
I need to provide tasks from queue only to new worker type but if the task fails to process after some time (i.e. 30 seconds), it should be provided to other workers instead to meet SLA.

How should I implement this?

In short I need to deliver message to one queue, and it is stays there for more than 10 seconds, move it to another queue.

2 Answers 2

2

Well, RabbitMQ can do exactly what you need to: https://www.rabbitmq.com/ttl.html#per-message-ttl-in-publishers

After TTL expires, there is an option (see doc) to move a message into another queue.

1

You can use RabbitMQ.

Set TTL for first RabbitMQ queue 10 sec and use Dead Letter Exchanges and bind the second queue to it as says here https://www.rabbitmq.com/docs/dlx and here https://www.rabbitmq.com/ttl.html

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.