Options to maintain queue of resources that will be uniquely returned

Question

I need to maintain a queue of resources in a cache such that when multiple clients try to pop elements from the queue concurrently, each element would be uniquely returned to only one of the clients. Is redis list a good option for this? What other options can I consider? Also, what is the property I am looking for? I know its not thread-safety or atomicity...what is it?

Answer 1

Any queueing service supporting multiple consumers will do. REDIS lists are suitable for this, as are the various message brokers such as RabbitMQ or ActiveMQ.

You need to consider how you handle crashes of consumers while they hold a resource. This may get tricky. Message brokers often have an acknowledgement mechanism so that messages are only removed when the consumer acknowledges that it has completed work on that resource or task. If you use REDIS, you need to implement such a mechanism yourself.

You also need to handle blocked consumers somehow, probably by killing them after some timeout, so incomplete messages are returned to the queue.