Skip to main content
3 of 4
deleted 75 characters in body

Distributing tasks to several clients

I describe a very very simple scenario. I try to develop a system where many machines can help another one (server). I tried RabbitMQ and Redis, but both don't fulfil all requirements.

I have 1-server and n-clients. The server has several tasks to solve with n-subtasks. A task is called calculate-A and has 5 (subtask). Each subtask is send to a Queue and gets processed by a client. The results are send back to the server.

  • A subtask is never solved by more than 1 client
  • Subtasks are independent to each other
  • I need to be able to remove all subtasks of 1 specific task from the queue

Any ideas which approach I could use here?

                                                    +-----------+
                                                    |           |
                                                    |  Client A |
                    +----+----+----+----+----+      |           |
                    |    |    |    |    |    +----->+           |
    +-----------+   |Sub |Sub |Sub |Sub |Sub |      +-----------+
    |           |   |Task|Task|Task|Task|Task|
    | Server    +-->+5-B |4-A |3-B |2-A |1-A |
    |           |   |    |    |    |    |    |      +------------+
    +-----------+   |    |    |    |    |    +----->+            |
                    +----+----+----+----+----+      |  Client B  |
                                                    |            |
                                                    |            |
                                                    +------------+