3 added 61 characters in body
source | link

Here are some requirements for a queue:

  • Every few days add ~100k tasks with various priorities
  • Workers will pull tasks at typically less than 10 / second
  • Tasks need to be completed ~3 times by ~2 unique workers (for error checking), and then depending on outcome potentially by an additional worker
  • Persistent storage of tasks

Since the task processing rate is quite modest is it worth adding a dedicated message queueing system to my stack, or reusing the database (MongoDB)?

Here are some requirements for a queue:

  • Every few days add ~100k tasks with various priorities
  • Workers will pull tasks at typically less than 10 / second
  • Tasks need to be completed ~3 times by unique workers (for error checking)
  • Persistent storage of tasks

Since the task processing rate is quite modest is it worth adding a dedicated message queueing system to my stack, or reusing the database (MongoDB)?

Here are some requirements for a queue:

  • Every few days add ~100k tasks with various priorities
  • Workers will pull tasks at typically less than 10 / second
  • Tasks need to be completed by ~2 unique workers (for error checking), and then depending on outcome potentially by an additional worker
  • Persistent storage of tasks

Since the task processing rate is quite modest is it worth adding a dedicated message queueing system to my stack, or reusing the database (MongoDB)?

    Tweeted twitter.com/StackProgrammer/status/759002594295504898
2 added 32 characters in body
source | link

Here are some requirements for a queue:

  • Every few days add ~100k tasks with various priorities
  • Workers will pull tasks at typically less than 10 / second
  • Tasks need to be completed ~3 times by unique workers (for error checking)
  • Persistent storage of tasks

Since the task processing rate is quite modest is it worth adding a dedicated message queueing system to my stack, or reusing the database (MongoDB)?

Here are some requirements for a queue:

  • Every few days add ~100k tasks with various priorities
  • Workers will pull tasks at typically less than 10 / second
  • Tasks need to be completed ~3 times by unique workers (for error checking)

Since the task processing rate is quite modest is it worth adding a dedicated message queueing system to my stack, or reusing the database (MongoDB)?

Here are some requirements for a queue:

  • Every few days add ~100k tasks with various priorities
  • Workers will pull tasks at typically less than 10 / second
  • Tasks need to be completed ~3 times by unique workers (for error checking)
  • Persistent storage of tasks

Since the task processing rate is quite modest is it worth adding a dedicated message queueing system to my stack, or reusing the database (MongoDB)?

1
source | link

Slow throughput - still worth using a dedicated message queue?

Here are some requirements for a queue:

  • Every few days add ~100k tasks with various priorities
  • Workers will pull tasks at typically less than 10 / second
  • Tasks need to be completed ~3 times by unique workers (for error checking)

Since the task processing rate is quite modest is it worth adding a dedicated message queueing system to my stack, or reusing the database (MongoDB)?