1

We will implement a system , which will receive quite a lot of requests through web service interface. We will have to queue the requests and process them asynchronously. Consequently we will have to have also some output queue with responses (QR). The web service client will have the possibility to query his (and only his) processed requests in QR using another web service method. Also, client will have the possibility to find the exact response from the QR by providing id of the request (returned after posting the request). Also, client will be abel to delete the request from QR. There will be cca 60 requests per second.

The question is how to implement QR. We need reliability and durability (nothing is allowed to be lost).

Now the question how this can be implemented ... . There are several options, which come to my mind:

  1. Using RDBMS - but this is far from ideal as the performance would not be ideal
  2. employ something like Redis or memcached - having one incoming queue in redis and for each client list with processed items (responses), but i am sceptical about reliability and durability. We can not afford to lose any of the requests or responses.
  3. Some other noSQL solution ?
  • 1
    Not familar with Redis or memcached so can't advise there. However, it sounds like a message queue to me so RabbitMQ might be a good fit. – Robbie Dee Jun 6 '16 at 15:20
  • If your bottleneck is the DB, you can instantly respond with a holding id (using a GUID or similar). The data layer can deal with each in due turn and stick them on the output queue as they're done. The app can also query using the holding id to see where in the queue they are. – Robbie Dee Jun 6 '16 at 15:31
2

I think you're overthinking things. You can use a traditional SQL-based RDBMS. It may or may not be fast enough (although I suspect you're worrying prematurely about optimization), but the only way to tell would be to try it.

Just make sure that you write your code that interacts with the storage system in an abstract-enough way that it is simple to change to another storage method later if your RDBMS turns out not to be good enough.

As an aside, I see nothing in your problem description that is a good fit for a NoSQL database of any kind. You have no non-persistent or easily-reproduced data (so it is not appropriate for a memory-based database). You need strong consistency guarantees (so it is not appropriate for the various systems that let ACID compliance slide in order to get a performance boost). Your data sounds like it is relatively simply structured (so you won't have Object/Relation "impedance mismatch" issues that could push you towards a document store or an object database). It doesn't sound like the relationship between items is more important than the items themselves (so a graph database is unlikely to be the best choice). It sounds like you've picked on NoSQL because NoSQL has a reputation of being more scalable than RDBMSs -- this is sort-of true, but only for applications where it is a natural good fit. Evaluate whether your application uses any of the specific fceatures of a NoSQL solution before just assuming it will create a performance boost.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.