Software Engineering Stack Exchange is a question and answer site for professionals, academics, and students working within the systems development life cycle. It only takes a minute to sign up.
Sign up to join this community
I am working on the application which in near future (hopefully) will have to process tens or hundreds of thousands of items (item is currently one row in relational database table) per second.
If I understood it right NoSQL databases are superheroes of data retrieval thanks to map-reduce magic. But will NoSQL databases provide extra performance in comperatison to relational ones in my case?
100,000's of rows (of reasonable size) is a small enough that it can probably fit in main memory of one or possibly a few compute nodes. It's probably going to be in your best interest to design your application to operate on large chunks of the dataset at a time (shard your dataset by primary key), and use a database to persist the information periodically.
If you really need a higher level of consistency (say, you are doing financial transactions), then you don't have much choice but to use a regular database (relational or document) which provides the necessary integrity and shard your dataset at the application level.
I remember there is a feature in RavenDb that indexes are updated in separate threads. The advantage of this is higher performance of modification operations and disadvantage is that index is not always up-to-date. But I am not sure whether some relational databases have this feature.