I have some concurrent key-value store implementations that are implemented with hash tables and search trees that I would like to compare. I would like to benchmark them with a real world application where several threads stresses the key-value stores.
I already have some micro-benchmark that stresses the key-value stores by executing random operations on the key-value stores in parallel. What I'm interesting in now is applications that do some work that could be useful in the "real world" and where one or several key-value stores are important for scalability and speed. I would like to have a benchmark that is easy to set up and that could be run on many different system. I would prefer if it does not involve network communication etc.
An example of the kind of applications I'm looking for is the parallel PageRank algorithm. It is used as a benchmark of different key-value stores in the paper "Concurrent Tries with Efficient Non-Blocking Snapshots (PPoPP'12)".
The reasons why I'm not satisfied with "artificial" benchmarks that do X% inserts, Y% deletes and Z% lookups are:
- It can be more convincing with a benchmark that also solves a real world problem. The risk with an "artificial" benchmarks is that they might not correspond to any real world situation.
- Some usage scenarios that happen frequently in real world applications might not be covered by the artificial benchmark.