What technical approach does NoSQL databases use for fast key-value access

Question

For example,

In NoSQL, technically replication and sharding approach is used for supporting large data.

Was reading this article about NoSQL use cases. It mentions that NoSQL can be used for faster key-value access:

This is probably the second most cited virtue of NoSQL in the general mind set. When latency is important it's hard to beat hashing on a key and reading the value directly from memory or in as little as one disk seek. Not every NoSQL product is about fast access, some are more about reliability, for example. but what people have wanted for a long time was a better memcached and many NoSQL systems offer that.

What technical approach does Key-Value NoSQL databases take to provide faster key-value access?

replication and sharding is to large data

??? is to faster key-value access

Answer 1

The first big difference is that key-value stores require the user to have the entire key for lookup, and can only be looked up by the key. Contrast that with relational databases which can typically be looked up by partial values of any column. This means a key-value store can create a hash of that key and use that hash to determine a precise node and disk location where the record is stored. Relational databases have a much more involved process to locate and retrieve a record, because of inherent extra complexity.

This single key also makes possible a single ordering on disk. Key-value stores typically keep a memory cache, an append-only commit log, then occasionally write out a more permanent compacted and sorted storage to disk. Relational databases don't have a single ordering that makes sense, so they can't take advantage of this optimization.

Most key-value stores also offer the ability to tune the consistency level, so if you have 3 replicas of each record, you can accept a write as complete after only 2 nodes have reported complete, for example, or even 1, if you like to live on the edge and the data isn't critical. All 3 replicas will be written eventually, but the client doesn't wait for it. This is called eventual consistency. Most relational databases maintain absolute consistency at all times, which keeps your data somewhat safer, but also costs speed.

Last but not least, since key-value stores can only be looked up by their keys, users end up trading storage space for speed. Where relational database users might do a join that is relatively slow to execute at query time, key-value users will have written redundant tables in parallel at write time, and can query just as fast as any other single-table query. For example, a relational join of a players and location table would end up as a query using a (player, location) tuple for a key, with completely redundant records to a table containing just the player key.

In summary, key-value stores accept limitations like requiring full keys for lookup, lack of joins, eventual consistency, and need for extra storage in exchange for greater speed.

Answer 2

If you're just looking at NoSQL generally, versus confined to key-value stores, the possible answers increase.

Certainly with BaseX, and I'm sure other XML, or, even some JSON based databases, XQuery provides most of the general functionality of SQL:

The programming language XQuery defines FLWOR (pronounced 'flower') as an expression that supports iteration and binding of variables to intermediate results. FLWOR is an acronym: FOR, LET, WHERE, ORDER BY, RETURN.1 FLWOR is loosely analogous to SQL's SELECT-FROM-WHERE and can be used to provide join-like functionality to XML documents.

But, that's primarily for, at least, XML databases, and explicitly not key-value stores.

Looking at the history of SQL, today's cheap storage, memory and processing would've no doubt changed how that language was designed. For, probably, some very large databases, any XQuery processing might very well be sufficiently fast.