-4

I am lost when reading Design Data Intensive Applications.

In a nondistributed database system, nondistributed transactions are to satisfy ACID.

In a distributed data store, what are distributed transactions to satisfy?

  1. Does ACID or part of it still apply to distributed transactions?

    • Looks like Atomicity, Isolation and Durability still apply.
    • Is Consistency with respect to replication part of requirement on distributed transactions? Note that the word Consistency is overloaded from nondistributed database systems (application-specific) to distributed data systems (application-nonspecific, e.g. linearization consistency, sequential consistency).

  2. Not sure if related to distributed transactions: the book and its author seem to discourage using the CAP theorem.

The answer does not necessarily come from the book. I am also looking for answer from books on distributed systems (e.g. Coulouris or Tanenbaum's books) and database systems (e.g. Silberschatz's book), but I have been too confused to find one.

Thanks.

Improve this question
  • "In a nondistributed database system, nondistributed transactions are to satisfy ACID." Where is this coming from? (please answer by editing the question) – candied_orange Dec 26 '19 at 21:58
  • Distributed transactions satisfy the same need as non-distributed transactions, except that they will work across machine/data store boundaries. – Robert Harvey Dec 26 '19 at 22:09
  • 1
    What does "discourage using the CAP theorem" mean? The CAP Theorem proves that it is impossible to obtain certain desirable properties. You cannot "not use" it. The CAP Theorem is not something you "use", it is a mathematically proven fact. It is an impossibility theorem, just like the Halting Problem. You can't choose to "not use" the Halting Problem either. – Jörg W Mittag Dec 26 '19 at 22:50
  • @RobertHarvey Is Consistency with respect to replication part of requirement on distributed transactions? Note that the word Consistency is overloaded from nondistributed database systems (application-specific) to distributed data systems (application-nonspecific, e.g. linearization consistency, sequential consistency). – Tim Dec 26 '19 at 23:38
1

I think you are looking for "BASE" (Basically Available, Soft state, Eventual consistency). BASE is usually regarded – or least proposed – as an acceptable alternative to ACID for distributed databases.

I like this explanation from ACID vs. BASE:

  • Basically Available: This constraint states that the system does guarantee the availability of the data as regards CAP Theorem; there will be a response to any request. But, that response could still be ‘failure’ to obtain the requested data or the data may be in an inconsistent or changing state, much like waiting for a check to clear in your bank account.
  • Soft state: The state of the system could change over time, so even during times without input there may be changes going on due to ‘eventual consistency,’ thus the state of the system is always ‘soft.’
  • Eventual consistency: The system will eventually become consistent once it stops receiving input. The data will propagate to everywhere it should sooner or later, but the system will continue to receive input and is not checking the consistency of every transaction before it moves onto the next one. Werner Vogel’s article “Eventually Consistent – Revisited” covers this topic is much greater detail.

See also:

About Consistency on the distributed database, from the paper "Notes on Distributed Databases":

Transactions in a distributed environment can archive the appearance that all data is stored as a single copy at a single site. For data which is not replicated is has been shown that a distributed database management system need only obey the rules for non-distributed database management:

  1. Lock entities before using.
  2. Hold all locks until end of transaction.

For replicated data, two additional principles are required in order to archive single site, single copy equivalency:

  1. Updates must be broadcast to all replicas before the transaction ends.
  2. If updates are not immediately broadcast and performed by replicas, then all accesses after an update must be to an updated copy.

It has been shown that these four conditions are sufficient to guarantee the equivalent of a single copy, single site, serial user system.

The linked paper discusses various update strategies and distributed transaction management among other things.

Whatever or not we require all nodes to agree on the transaction to consider it completed is an implementation detail. BASE only requires that the nodes will eventually agree at some point after the transaction completed.

About consistency as in CAP. BASE does not guarantee that the most up to date value will be returned. Only that if no more writes happen and the client keeps reading, it will eventually get the most up to date value. That is, the reads could be looking at a state that lags behind. Yet that state is valid, the atomic nature of transactions is preserved.

The flip side is that BASE does not require the data to be replicated to all nodes to return it either. Depending on the implementation, it could be possible that the most up to date value is returned to a client while it has not been replicated to all nodes.

Improve this answer
  • Thanks. Is Consistency with respect to replication part of requirement on distributed transactions? Note that the word Consistency is overloaded from nondistributed database systems (application-specific) to distributed data systems (application-nonspecific, e.g. linearization consistency, sequential consistency). – Tim Dec 26 '19 at 23:45
  • @Tim expanded the answer. I'm not sure what you mean, yet I hope I covered. – Theraot Dec 27 '19 at 0:22
  • @StackExchangeforAll you know, in a threaded system, whatever value we read from shared memory, we cannot assume it has not changed since, not even one single instruction after the read. I think of these databases in the same terms. Because of that, I believe it is correct to drop consistency from CAP. Whatever value you got from the database, we can assume it could have been changed by another process before the next intruction in our code executes. Forget about pretending you have the most up to date value. – Theraot Dec 28 '19 at 23:32
1

In a nondistributed database system, nondistributed transactions are to satisfy ACID.

Yes.

In a distributed data store, what are distributed transactions to satisfy?

ACID - but over a distributed system.

Example

You have a data message in a queue-system e.g. AMQP that you want to insert into multiple database tables. So there is two systems; queue-system and database-system. If e.g. one of the inserts into the database fails, you want to undo (Atomicity in ACID is this) the whole operation, so the message should still be in the queue-system (not deleted) (Consistency in ACID).

If you read more in the great book Designing Data-Intensive Applications, you will see that "distributed transactions" is usually implemented using two phase commit which is a (not so good - may block the system) special case of consensus algorithm. A better consensus algorithm is Raft (non-blocking). Distributed transactions (using two phase commit) is nowadays avoided and systems with optimistic algorithms like Raft is used instead.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.