28

Based from what I've understood about eventual consistency, all these services (consumers) will receive the event at the same time and process them separately which, in a good scenario, will lead to data being consistent. No, not necessarily. As I commented, we can't undo a sent email, so we still need a sort of "sequence". IPC over event-driven data ...


15

Consider non-functional requirements when designing your functional tests -- if your service has a non-functional requirement of "Consistent within x (seconds/minutes/etc)", simply run the PUT requests, wait x, then run the GET requests. At that point, if the data has not 'arrived' yet, you can consider the PUT request to be non-conformant with your ...


14

What is "eventual consistency"? How does it compare to "transactional consistency"? When does it happen? Consistency models describe how a system (nominally a distributed system) responds to change. In an eventually-consistent system, all nodes will eventually have a consistent view of the overall system state. However, there will be a period of time after ...


12

What makes updating multiple Aggregate Roots in one request/transaction such a bad practice? The problem is the other way around - trying to modify multiple aggregates in a single transaction is an indication that you haven't modeled your aggregate boundaries correctly. Put another way: modifying two different aggregates in the same transaction introduces ...


12

What I'm not super clear on is why you would ever rehydrate your Aggregates from the Event Store itself. Because the "events" are the book of record. If projecting changes to "read" databases is so easy, why not always project changes to a "write" database whose schema perfectly matches your domain model? This would effectively be a snapshot database. ...


7

You really want your tests to be fast and consistent. If you start creating tests that may occasionally fail due to eventual consistency, you'll ignore the test when it fails, and then what use is it? Create a fake service which handles the PUT and GET requests, but has an additional operation to make it consistent. Your test is then: datastore.do_put(...


6

After PUT, retry GET N times until success. Fail if no success after N tries. Sleep between PUT and GET Unfortunately, you have to pick magic values (N or sleep duration) for both of these techniques.


6

In a traditional N server <-> 1 RDBMS style system, the database is used as a central point of synchronisation which helps prevent such inconsistencies. In event sourced systems, the "event store" serves the same role. For an event sourced object, your write is an append of your new events to a particular version of the event stream. So, just as with ...


6

In your case you cannot just process all three things at once. What you need is a process. Here is an extremely simplified example: It is important to know that state altering operations MUST be always made on a consistent entity. Unless you can guarantee strong consistency, it has to be made on a master record. Your system should guarantee that before any ...


6

Since you don't specify what would the purpose of the "write" database would be, I will assume here that what you mean is this: when registering a new update to an aggregate, instead of rebuilding the aggregate from the event store, you lift it from the "write" database, validate the change, and issue an event. If this is what you mean, then this strategy ...


5

I don't think this is about atomic-operation, synchronization, asynchronous-programming, or transactions. I think this is about meaningful state. Right now you have three possible states. The object's uploaded field is true, it's false, or the objects record doesn't exist. The question then is what do these three states mean? How do we arrive at them? ...


5

Event storage Event storage will depend on your application but it is a lot easier if you use a flexible storage for your events (so a NoSQL DB, JSON or XML). How you deal with updates will depend on several factors you need to take into account such as requirements for availability and timeframes for updates, amount of events, etc... Update your events ...


5

The meanings are slightly different I think. In short: Consistency in ACID means that no dataset may be an invalid state or represents data which are semantically invalid after a transaction is committed ("internal consistency"). Consistency in CAP means that after a transaction is executed this dataset must be updated in all replications too.


5

In my experience, most questions about transactionability and microservices are caused by the following two reasons: The transactional data is placed in different microservices: This is wrong by definition. Data that should be modified transactionaly belongs to the same service. Grouping the right data in the right service is very tricky and you need to put ...


4

OK, so. "What Are You Testing" is the key question. I am testing my internal logic of what happens assuming the google stuff works In this case you should mock the google services and always return a response. I am testing my logic can cope with with the transient errors I know google will produce In this case you should mock the google services and ...


3

Just change your point of view: interpret the changing of the uploaded field to true as part of the upload operation. If a failure occurs because the upload failed or the application crashed between the moment the file is uploaded and the local database is updated does not matter. I guess in most real-world cases the second situation will occur so seldom ...


3

Update: Can etags + If-Match work? Yes, I'm sure it can be made to work. Your client will have to interpret the 412 error as an optimistic concurrency failure. As far as using it with POST, the spec does not restrict which methods it is allowed on. The If-Match request-header field is used with a method to make it conditional. I would also return 412 (...


3

I would say that the command interface isn't supposed support queries, and the write model should not necessarily be exposed to client applications for that purpose. However, the notion you're looking for in the first question: is it conceptually wrong to query write model in order to get the most recent state of an entity for the page? seems to be "...


3

A little remark, and then a tentative answer. Use eventual consistency between aggregate's boundaries (before asking whose job it is) That's aggregate design by definition. We might argue whether this is a good or bad decision, but the Aggregate in Domain-Driven Design is a "unit of consistency". This is not to say that this is "the right way to do ...


3

From everything I've read, here and elsewhere, the reason seems to be that changing multiple aggregates in one transaction creates the requirement that they are stored on the same database host. (Let's not even consider two-phase commit.) This introduces a trade-off, one that makes this a consideration rather than a rule of thumb. Is your bounded context ...


3

The main reason is performance. You could store a snapshot for every commit (commit = the events that are generated by a single command, usually only one event) but this is costly. Along the snapshot you need to store also the commit, otherwise it would not be Event sourcing. And all this must be done atomically, all or nothing. Your question is valid only ...


3

I'm wonder how much read model delayed? What reason(s) cause it for delay? If I have only little preprocessing process data for read model. I have no experience at scale. "It depends." The cause of the delay is the propagation of the information from the data structure that supports the write to the data structure that provides the read. So some of the ...


2

Conceptually wrong? I'd say: "Yes!" Looks like you're willing to do CRUD. Nothing intrinsically wrong with that. But it's conceptually not CQRS. If you just need CRUD to solve your problem, just use CRUD.


2

First thought: make the implicit explicit. The traveler is offering your an opportunity to derive business value; your absolute top priority at that point is to capture that opportunity -- everything else can wait. So you are missing a ReservationRequested event; choose whatever spelling is appropriate in the ubiquitous language of your domain. This ...


2

Sounds like you could implement a business process (saga in context of Domain Driven Design) for the user registration where the user is treated like a CRDT. Resources https://doc.akka.io/docs/akka/current/distributed-data.html http://archive.is/t0QIx "CRDTs with Akka Distributed Data" https://www.slideshare.net/markusjura/crdts-with-akka-distributed-...


2

Queuing systems are not quite as fragile as you might think. If we were writing all three processes to a relational db, we might use a transaction to handle a mid processes failure. Without the final commit the partial work would be discarded. In a queue bases system you will have a similar options when you read a message from the queue to handle mid ...


2

Regardless of what kind of channel or medium of communication exists between two interfaces, when the producer modifies the format of the message that consumers know how to accept, there is a risk of change on clients. The whole idea of using a AMQP broker is to be sure that the services can communicate with each other without them knowing each other. So ...


2

What you're facing here is the two generals problem. In essence: how can you be sure a message is received and a response to that message occurs? In many cases, a perfect solution does not exist. In fact, in a distributed system it is often impossible to get an exactly-once delivery of messages. A first obvious remark is that the service that changes the ...


2

Generally you take a snapshot in time of existing data and either process that snapshot via an external process or by replaying the snapshot’s messages over the bus/queue. Practically, you’ll need to account for out of order messages, as well as duplicate messages because the snapshot and normal message stream need to overlap a little to ensure there’s no ...


Only top voted, non community-wiki answers of a minimum length are eligible