Questions tagged [eventual-consistency]
The eventual-consistency tag has no usage guidance.
74
questions
0
votes
0
answers
49
views
Event sourcing, bulk update and storing multiple events vs one event with multiple ids
Event-sourcing often implies to have one row per aggregate id :
event_id
event_type
entity_type
entity_id
event_data
102
OrderCreated
Order
101
{...}
103
OrderUpdated
Order
101
{...}
This is ...
- 51
0
votes
1
answer
184
views
Ordering events across multiple topics in a distributed system
Our platform is growing in the number of events we're raising, and these different events are often related to one another:
ProductCreated
ProductUpdated
ProductTakenOffSale
ProductPutOnSale
...
- 323
1
vote
2
answers
73
views
Phantom read race condition keeping RDBMS in sync with external storage
Here's an interesting scenario, consider a cache with many buckets, and resources that can be shared between buckets:
Bucket
Highest to lowest priority
Foo
A, B, C, D
Bar
B, C, D
Baz
A
In the ...
- 111
3
votes
0
answers
212
views
Microservice architecture but with strong data consistency [closed]
I'm working on a project that has microservice based architecture.
Recently we started facing problems where a single user requests requires updating multiple microservices each having it's own ...
- 139
0
votes
0
answers
47
views
Designing Aggregate Roots with Logic Spanning over Roots
This seems like a common scenario that someone must have solved by now!
I have a setup where a Company may have a set of Default Roles assigned, and Users of that Company should inherit those roles. ...
- 193
1
vote
1
answer
262
views
Internal/external commands in CQRS
I have several questions related to CQRS in eventually consistent systems with a need of resilient Command/Event processing.
Is it OK to have external/public synchronous Commands (triggered by API, e....
- 111
0
votes
1
answer
564
views
Handling asynchronous domain events with multiple handlers & allowing them to fail independently
We're building a service using DDD, CQRS, & Hexagonal Architecture that allows the user to upload a CSV feed for which every row will be transformed and sent to a third party over HTTP.
We have ...
- 111
2
votes
3
answers
251
views
Foreign key check without data duplication in separate microservices
If for example we have a user, project and a task service where a task is assigned to a single user and user can be part of multiple projects - how can one enforce a rule so that one cannot assign a ...
- 21
0
votes
2
answers
207
views
How is cross device eventual consistency achieved in real time?
Assume there is a distributed system which works on the principles of eventual consistency.
Let's consider a scenario where I update my profile picture using my phone. If I log in using another system,...
- 111
-1
votes
2
answers
209
views
Transactional and eventual consistency together -- which operations MUST be transactionally consistent?
I find a lot of resources regarding strong consistency versus eventual consistency, and when to choose one or the other.
I'm wondering how and when you would have both together. Consider this specific ...
- 115
0
votes
1
answer
190
views
Is publishing to the same event store stream from multiple places an anti pattern in a distributed transaction?
Below is a little context to the problem I'm trying to solve.
There are two separate microservices with separate frontends , APIs and databases.
Customer Microservice - Manages customers
Orders ...
- 302
2
votes
1
answer
156
views
How do we maintain consistent-read promise to clients + handling ID collision when using a fallback queue?
In my company, we are using Event Sourcing pattern to implement a storage for all changes to the price of a booking. Across the company, different services might try to append events to a booking ...
- 245
3
votes
2
answers
418
views
Foreign Keys and Eventual Consistency
We have a project that loads metric data from various etls and services into an AWS SQS queue, where that data is processed and inserted into a metrics database, also running on AWS RDS.
When we ...
- 2,343
0
votes
2
answers
649
views
Why is it considered hard to maintain strong consistency in a distributed system?
Why can't you just use strongly consistent reads for all your DB reads, with retries on 500 responses? According to CAP theorem increasing consistency should probably lower availability, but can't the ...
- 421
2
votes
1
answer
711
views
Communication between services in a microservices architecture, is direct communication okay?
I've been reading a lot on why one should use messages (w/ event sourcing/CQRS) for microservices communication and I get it. But most of the stuff I read online state that the main reason for using ...
0
votes
3
answers
1k
views
How to handle data inconsistency in microservice architecture?
Let's assume we have two services - user and auth with a message broker in between. user service handles CRUD action on user entity and auth service handles authentication. When a user is created/...
- 156
3
votes
2
answers
444
views
Multiple sources of truth - Optimistic concurrency & Eventual consistency
I have a couple of web applications that write to their own databases. They also share a few entities, for example, the customer entity. My business case is such that the same field for the same ...
- 39
2
votes
2
answers
845
views
In which database CQRS Command performs a Query?
I read the discussion Should CQRS Command perform a Query?
But I don't understand in which database the query is performed, because I have two distinct databases - one for reads (queries) and one for ...
- 67
-2
votes
2
answers
258
views
How can one choose between consistency and availability if the system you are designing is Tiny URL
I am putting together a system design for something similar to Tiny URL. It provides an API for mapping a URL to a short URL. Once created the short URL can be used to access the original URL. This ...
0
votes
1
answer
133
views
How to deal with second hand reference data in event driven architectures?
What is the best way of dealing with secondary reference data in event driven architectures?
For example, if you have an event from an "Employee" domain that includes some reference data ...
- 166
3
votes
4
answers
4k
views
What are the problems with command handlers returning data in CQRS?
I have a system using CQRS/ES, where:
The command handlers are actually implemented as an application service layer, which executes synchronously. I.e. events are persisted in the event store before ...
- 502
2
votes
1
answer
2k
views
When to publish domain events when handling transactions in the application service
This is something that has been bothering me for quite some time, I tried different approaches over the past year but I always come back to reflect on this, so I'll throw myself out here !
So, first ...
- 447
1
vote
1
answer
139
views
Is weak consistency still relevant in systems that record timestamps?
There are multiple aspects and quality that are desirable for a distributed system. Among these are consistency and availability. According to the CAP theorem, if partition tolerance is a must, the ...
- 133
1
vote
3
answers
676
views
Be sure to send email only once
I'm thinking about "simple" problem. I have database with messages and email adresses to send:
emailAddress string
message string
Sent bool
Is it possible to write a system in which I can be sure ...
- 117
0
votes
3
answers
194
views
Keeping up logical order in a messaging environment
My setting is the following (Please assume that this points are entirely rock solid unchangeable, some for good reasons some: "just because"):
In a super scalable microservice environment i receive ...
- 207
-4
votes
3
answers
567
views
How does a distributed system both tolerate network partition and achieve consistency?
In the CAP theorem, there are only three possible cases:
C and A without P
C and P without A
A and P without C
How can we have both P and C in the second case?
Doesn't propagation of update from ...
- 5,415
-4
votes
1
answer
67
views
Does stale data due to weak level of consistency count as Byzantine failure?
I have difficulty understand Section 18.3 Fault Tolerance Services under Ch18 Replication in Coulouris' Distributed Systems. If my reading and understanding is correct (which might not),
Section 18....
- 5,415
1
vote
2
answers
149
views
Aggregating data between microservices in twitter-like application
I'm developing a Twitter-like app and I have some doubts about my service oriented architecture.
I have a User Service with a REST endpoint POST /users/{userId}/follow
so the "connected" user starts ...
- 727
2
votes
3
answers
2k
views
CQRS - How can a command properly validate when queries are required?
I'm aware that this question has been asked several times, but I have some concerns regarding querying from the write side that I don't see addressed in the already existing questions, more ...
3
votes
2
answers
685
views
How to make cross-service transactions in a microservice architecture? [duplicate]
One question im struggling with is how to make linear, cross-service operations/transactions in a distributed system.
For example, I have two services: Orders and Payments. How would the Orders ...
- 139
0
votes
0
answers
190
views
What is a good design for maintaining consistency in a last write wins database without event sourcing
We have an application which on every resource update(resource update, create, delete)creates a message and puts it on a queue. This message is picked up by a worker process, converted to a format ...
- 894
1
vote
1
answer
735
views
How can an event subscriber catch up with missed events?
At a conceptual level, what ways can a subscriber catch up with events it has potentially missed but needs to know about at a later time.
For example, a service is running and subscribes to events on ...
- 405
1
vote
0
answers
101
views
How are changes populated to FrontEnd in Microservices Event First scenario?
I've been reading recently about Event First approach to microservices where Event Store is a Single Source of Truth for all the applications (https://dzone.com/articles/data-consistency-in-...
- 302
0
votes
1
answer
182
views
Where to store side effect state in an event sourcing system
I also had a look at How do I deal with side effects in Event Sourcing? but the solution wasn't clear to me.
If I store "EmailSent" event in the event stream, I might issue the external request to ...
-1
votes
1
answer
270
views
Is it a bad idea to persist a javascript function?
I'm working on eventual consistency. In short I have a bunch of promises and I want to make sure they get retried until they succeed. It would make my life a lot easier if I could just persist the ...
0
votes
1
answer
41
views
Updating a Bound Context in One Database From A Master Database
Let's say I have two domains: Customers and Jobs.
Customers is responsible for knowing what customers a user has access to. Jobs is responsible for knowing what jobs a user has access to (by ...
- 560
0
votes
1
answer
758
views
How to synchronize databases using event streams?
Say you have a bunch of microservices each accessing dedicated databases. Due to business logic reasons, data from some DBs need to be replicated to other DBs (e.g. to serve as local cache).
One way ...
- 101
3
votes
1
answer
468
views
is Event sourcing using Database CDC considered good architecture?
When we talk about sourcing events , we have have a simple dual write architecture where we can write to database and we can write the events to a queue like Kafka . Other downstream system can read ...
- 139
0
votes
1
answer
611
views
DDD - Handling temporary inconsistency when handling domain events
I realize that DDD advocates eventual consistency, thus allowing for a certain period of time in which the system might be inconsistent. Embracing eventual consistency, we can therefore model our ...
- 386
9
votes
2
answers
10k
views
DDD - Domain events vs application events
I'm considering two types of notifications:
Domain events used internally within a bounded context
Application events used to exchange notification messages between bounded contexts
Each aggregate ...
2
votes
2
answers
67
views
How to implement semantics of serializability in eventually consistent system?
In a distributed asynchronous system serializability cannot be achieved, but we still have to somehow provide the semantics.
Consider quite natural requirement that user must have unique email or no ...
- 1,640
12
votes
4
answers
6k
views
How to handle transactional operations in an event-driven architecture?
I'm trying to flesh out an eCommerce system using microservices (.NET Core and Kubernetes), event sourcing (Kafka), and CQRS. The particular use case I've been thinking about is as follows.
There is ...
- 1,344
2
votes
0
answers
210
views
UI syncing in case of bulk operation api
In a CQRS system, the convention that we follow is we render the detail view of an aggregate(i.e getById) using the write model and views containing multiple aggregates(like a dashboard) using read ...
3
votes
0
answers
431
views
How to design a sync api exposed to third party which involves a business transaction across aggregates?
Business Domain
In the Sales domain, I have 3 entities - Lead, Contact and Opportunity. A lead is injected into the system from many sources and nurtured. If it is a prospect then it is converted ...
3
votes
3
answers
246
views
How are new bounded contexts deployed, and bugs fixed, in message-driven "autonomous" microservice architectures?
I'm coming from the monolith background, using a single large relational database. From my research, many proponents of the microservice architecture favor the event-driven rather than REST driven ...
- 209
0
votes
1
answer
831
views
Dealing with eventual consistency when persisting and publishing events
Currently working on a DDD application that is using event sourcing with redis as my main persistence store. So unfortunately I do not have built in rollbacks should something fail. The application is ...
- 103
3
votes
3
answers
916
views
Strategy for ensuring consistency when integrating using an API?
How do I ensure consistency between two separate systems which store and maintain the same set of data, when the method of communication between them is REST APIs and webhooks? Is it possible?
I find ...
- 159
5
votes
2
answers
487
views
How is Eventual Consistency (EC) better than Strong Consistency (SC) when N = 3, R=W=2 when SC gives better consistency gurantee in same config?
In dynamo paper, it says that in production most of the users set N = 3, R = 2 and W = 2.
The common (N,R,W) configuration used by several instances of Dynamo is (3,2,2). These values are chosen to ...
- 251
0
votes
1
answer
1k
views
CQRS, What reasons cause read model update delay? How to prevent user from taking inconsistence data?
I am considering CQRS for building a realtime game server (not gameplay but the economy).
For some reasons. Scaling, Event logging, Model changing.
However in realtime game when player upgrade, for ...
- 687
7
votes
3
answers
2k
views
DDD: deciding when to lean towards eventual vs transactional consistency
I am reading Vaughn Vernon's series of articles about effective aggregate design.
On the subject of deciding between transactional vs eventual consistency, it states the following:
Discussing this ...
- 2,764