60
votes
No central database
Plenty of sensitive information gets stored in databases. In fact, a central database is probably the most secure way to store this data. Large enterprise databases have tons of functionality to do ...
- 12.7k
41
votes
Is "releases mutexes in reverse order" required to make this deadlock-prevention method work?
For a deadlock (more specifically, a circular wait) to occur, there needs to be a circular chain of n ≥ 2 mutexes (or other exclusively lockable resources) R1, R2, …, Rn such that, for each k from 1 ...
- 1,729
38
votes
No central database
You need to back up a couple steps and, in consultation with your client, work out a threat model. (Yes, that's a link to a 600-page book; yes, I am seriously recommending you read the entire thing.)
...
- 2,620
24
votes
How to justify using await instead of .Result() or .Wait() in .NET Core?
You have missed the point of async/await
It wasn't created to frustrate developers with deadlocks. It was created to free up Processing power while we wait for a non CPU based task to complete.
The ...
- 72.5k
17
votes
Best way to synchronize data between two different databases
This is pretty much what I have been doing or a living the past few years, and my gut instinct is that the time to read 500,000 items from the source database and sync in the destination will not take ...
- 1,323
14
votes
Why do we need read locks in read-write locks?
A read lock allows multiple concurrent readers of some data, but it prevents readers from accessing the data while a writer is in the middle of changing it. That ensures that a reader will never see a ...
- 73.3k
13
votes
Is "releases mutexes in reverse order" required to make this deadlock-prevention method work?
For a deadlock to occur a system has to have several properties simultaneously. Wikipedia has some more details on this but for short:
Mutual Exclusion
Incremental Acquisition
No preemption
Circular ...
- 16.1k
12
votes
What are some approaches used in financial applications to prevent deposit/withdrawal sync problems?
Transactions
A transaction wraps all of the required steps for a particular business operation and guarantees that either all of the steps succeed or they all rollback to the original state in the ...
- 199k
11
votes
Is "releases mutexes in reverse order" required to make this deadlock-prevention method work?
Let's have a look at the simplest form of deadlock: the kiss of death of two processes trying to acquire 2 mutexes:
(1) | (2)
Lock mutex A (success) | Lock mutex ...
- 75.9k
9
votes
Accepted
Data from devices with unreliable clocks
There's actually quite a bit you can do to recover something close to the actual time of most of your events.
Android gives you a few useful tools to work with, notably broadcast intents sent when ...
- 20.3k
8
votes
No central database
One option to do the synchronization would be to do it peer-to-peer. This will still require a central server, but that server won't handle any of the data.
When a device goes online, a central ...
- 23.3k
8
votes
Do critical section and atomicity imply each other?
Atomicity should be understood to mean that the thing it refers to (a transaction, an operation, etc.) is all-or-nothing. There is no defined mechanism to observe partial results.
For a database ...
- 73.3k
7
votes
Why do we need read locks in read-write locks?
An update lock is an exclusive lock, that is, only one lock can be held at a time. It is used to prevent concurrent updates.
A read lock is a shared lock, multiple readers can held a read lock ...
- 12.3k
7
votes
What will you do if multiple users access your application at the same time?
This is a really broad question, since concurrency issues are handled in a variety of ways in complex system. For example a synchronized block in Java is only locked for other threads in the same ...
- 57.9k
6
votes
Accepted
How to resolve data-binding efficiently
This is either fundamentally meaningless or you've left out an important detail.
Your diagrams give each node 3 states:
● Unchanged, don't propagate
◌ Just now changed and in need of propagation
○ ...
- 105k
6
votes
What are the existing theories or methodologies around synchronizing tree structures?
The main challenge/consideration when doing this is determining what you consider a difference to be. This can be more tricky than you might think. The experience I had that lead me to that ...
- 26.1k
5
votes
A simple approach for data synchronisation
Syncing between multiple clients is a very complext task.
Let's not call it syncing, let's call it bash transaction processing.
Let's get rid of "last server update timestamp", let's just every ...
- 39.4k
5
votes
No central database
Make it somebody else's problem.
Store the data locally in each app, then give users the option to enable synchronization using their own account with a third-party service (Dropbox, Google Drive, ...
- 159
5
votes
Bi-directional sync of 2 different APIs
Welcome to the world of integration. The hardest thing about it is how poorly most vendors understand what you need and the resulting gaps in their interfaces.
In this case you seem to have a really ...
- 26.1k
5
votes
Accepted
Best practices for app-cloud synchronization of database representation
You are trying to create a distributed eventually-consistent system. Those are inherently complicated. It's understandable that you are having difficulty coming up with a good solution because there ...
- 134k
5
votes
When does a *fixed delay* make sense?
Yes. To delay to give time for completion prior to checking.
Yes. To animate, because humans see in the order of ~80hz.
Yes. To throttle, running at 100% isn't always the right option.
No. There is ...
- 16.1k
5
votes
Why do we need read locks in read-write locks?
Presuming a write cannot be done atomically:
Only one thread at a time can safely write data without corruption.
No reads can happen during a write without getting corrupt data.
Multiple reads can ...
- 1,975
5
votes
Hexagonal/Clean Architecture - threads, locks, synchronization
Generally the best place to put all the multithreading stuff is "as far away from you as possible".
That's a joke, but also a very serious point - it's hard to get right, so most architects ...
- 12.7k
5
votes
Accepted
One producer - multiple consumers queue, best way to guard against rare duplicates?
[As discussed in comments: OP is moving towards an actual queue implementation - a genuine single-producer multiple-consumer pattern.]
Here's one approach based on a queue:
Have a map from underlying ...
- 732
4
votes
Conflict resolution for two-way sync
I guess there is no standard way to do it, each system uses their own policies for the conflict resolution.
I made some simulations using two devices, computer and phone, and Google Spreadsheet to ...
- 141
4
votes
What's the best implementation for offline mobile app synchronization?
If you're going to sync against your master SQL database you will always have the same problem: how do you resolve conflicting changes?
CouchDB doesn't help you there. It explicitly does not handle ...
- 12.9k
4
votes
Accepted
Threads - When to use barrier?
Barriers are used to make multiple threads within the same program do something in a synchronized fashion.
The real-world analog would be the starting gate at a horse race. Ten horses in a race won'...
- 20.3k
4
votes
Sync one thousand vector3 across network with minimal bytes?
Standard compression algorithms are pretty efficient even at the fastest setting. As a quick test I filled a text file with 34k bytes. Compressing with 7-zip on fastest setting gave me a file of 1k. ...
- 5,118
4
votes
Are there any techniques for detecting redundancies a stream of changes to a filesystem?
All you have to do is buffer your structured events such that you can look them up by affected path (source and destination, probably).
The actual mechanism for combining any given pair of events ...
- 12.6k
4
votes
Synchronization of data across microservices
I would take a look at one of Microsoft's newer projects code named "Ambrosia" (link will take you to their Github page where the project is being developed open source) which focuses on providing a ...
- 142
Only top scored, non community-wiki answers of a minimum length are eligible