Questions tagged [distributed-system]

This tag can be used by questions on distributed system concept, design, and implementations.

38 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4 votes
2 answers
194 views

How do replicas of an app instance assure the task was done only once when the main instance fails?

Let's assume i app have with basic flow: Client send request to server Server do some business logic Server puts message into some message queue Server returns success to the client microservice A ...
Johnyb's user avatar
  • 407
3 votes
0 answers
92 views

Difficulty understanding how federation increases cache performance for databases?

I am studying system design for distributed systems and in this page (https://github.com/donnemartin/system-design-primer), one of the following advantages was mentioned for federation for databases ...
Henry Zhu's user avatar
  • 153
3 votes
0 answers
671 views

System Design: Chat application in a distributed system context

I am currently working on a side project so I can learn new technologies and understand how the components fit together. For this project, I decided that I wanted to work on Chat Applications in the ...
Filipe Costa's user avatar
3 votes
0 answers
55 views

Release Phase in Lamport's Mutual Exclusion Algorithm

I was studying Lamport's mutual exclusion algorithm from the original paper, and noticed a difference in the release phase to the one given in Wikipedia. In the original paper, author states that : ...
Varun Hegde's user avatar
3 votes
0 answers
66 views

Sharing tasks among parallel independent jobs

I am getting files on HDFS and I have a legacy java app which checks for new files every 15 mins, does some transformations and write them to some remote server. Now, I am facing scaling issue with ...
banjara's user avatar
  • 219
3 votes
0 answers
84 views

Architecture and implementation of a de-centralised web service

I'm a web programmer with several years of experience in RoR and node.js. I have a project idea that is quite big and I need some input in how it can be achieved. I heard of the Diaspora project, and ...
Lanbo's user avatar
  • 291
3 votes
0 answers
822 views

Vector clock conflict resolution in event sourced system via local timestamp - is this safe?

We have an occasionally connected, event sourced system: a budgeting application where each client can pull other client's event store from a separate storage (network share, dropbox,...). It is our ...
urbanhusky's user avatar
3 votes
1 answer
150 views

Best practices for managing software releases with CI/CD with distributed software components running on a customer on-site machine?

I'm looking for some resources on learning how to best manage our software releases in an efficient manner, as our current method slows release schedules down. My current team produces and administers ...
Thomas Smith's user avatar
3 votes
1 answer
165 views

Database consistency for a distributed system

I'm working on a phone call monitoring project. The aim is to have one row in the calls table for each ongoing call. Each call instance may be updated by many different servers/threads as more ...
widavies's user avatar
  • 141
2 votes
0 answers
216 views

Automatically extended database-based mutex lock - is it a good idea?

Since I am working in a multi-instance microservice environment, I came accross a problem with making some operations being performed at most by one of the running instances at once. The solution ...
PJDev's user avatar
  • 129
2 votes
0 answers
174 views

SAGA pattern - how to handle situation when some message is not received by orchestrator

Recently I'm studying Saga pattern, but I'm struggling to imagine how it will work in real system. Standard example is making an order, which consist of reserving flight, hotel and a car. We can ...
Maciej Pszczolinski's user avatar
2 votes
0 answers
127 views

Distributing tasks to several clients

I describe a very very simple scenario. I try to develop a system where many machines can help another one (server). I tried RabbitMQ and Redis, but both don't fulfil all requirements. I have 1-...
Daniel Stephens's user avatar
2 votes
1 answer
445 views

Design question on synchronization of two asynchronous data streams

I have two async streams suppose- Trip : {tripId, date, city} Bill : {billId, tripId, date, amount}. I need to design a system to get real time aggregated view of following nature: City, TripCount, ...
Himanshu Verma's user avatar
2 votes
0 answers
185 views

Testing an algorithm for safety in mutual exclusion

I am attempting to write a code to test Lamport's Mutual Exclusion algorithm for safety as a correctness measure. I am running the alogrithm on a single core cpu machine with multiple processes ...
Varun Hegde's user avatar
1 vote
0 answers
158 views

How does Stackoverflow comment system work

When someone comments on my question/answer/comment I will see the notification from the inbox, and it also shows under my response tab. Later when the comment is edited, the comment shown under the ...
MLEE's user avatar
  • 111
1 vote
0 answers
28 views

Do Multi-Primary replicated distributed systems guarantee sequential consistency?

I know that Primary-backup replicated distributed systems guarantee sequential consistency. My question is whether multi-primary systems also achieve that. I mean if they use consensus (i.e: Paxos ...
Omar Hossam Ahmed's user avatar
1 vote
0 answers
56 views

Byzantine Failures - Client Voter and Replica on Same Processor

I need help understanding the following text from section 4.2 on this paper: Implementing Fault-Tolerant Services Using the State Machine Approach: A Tutorial Dependent-Failures Output ...
KJP's user avatar
  • 117
1 vote
0 answers
763 views

In a distributed system, how to prevent duplicated services from acting multiple times on a common data source?

We are currently rewriting our infrastructure from a monolith to a distributed system because the old system didn`t fit the workload anymore. In one part of our distributed system, we have an Entity (...
looper's user avatar
  • 191
1 vote
0 answers
103 views

Scaling Systems With Shared Data

I'm working on an in memory on disk cache and am starting to think about strategies for scaling the system. At the moment the system persists data by creating a file for each key value pair, where the ...
rykeeboy's user avatar
  • 121
0 votes
0 answers
35 views

Handling competing requests for the same "id" in different Kubernetes PODs

I have the following problem, I am implementing a delivery management service and a single order can be competed between N deliveryboys, this service is being executed on Kubernetes with 5 PODs of ...
Carlos Rodrigues's user avatar
0 votes
1 answer
239 views

Capture a time-ordered sequence of item-level modifications via events when using Amazon Aurora

I'm designing a small platform based on a series of event-based micro-services. The persistence storage I'm targeting is (the managed) Amazon PostgreSQL (Amazon RDS for PostgreSQL) — although I can go ...
x80486's user avatar
  • 119
0 votes
0 answers
46 views

How to handle multi-region writes for a joint user registration?

UserA and UserB are signing up for some joint service together. They both have to independently agree to T&Cs before sign-up is complete. Once complete they both receive an email to sign-in. A 1 ...
mockitodorito's user avatar
0 votes
0 answers
63 views

What is the best way to desgin my api workflow?

I am building a bank management system using the microservices architecture. I have many microservices but I am going to talk about two. I have customer account service and transaction service. Let's ...
ferrocene's user avatar
0 votes
0 answers
481 views

Dealing with third party api rate limit when using multiple services

I have 1 api key that has a rate limit 1000req/1min. I would like to somehow use 50% of that limit in service 1 and 50% of that limit in service 2. Let's say they're just 2 separate processes using ...
Konrad's user avatar
  • 1,559
0 votes
0 answers
143 views

Load balancing the load balancers

Currently I have a system where I have installed HAProxy on one machine and my other 3 machines serves the webapps and the fourth machine for the database. Now I need to add another load balancer in ...
Himanshuman's user avatar
0 votes
0 answers
89 views

Realtime report generation and downloads

We host a dashboard where users can view different reports, each powered by a templated SQL query. The users have an option to "download" those reports as CSV. Currently the above use case ...
phraniiac's user avatar
  • 101
0 votes
0 answers
95 views

Experiences with coarse grained and fine grained message designs in an ECST pattern

I am currently facing a dilemma in the design of a message oriented distributed system following an Event Carried State Transfer pattern. This is essentially a "near real time ETL" used to ...
allmhuran's user avatar
  • 306
0 votes
0 answers
537 views

log centralization and central aggregation system design

I'm thinking of how would I go about creating my own logging centralization and centralized aggregation system. Imagined system has 10 - 100 nodes. Each node runs a LAMP(Linux, Apache, MySql, PHP) ...
Dan's user avatar
  • 109
0 votes
0 answers
38 views

Turn based access to update shared resource such as inventory

I am trying to design a system where a Product can only be sold while inventory is available. This is an e-commerce platform where users can buy specific products with designated inventory. A similar ...
Anup Marwadi's user avatar
0 votes
0 answers
198 views

What is a good design for maintaining consistency in a last write wins database without event sourcing

We have an application which on every resource update(resource update, create, delete)creates a message and puts it on a queue. This message is picked up by a worker process, converted to a format ...
m_d_p29's user avatar
  • 894
0 votes
0 answers
50 views

How should the interaction between the MIS instances of a large system happen?

Overall situations: The plan was to build a large system (Management Information System or MIS type) to be used throughout the country, by say 100,000 users daily. The team decided to split that into ...
Musa Haidari's user avatar
0 votes
0 answers
54 views

How to arrange synchronicity reporting in an eventually consistent, distributed system?

Context: A typical corporate landscape with many systems located in different data centers, with firewalls, without uniformised interfacing. Some data X is being replicated for availability and ...
Lord_Farin's user avatar
0 votes
0 answers
43 views

Do web companies use the master replica in data center nearest to the user to do writes?

I am interested how multi datacenter (multi homing in Google terms) systems handle writes to strongly consistent user data? Is the master replica for each user placed in the same datacenter and writes ...
TohaSt's user avatar
  • 1
0 votes
1 answer
457 views

Regarding data modeling in a NoSQL use case

I am reading the following link to learn about system designs of various systems. (This is a paid link, I am attaching all explanations below.) In an attempt to explain the system design of Instagram ...
amiageek's user avatar
0 votes
0 answers
148 views

How to implement a fair queue, for multitanent, limitation per machine and limitation cross cluster?

I'm maintaining an already built system. The current system operates as following: Current situation: Client (UI) submits a request. Web layer round-robins to one of backends. Backend takes request ...
Jas's user avatar
  • 507
-1 votes
1 answer
149 views

Checking a related aggregate existence is Domain concern?

This is my specific situation: I am following DDD and Event sourcing to implement my application (using C#). I have a Domain layer that contains Aggregates' implementation. I write unit tests for this ...
Mehrdad's user avatar
  • 99
-1 votes
1 answer
70 views

How to design a topped-up internal balance for a replicated API?

My product is an API which charges an amount of USD every N requests. I would like to allow my users to top-up their internal balances through PayPal, then decrease their balances when they use the ...
Sony's user avatar
  • 99
-1 votes
1 answer
57 views

Whats the pros and cons of RPCs vs. HTTP in information exchange in microservice architectures?

What type of components could benefit from RPC over HTTP, vice-versa? Does there exist other information exchange mechanisms, what are their use? I've seen smart people use both, I'm sure for ...
heretolearn's user avatar