Questions tagged [distributed-system]

This tag can be used by questions on distributed system concept, design, and implementations.

Filter by
Sorted by
Tagged with
0 votes
1 answer
93 views

Architectural decision regarding at-most-once semantics cron jobs in distributed systems

please note that while I refer to a specific web application framework in the following, the problem also arises with most other web application frameworks I know so please don't be afraid to reply ...
dfsg76's user avatar
  • 109
0 votes
2 answers
161 views

Fault tolerance in aggregated distributed state

I have a scheduling system that is horizontally scaled, and stores shared state in a redis key. The purpose of the system is to implement something similar to classic rate limiting, but a bit ...
NadavShmayo's user avatar
1 vote
2 answers
104 views

Generating unique number in distributed environment

I have little app that i plan to run behind load balancer, with up to 5-8 instances. Inside the app has service, that needs to generate unique number that has up to 10 digits ( so 1-10 digits) each ...
Darlyn's user avatar
  • 209
1 vote
2 answers
88 views

What is the starting point of a geo-distributed server farm?

Scenario: A web application that wants to implement geographically high availability (HA), has multiple web servers in different cities around the globe. Each server is connected to a database in the ...
Parsa99's user avatar
  • 121
1 vote
1 answer
123 views

App that can be deployed both as monolith and microservices?

We have a monolith C++ application and I consider doing some major refactorings. Since we're not completely ready to go full "microservices only" (maybe in about two years time), I want to ...
glades's user avatar
  • 377
0 votes
2 answers
245 views

How to effectively scale one microservice to multiple instances

Let’s say I have a booking microservice (single instance) with its own DB. If I want to horizontally scale the service with multiple instances(exact same copy of the business logic code) running on ...
dig_123's user avatar
  • 137
-2 votes
1 answer
61 views

Rate limiting design

I am designing rate limiting in my application in which the rules can be dynamic. I have a basic design in which every request is validated against the rate limiting quota, which is stored in redis. ...
Sandeep's user avatar
  • 199
4 votes
2 answers
194 views

How do replicas of an app instance assure the task was done only once when the main instance fails?

Let's assume i app have with basic flow: Client send request to server Server do some business logic Server puts message into some message queue Server returns success to the client microservice A ...
Johnyb's user avatar
  • 407
1 vote
3 answers
255 views

How to synchronize cache writes in a distributed system?

I am looking for a complete pattern implementing cache-aside when used in a distributed system across multiple nodes read/writing from/to cache. Specificly how to avoid making multiple requests to db ...
eran otzap's user avatar
1 vote
3 answers
446 views

What algorithm can I use in order to not let there be two rows with same "mobile number" in database?

The issue that I'm facing is on a banking app. There are two ways to register for internet banking. A) Self Register B) Ask for bank to register via their GUI panel backend. Consider this scenario: ...
tuck_y's user avatar
  • 135
-2 votes
1 answer
96 views

SERVICES in sending ENTITIES in distributed systems

Here is a passage from the Blue book: Distributed system architectures, such as J2EE and CORBA, provide special publishing mechanisms for SERVICES , with conventions for their use, and they add ...
cool_cat's user avatar
1 vote
1 answer
231 views

How does a flow of architecture of a digital banking system looks like?

I am working at a digital banking company i.e company that provides software for banks so that the customers of that bank can use banking services via smartphone. But the documentation there is non-...
tuck_y's user avatar
  • 135
-1 votes
3 answers
291 views

Avoid race condition using RDBMS transaction

I am working on design of a distributed system which will process some events. For simplicity lets say, multiple instance of same service will be consuming from same queue, every message will have a ...
Siddhartha Sadhukhan's user avatar
10 votes
3 answers
2k views

How do distributed databases follow unique constraints?

Lets say i have an application where user can register, and the username has to be unqiue value. Now lets say i have N partitions and for each partition i have M replicas with multiple leaders. Now i ...
Johnyb's user avatar
  • 407
1 vote
1 answer
590 views

GRPC over internet

I want to build a grpc server which will be run by a company (service provider) and multiple client each run by a different company (a client). I thought of this instead using a Rest API to get ...
karim alpha's user avatar
3 votes
1 answer
162 views

Best practices for managing software releases with CI/CD with distributed software components running on a customer on-site machine?

I'm looking for some resources on learning how to best manage our software releases in an efficient manner, as our current method slows release schedules down. My current team produces and administers ...
Thomas Smith's user avatar
1 vote
2 answers
77 views

Phantom read race condition keeping RDBMS in sync with external storage

Here's an interesting scenario, consider a cache with many buckets, and resources that can be shared between buckets: Bucket Highest to lowest priority Foo A, B, C, D Bar B, C, D Baz A In the ...
danielrs's user avatar
  • 111
0 votes
2 answers
98 views

How to handle the consumption of multiple event types when timing is crucial?

I have a distributed platform which allows customers to make purchases, and the items which are purchased are stored in an inventory: Sales app -> PurchaseEvent -> Inventory app The Sales app ...
FBryant87's user avatar
  • 323
2 votes
2 answers
90 views

Should a mobile app keep an 'outdated' version of the data and refresh it from database or always fetch from database?

I'm building a mobile, in which the user is able to create,modify and delete entries in a database. There are multiple screen where some of the entries are displayed: (here are some examples, not ...
Corentin Aulagnet's user avatar
0 votes
1 answer
113 views

Redux Pattern in a distributed system?

I would love an opinion on system design in a distributed system that operates under the same domain. I have a bunch of microservices following the Database per Microservice pattern, and updated via ...
user3353167's user avatar
0 votes
1 answer
740 views

Many-To-Many relationships in a Microservice architecture and if the services are correctly identified

I am creating a distributed chat system with a microservice architecture where a user can log in and chat with another user. The services that I identified is a conversation service that handles chat-...
thetruth's user avatar
0 votes
1 answer
81 views

Should I create access control as a microservice or keep the access control logic as part of the specific microservice?

I was wondering if I have a microservice called, for example, Weather, and only certain people can update certain properties around the weather such as Temperature, Precipitation, and Wind. These ...
David King's user avatar
1 vote
1 answer
209 views

Determine when distributed and parallel process is completed

In a distributred system that uses parallel processing to complete tasks and I want to determine when all sub tasks have been completed, what design method or principals can i apply ? I am using ...
Manse's user avatar
  • 149
0 votes
0 answers
36 views

Handling competing requests for the same "id" in different Kubernetes PODs

I have the following problem, I am implementing a delivery management service and a single order can be competed between N deliveryboys, this service is being executed on Kubernetes with 5 PODs of ...
Carlos Rodrigues's user avatar
0 votes
1 answer
253 views

Capture a time-ordered sequence of item-level modifications via events when using Amazon Aurora

I'm designing a small platform based on a series of event-based micro-services. The persistence storage I'm targeting is (the managed) Amazon PostgreSQL (Amazon RDS for PostgreSQL) — although I can go ...
x80486's user avatar
  • 119
0 votes
1 answer
77 views

How to design a fifo coupon distribution system

I have an app written in java and using mysql as DB. I need to create a coupon distributed system where I have 100 coupons stored in DB and the first 100 customers who login should get those coupons. ...
dev Joshi's user avatar
  • 109
3 votes
2 answers
197 views

How to propagate seldomly changing data in a distributed system

We store data – some kind of metadata like car manufacturers or types of computer parts – in one of our applications. This data changes rarely. Let's say it changes once every two weeks. This data is ...
Harold L. Brown's user avatar
3 votes
0 answers
221 views

Microservice architecture but with strong data consistency [closed]

I'm working on a project that has microservice based architecture. Recently we started facing problems where a single user requests requires updating multiple microservices each having it's own ...
JavaLearner's user avatar
0 votes
0 answers
46 views

How to handle multi-region writes for a joint user registration?

UserA and UserB are signing up for some joint service together. They both have to independently agree to T&Cs before sign-up is complete. Once complete they both receive an email to sign-in. A 1 ...
mockitodorito's user avatar
4 votes
3 answers
381 views

How to think about system design questions methodically?

I'm working on my first big system design project at work for the first time in my career. In the past, while i've worked on large systems, I was always working with someone else, so I wasn't really ...
Jeremy Fisher's user avatar
2 votes
1 answer
476 views

Reprocessing out of order events in a stateful stream

Say that you have a monitoring application that reports downtimes for a large number of remote systems (think IOT). A monitoring daemon polls the remote system and reports that status (on/off) via an ...
bangnab's user avatar
  • 746
0 votes
1 answer
54 views

Queue system with priority on Consumers

Imagine a queue system with different consumers - some are low cost whist (like 'reserved/spot' instances in the AWS world) others would be high cost. The system should ensure the messages are ...
Ryan's user avatar
  • 320
0 votes
1 answer
98 views

Operating two independent and competing clusters?

My organization has a redundant setup across 2 data-centers. During normal operations both data-centers are up and handling workloads. For whatever reason, the infrastucture team has decided to ...
LordOfThePigs's user avatar
14 votes
5 answers
6k views

How can I efficiently diff a CSV file against a database?

I have an inventory of products stored in Postgres. I need to be able to take a CSV file and get a list of changes—the things in the CSV file that are different to what is in the database. The CSV ...
Isvara's user avatar
  • 630
0 votes
0 answers
63 views

What is the best way to desgin my api workflow?

I am building a bank management system using the microservices architecture. I have many microservices but I am going to talk about two. I have customer account service and transaction service. Let's ...
ferrocene's user avatar
-2 votes
1 answer
452 views

Is this the right mechanism for 2 way heartbeat?

I have a software application where we have 2 services, say A and B. A delegates some long running work(tracked by an id) to B which is done asynchronously. They both talk via HTTP. Post submitting ...
Saurav Prakash's user avatar
0 votes
1 answer
84 views

How to load consistent test data into multiple applications (which communicate by message streams)?

I have a system landscape consisting of multiple applications. Each application provides REST interfaces to communicate with the outside world. Each application has its own data storage (some kind of ...
Harold L. Brown's user avatar
3 votes
0 answers
92 views

Difficulty understanding how federation increases cache performance for databases?

I am studying system design for distributed systems and in this page (https://github.com/donnemartin/system-design-primer), one of the following advantages was mentioned for federation for databases ...
Henry Zhu's user avatar
  • 153
-1 votes
1 answer
149 views

Checking a related aggregate existence is Domain concern?

This is my specific situation: I am following DDD and Event sourcing to implement my application (using C#). I have a Domain layer that contains Aggregates' implementation. I write unit tests for this ...
Mehrdad's user avatar
  • 99
1 vote
2 answers
227 views

How is cross device eventual consistency achieved in real time?

Assume there is a distributed system which works on the principles of eventual consistency. Let's consider a scenario where I update my profile picture using my phone. If I log in using another system,...
GenZ Dev's user avatar
  • 121
1 vote
1 answer
153 views

Streaming Market Data Over The Public Internet

I'm currently working on a design for a system that can service market data to external clients over the internet. The current solution involves consuming market data from a feed (Feed Handler), this ...
john's user avatar
  • 119
0 votes
1 answer
68 views

designing an order management and dispatch management system for couriers

I'm studying to develop an order management system and order dispatch for the couriers, this is a personal project, for study purposes. Some important points about the deliverymen, they are bikers and ...
Carlos Rodrigues's user avatar
0 votes
0 answers
482 views

Dealing with third party api rate limit when using multiple services

I have 1 api key that has a rate limit 1000req/1min. I would like to somehow use 50% of that limit in service 1 and 50% of that limit in service 2. Let's say they're just 2 separate processes using ...
Konrad's user avatar
  • 1,559
-1 votes
1 answer
117 views

Design approach to decouple DB write to reduce application latency

I want to use DynamoDB to save application operation data at the end of a session for auditing purposes without affecting the actual application latency. Because this date is only for auditing so we ...
Saloni1308's user avatar
0 votes
0 answers
143 views

Load balancing the load balancers

Currently I have a system where I have installed HAProxy on one machine and my other 3 machines serves the webapps and the fourth machine for the database. Now I need to add another load balancer in ...
Himanshuman's user avatar
3 votes
0 answers
677 views

System Design: Chat application in a distributed system context

I am currently working on a side project so I can learn new technologies and understand how the components fit together. For this project, I decided that I wanted to work on Chat Applications in the ...
Filipe Costa's user avatar
0 votes
1 answer
207 views

Is publishing to the same event store stream from multiple places an anti pattern in a distributed transaction?

Below is a little context to the problem I'm trying to solve. There are two separate microservices with separate frontends , APIs and databases. Customer Microservice - Manages customers Orders ...
cah1r's user avatar
  • 302
5 votes
1 answer
565 views

Can an api gateway handle websockets requests besides regular http requests at the same time?

I am working on social platform with a distributed architecture. The platform should offer a chat module, but since the most efficient way to implement a chat is using websockets. I am not sure what ...
cristid9's user avatar
  • 159
-3 votes
1 answer
230 views

How to manage a distributed application?

I have a distributed application backed by a distributed database. (Distributed here means running across multiple hosts with one or more instances on each host). Application has several toolings ...
kernel0707's user avatar
3 votes
1 answer
2k views

Best strategy to flush Redis cache used in a distributed environment

Current setup: We follow the cache-first strategy right now. Always fetch from the cache first. If absent in cache, fetch from DB and update the cache. Cache TTL is 30 min. After this, the cache-key ...
YetAnotherBot's user avatar