Questions tagged [distributed-system]

This tag can be used by questions on distributed system concept, design, and implementations.

Filter by
Sorted by
Tagged with
0
votes
0answers
26 views

Different Ways to implement validation registry

Task To Accomplish I am creating a validation service which validates a user profile data for correctness. A user profile can have following information: Name Address Social Security Number(SSN) ...
1
vote
3answers
120 views

Which code to copy-paste and which to centralize?

In a fairly complex system with a dozen or more git repositories, most of them are done in a cookie-cutter fashion. What is or would be your best approach to deal with duplication? In my opinion the ...
0
votes
1answer
55 views

Is sequential consistency equivalent to performing memory accesses by a processes in program order and performing each memory access atomically?

In Fundamentals of Parallel Multicore Architecture, by Yan Solihin, p304 defines sequential consistency memory model: Overall, we can express programmers’ implicit expectation of memory access ...
0
votes
0answers
20 views

Pubsub message design in microservices

Example We're designing a social network, and when the user logs in, we need to authenticate them and then load their feed. In order to do this, the client first makes a HTTPS request to the server, ...
4
votes
1answer
78 views

Do coherence and transactional isolation mean the same?

About coherence vs consistency, https://en.wikipedia.org/wiki/Consistency_model says Coherence deals with maintaining a global order in which writes to a single location or single variable are seen ...
2
votes
1answer
56 views

How to handle changes to local and global representation of an Entity in a distributed environment?

I'm a developer for a retail enterprise IT department, and am currently stuck on the following scenario: Department A is responsible for a CRUD service that's used for handling stores - so when our ...
3
votes
1answer
132 views

Database consistency for a distributed system

I'm working on a phone call monitoring project. The aim is to have one row in the calls table for each ongoing call. Each call instance may be updated by many different servers/threads as more ...
27
votes
4answers
6k views

I'm not seeing 'tightly coupled code' as one of the drawbacks of a monolithic application architecture

One of the most common things I see when discussing pros/cons of microservice vs monolithic architecture is that monolithic applications have, or always trend toward, 'tight coupling.' To be honest, I'...
0
votes
2answers
162 views

Why is it considered hard to maintain strong consistency in a distributed system?

Why can't you just use strongly consistent reads for all your DB reads, with retries on 500 responses? According to CAP theorem increasing consistency should probably lower availability, but can't the ...
5
votes
1answer
88 views

Architecting basket in high scale distributed application

I'm designing an e-commerce application. Main flow is pretty straightforward: customer add items to basket, checkouts the basket (place an order) and waits for delivery. There are following ...
1
vote
1answer
66 views

Can passive and active replication handle byzantine failures?

Currently studying about replication and I was wondering: In Passive replication, we have an agreement stage where the Primary replica manager (RM) waits for ack from the backup RM, wouldn't it be ...
1
vote
0answers
109 views

How does Stackoverflow comment system work

When someone comments on my question/answer/comment I will see the notification from the inbox, and it also shows under my response tab. Later when the comment is edited, the comment shown under the ...
4
votes
2answers
143 views

What is the meaning of fan-out

I see this word in many places but don't get it. From the Wikipedia, In message-oriented middleware solutions, fan-out is a messaging pattern used to model an information exchange that implies the ...
2
votes
1answer
155 views

How to protect an open-source distributed application consisting of clients and servers from forks made by hackers?

TL;DR: How distributed open-source apps like Scuttlebutt are secured from DoS and hackers who can make custom version of application? I'm struggle with designing an open-source distributed application ...
1
vote
1answer
59 views

Should I use ROA or SOA for a distributed application and how could I implement REST If using ROA

For my final year project I'm looking to build a distributed version of a popular benchmarking client (this has already been done using various methods involving some form of existing frameworks), I ...
4
votes
2answers
95 views

Designing an ETL with where there are a few points of entry

I'm trying to think of a scalable solution for my current system. The current system is 3 microscopes 1 processing machine 1. 60-100GB Files come from 2-3 microscopes every 30 minutes 2. That data ...
4
votes
3answers
275 views

How can adding redunancy adversely affect performance

I have been reading about system design and was going through this popular article - A Word on Scalability The author here says: An always-on service is said to be scalable if adding resources to ...
2
votes
0answers
34 views

SAGA pattern - how to handle situation when some message is not received by orchestrator

Recently I'm studying Saga pattern, but I'm struggling to imagine how it will work in real system. Standard example is making an order, which consist of reserving flight, hotel and a car. We can ...
1
vote
1answer
137 views

Distributed message queue, propagating queue leader/follower information

I'm designing a distributed queue message queue and I'm not sure how to get the leader information to another component. This is my current design: I have the following components: FrontEnd: request ...
0
votes
1answer
144 views

What should we consider when designing a load balancer with minimum cost

Given a set of processes running on a cluster of hosts, I'm designing a system that load balances the hosts through live migrations (carried out by an external service) of the processes. The goal of ...
0
votes
1answer
37 views

Distributed processing of a large number of queues

I have a rather basic application hosted on Kubernetes, which connects to a Mongo database. The app has a wallet feature. A user can put money in their wallet using real-world payments (e.g. via ...
2
votes
1answer
64 views

Distributed cart state in microservices

We have legacy app which keeps state of a cart in local storage on a Frontend (this decision was made long time ago by other devs). The cart is cleared from local storage after 2 hours if it is not ...
0
votes
1answer
112 views

Managing consistency in distributed database

I have question regarding system design. Lets say I want to design a Flight booking system ( such as American Airlines). So, consistency when booking a ticket is very important. This booking system is ...
0
votes
1answer
45 views

System design to store pieces of data with specific ttyl to then refetch

I have a system which generates pieces of data that needs to be tracked and assigned a specific TTYL value (a value in seconds) for how long they are supposed to live before being re-processed. I ...
-2
votes
2answers
98 views

How can one choose between consistency and availability if the system you are designing is Tiny URL

I am putting together a system design for something similar to Tiny URL. It provides an API for mapping a URL to a short URL. Once created the short URL can be used to access the original URL. This ...
1
vote
1answer
92 views

Implemeting a distributed n-Producers/1-Consumer service for critical-mission system

I am trying to implement a distributed version of multi-producer/1-consumer for a critical-mission system. I'm looking for good alternatives to the current approach based on RDBMS. The problem The ...
3
votes
4answers
491 views

If a system talks to a database to get some previous information to serve a request, does that make the system **stateful** or **stateless**?

I've read this example which basically states that, since a method has to hold a single private field.. It is considered a stateful system. So basically ANYTHING system that calls upon some sort of ...
1
vote
2answers
128 views

URL shortening : data partitioning

I've been looking in many places about this question - most top Google search results are copy-pasted from a single source, and others are not particularly helpful. Not sure if I am allowed to include ...
2
votes
1answer
120 views

What architecture/design pattern for sharing data between several people, without centralization?

Let's say my software lets you place a square on the map and say "This is a good place to find Pokemon". I need this data to go to my gym leader's map, and to my friend's copy of Pokemon Go (...
2
votes
1answer
147 views

What are the different messaging patterns for multiple producers to single consumer through a distributed queue?

We have multiple producers that publish messages to the same SQS queue. We have a single consumer that processes the messages. The producers do not care about the response. It is more like a broadcast ...
0
votes
0answers
25 views

Experiences with coarse grained and fine grained message designs in an ECST pattern

I am currently facing a dilemma in the design of a message oriented distributed system following an Event Carried State Transfer pattern. This is essentially a "near real time ETL" used to ...
8
votes
2answers
288 views

When bounded contexts and “microservices” collide. A distributed systems dilemma in diagram form

You can't please everyone. Some people want a lot of context and background on sites like this. Others do not. If you don't want the background, skip the first three paragraphs. I am a software ...
2
votes
1answer
89 views

The publish-subscribe pattern: garbage-collecting old subscriptions

I've been studying distributed systems design, in particular the Udi Dahan's class. He talks about the publish-subscribe pattern as a common pattern in messaging-oriented designs. There's obviously a ...
1
vote
1answer
85 views

When keeping all state in an SQL DB, how to handle one-off variables?

I noticed this pattern in my programs: I'm building a system, like the one I'm building for my local hackerspace right now. It's using database as its "brain" - a scraper as a separate ...
0
votes
0answers
17 views

Messaging - Responsibility of Consumer vs Publisher for Active-Active failover

Say there is an external client-DC1/DC2, posting requests to a service in DC1 and DC2 respectively. Now, if DC1 is down, should the external client-DC1 responsible for routing the request message to ...
3
votes
3answers
302 views

How does a CDN get “data” when there's hundreds of terabytes stored in data centers?

I am still trying to wrap my head around CDN's in a real world scenario. Suppose I am building a Netflix clone. I have about 1,000 terabytes of video content stored in an S3 bucket. Since Netflix ...
3
votes
2answers
396 views

Alternatives to distributed transactions in .NET?

Lately, I've been working on a project which basically is a huge rewrite in .NET Core F# + Event Sourcing + PostgreSQL of an old sub-ledger legacy app written in .NET 4.6 C# + SQL Server. Since the ...
0
votes
1answer
79 views

Maintaining Objects Across API Deployment Instances

I am working on a web application as a hobby and trying to learn some concepts related to cloud development and distributed applications. I am currently targeting an AWS EC2 instance as a deployment ...
-1
votes
1answer
68 views

Facebook sharding by user_id or by photo_id?

I'm reading a design suggestion for facebook/instagram social network in grokking the system interview (closed content :( so I'll describe the relevant part in my question). They are speculating over ...
1
vote
1answer
69 views

Is weak consistency still relevant in systems that record timestamps?

There are multiple aspects and quality that are desirable for a distributed system. Among these are consistency and availability. According to the CAP theorem, if partition tolerance is a must, the ...
-1
votes
2answers
59 views

Concurrency issue in job-based system

Setup Say I have a database table called Events which contains events for when a user becomes active or inactive EventId|Timestamp |User|Status | -------|--------------|----|--------| 1 |...
2
votes
2answers
158 views

Synchronous communcation vs caching in microservices

I'm trying to build a distributed system and have been researching how best to manage dependencies between services. The common example in tutorials is an ordering system - let's say I have a Catalog, ...
3
votes
4answers
429 views

How to resolve cyclic dependencies in Event-driven systems?

Consider two services (bounded contexts by DDD): Sales Billing Sales is responsible for creating orders and Billing for handling payments. Sales tracks orders and Billing holds payments: Sales ...
-4
votes
2answers
326 views

Why can't we vertically scale a system infinitely?

When talking about scaling a system, it's often said that vertical scaling has limitations. So after a point, we need to scale the system horizontally. What are the limitations of: Main memory - ...
1
vote
0answers
23 views

Do Multi-Primary replicated distributed systems guarantee sequential consistency?

I know that Primary-backup replicated distributed systems guarantee sequential consistency. My question is whether multi-primary systems also achieve that. I mean if they use consensus (i.e: Paxos ...
0
votes
1answer
104 views

Need to test for equality on vector clocks

I've heard about vector clocks and how to test if a message was sent before another message. E.g. Message A was sent before message B if every element of the vector of message A is smaller or equal ...
0
votes
2answers
888 views

Race condition in Distributed Systems

We have a set of micro-services that are functionally dependent and have intertwined validation logic. e.g. consider 'Credit-Card' service and 'Loan' service. When user has a credit card then loan ...
-4
votes
1answer
232 views

What difference and relation are between fault tolerance and (high) availability?

From Coulouris' Distributed Systems 5ed Chapter 18 Replication 18.1 Introduction Increased availability: Users require services to be highly available. That is, the proportion ...
-2
votes
2answers
195 views

How does a distributed system both tolerate network partition and achieve consistency?

In the CAP theorem, there are only three possible cases: C and A without P C and P without A A and P without C How can we have both P and C in the second case? Doesn't propagation of update from ...
-4
votes
1answer
60 views

Does stale data due to weak level of consistency count as Byzantine failure?

I have difficulty understand Section 18.3 Fault Tolerance Services under Ch18 Replication in Coulouris' Distributed Systems. If my reading and understanding is correct (which might not), Section 18....