We’re rewarding the question askers & reputations are being recalculated! Read more.
27

However, if multiple users are being served by separate servers and both try to add the same item to their cart, for which there is only one remaining, there must be some "source of truth" for the quantity left for that item. Not really. This is not a problem that requires a 100% perfect technical solution, because both error cases have a business solution ...


15

What is "eventual consistency"? How does it compare to "transactional consistency"? When does it happen? Consistency models describe how a system (nominally a distributed system) responds to change. In an eventually-consistent system, all nodes will eventually have a consistent view of the overall system state. However, there will be a period of time after ...


13

Just because source control + distributed was a huge success, issue tracking + distributed isn't necessarily a good idea. What do we gain from distributed source control and would it apply to issue tracking? Easy branching and merging: not really. Actually it is crucial that everybody is on the same page. So branching would be highly undesirable. ...


13

When building distributed systems, the difference between a 'synchronous' system and an 'asynchronous' system is this: A synchronous system has known upper bounds on computation and message delivery times. So: you have an asynchronous system where certain events do not have these known upper bounds. How do you handle it? If these asynchronous processes have ...


11

The end-IP is not published. The process actually works in a way the client (a user hitting the balancer) believes they are communicating with the balancer, while talking to an actual node. In a very simple explanation, most transactions work like this: A user makes request to the load balancer. The balancer decides which node is the most suitable (based ...


9

It's simply to completely separate failure of each different tab and plugin. When one fail, whatever the way it fails, only this one will crash. The monitoring process will just report that it failed and will be able to reload it. C++ have shared memory by default and an exception system which makes easy to crash the whole application if one part isn't ...


9

The most architecturally sound approach I know of is to put that single source of truth behind a microservice. It is perfectly okay for multiple parts of the system to update that data, as long as they do it through something like a microservice that can ensure it's always done correctly and predictably. So for instance, Customer data is probably already in ...


9

whenever a slave loses its connection to the master, this slave immediately becomes unavailable That is not necessarily true. The CAP argument assumes that when the network is partitioned, there may be clients on both sides of the partition. ...So I'm keeping my consistency and I'm keeping my availability in the system. The CAP argument also assumes ...


8

Virtual machines may be the way to go, the problem is that if you setup 5 VM's on one server the response time between them will be effectively 0, which may not be true in your real environment where there could be a delay between servers. If that is an issue (or could be) I would suggest setting up some servers on AWS or the like to test. Chances are that ...


8

What exactly constitutes distributed computing? Distributed computing is an inherently parallel collection of processing elements that communicate with one another to tackle one or more problems. Those processing elements are sufficiently separated from each other that it is not practical to build a reliable and timely messaging fabric between them, and so ...


7

Both Redmine and TRAC are typical web based CRUD applications, architecturally wise, there isn't a reason why they can't be distributed. The simplest way would be to have them use a distributed database, and since both support MySQL the obvious solution would be MySQL Cluster. Then of course you could save yourself any hassle and run them on a cloud, for ...


7

The usual practice is to set the servers to all keep their time updated using NTP. There are limitations to the accuracy when using NTP time syncing which means that you should only rely on the time stamp to give a general idea of when events occurred, which is likely to be good enough for identifying the set of events you are interested in. Timestamps are ...


6

Distributed computing is a computing system that has processing occurring on different computers (i.e. on a distributed system). The individual programs communicate with each other through a series of communication channels. These channels are usually network connections (TCP sockets, for example), but often use other communication protocols and devices (...


6

Do multiple parallel threads trying to synchronize for access to a resource constitute a problem in the domain of distributed computing? They do if those threads could be running on different machines, or even if they're running on the same machine but in different processes.


6

Short answer: Apache Mesos doesn't provide distributed FS. So, apps have to work with local FS on slaves or you may run any distributed FS alongside Mesos. Mesos is typically deployed together with HDFS, and most of the frameworks that run on top of Mesos can work with HDFS (Hadoop, Spark, Storm, etc.) And in case your app doesn't support any distributed ...


6

Once you decide that clients cannot be trusted, then it is a given fact that each value is immutable, and each key is immutable. And immutability is forever. This in turn means that this is not a problem of storing and finding values, it is a problem of identifying the key of the most updated version of a value, so it is a versioning problem. When trying ...


6

In a traditional N server <-> 1 RDBMS style system, the database is used as a central point of synchronisation which helps prevent such inconsistencies. In event sourced systems, the "event store" serves the same role. For an event sourced object, your write is an append of your new events to a particular version of the event stream. So, just as with ...


6

A combination of hashing sharding replication distribution high fail-over key-value stores There's no magic, just more and more complex situations. Just like DNS, it is made to scale. The 'single version of the truth' is part of such systems. Generating a new key becomes a more complex operation than just generating the next number in the sequence. For ...


6

"Atomic" means that the process appears to have happened instantaneously, or at least it cannot be interrupted. Its possible to have atomic distributed processes, but its not easy, e.g. you could do the following: Start a SQL transaction Add the user to the database Send the email Rollback the transaction if the email failed to send Commit the transaction ...


6

You need to make a clear separation between events modifying the state of your read model, and events (potentially) modifying the state of external systems. Make sure you do not have any "mixed events" modifying both states together. That way, you can replay your events in a specific "replay mode" where those events for the external system are not fired ...


5

Order is the least of your problems. Not only does UDP not guarantee delivery order, it doesn't guarantee delivery at all. Lamport's algorithm requires cooperation of all non-requesting processes that would contend for a resource in the form of replies. The loss of a datagram carrying a request would cause the other processes not to send replies. That or ...


5

Every software development technique we've ever invented has been about managing complexity somehow. A huge portion of them have been and continue to be about abstraction, encapsulation and loose coupling. Microservices are yet another way of doing those things, which is probably why it resembles a lot of older techniques at a high theoretical level, but ...


5

Utilizing a webserver for this purpose is actually a standard approach, it is just a simple form of Service Oriented Architecture. Of course, this term might pretend more than there is actually behind it. To keep this lightweight, without the need using a fullblown Webserver, you can use a tool like node.js. It is the most simple solution I can think of for ...


5

I have seen the 'Last Item In Stock' problem solved in the following way: Update all the stock levels daily and flag products as high, low, on order or out of stock categories according to threshold levels. Obviously its the 'low stock' items which are problematic Items with high stock levels Don't bother checking the stock level. Just place the order ...


5

I don't think this is about atomic-operation, synchronization, asynchronous-programming, or transactions. I think this is about meaningful state. Right now you have three possible states. The object's uploaded field is true, it's false, or the objects record doesn't exist. The question then is what do these three states mean? How do we arrive at them? ...


5

The problem is that it can be hard to see such a flow as it's not explicit in any program text. Often the only way to figure out this flow is from monitoring a live system. There are two separate aspects to this: how do you centralize a flow's logic, and how do you identify flows when monitoring. For monitoring a possible solution is the use of correlation ...


4

We use this: http://lxc.sourceforge.net/ Linux Containers are extremely light weight virtual machines (basically zero overhead). We are using only 250 machines per physical machine, but there shouldn't be any issue going for thousands per machine. We successfully used both LVM partitions and overlay filesystems as the system root.


4

I just skipped through the paper that you mention, but here I go. We often use a virtualised environment to create experimental setups that involve multiple computers. Using Hyper-V on Windows Server (or similar software) you can easily create a private, isolated network of machines that you can configure on the fly and reuse as needed. You seem to be ...


4

I don't think being decentralized is as important as having off-line capabilities. Integration with source control is a big benefit, so you want each user to be able to conveniently handle both tasks. The closer together they do it the more continuity you'll have. Even the most distributed teams should be able to put together a web server and tracking ...


4

There are several such projects. The most publicized system (but certainly not the first) is Diaspora, which is a social network made of many individually-operated servers, called "pods". Pods can be freely set up using the AGPL-licensed source code. An individual user can set up a personal pod, or can join a public pod. Regardless of what pod your account ...


Only top voted, non community-wiki answers of a minimum length are eligible