I am currently working on a report regarding a matrix multiplication distribution system I have been working on. But after reading a lot of somewhat contradictory sources online I have become in doubt whether the network created by the system would classify as being decentralized or distributed.

As I have attempted to illustrate below it essentially uses a master/slave model but in addition masters may also serve as slaves for other master or themselves simultaneously. A master carries out jobs, which are equations that uses matrix multiplication, by partitioning them into tasks, that is a subset of the original equation. An example would be the equation: A*B*C*D*E*F, which could be partitioned into the following tasks: A*B, C*D and E*F. The tasks are then distributed and computed across the master's slaves. Once the master has received the result from each tasks they may produce a final result, ending the job.

enter image description here

To my understanding a decentralized network is one which features multiple centers of communication in form "central nodes". But, although I cannot find anyone specifying it, it seems to me that there cannot be any overlap in the nodes associated to the central nodes, which is the case in our network. At the same time it does not feel completely distributed either since each master is working in a centralized manner.

How would you classify it?

EDIT: In addition to the aforementioned distribution the system also maintains a "backup ring" that is every node has a dedicated backup node. In the case of a master, their backup node is essentially a standby-master. The master will periodically dump the state of it's job(s) to their dedicated backup node. The state of a job contains the tasks, results and who their slaves/workers are. In case a master fails the backup node will continue maintaining the jobs (from the state they were last synchronized). Since a slave does not maintain any jobs they do not synchronize anything with their backup node. But, in case they are backing up a failing master, they themselves will become the new master. The image has also been updated accordingly.

  • any system is distributed when it consists of multiple nodes. But you're right that it's quite tricky to classify your system as either centralized or decentralized.
    – amon
    May 4, 2019 at 20:30
  • @amon I see. I made the question on behalf of this paper pages.cs.wisc.edu/~akella/CS740/F08/740-Papers/Bar64.pdf. From the paper, I made the logical conclusion that one could classify a system as either distributed or decentralized.
    – Bach Elor
    May 5, 2019 at 8:59

1 Answer 1


If I query a decentralized system at my home computer I'm not centralizing it I'm just asking for a result. Same with your master.

The key feature of a decentralized system is partition tolerance. Any small part of it can go down or loose connectivity and the rest keeps working. Centralized systems tend to have single points of failure. You take this one part down, the whole thing fails.

I simply can't tell which your system is from your description because masters and slaves could exist in the same node and only that node, creating a single point of failure.

  • I see. The thing is that our system actually has a "backup ring", which we made to avoid having a single-point-of-failure in regards to any job being worked in the network. Initially I avoided adding this because it made the question a lot longer, but I see now that it is vital to the classification. Based on your definition of a decentralized system, would this not change the classification? Cheers.
    – Bach Elor
    May 5, 2019 at 7:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.