I already understand that Elastic Search is supposed to be deployed in a distributed topology, in that you can have multiple nodes for a cluster of ES instances.
I like the API, and it looks promising. But I wonder how Elastic Search solves some standard problems in distributed computing. My "architectural" questions are motivated out of trying to understand whether I ought to commit to using the product.
Consider membership problem: If I add a node to the cluster, how does Elastic Search cluster know to add that node wants to join? Or what if a node goes down (or the network to the data center it is in) gets too flaky. Is it using some sort of gossip protocol like Amazon Dynamo does?
Consider the consensus problem: if I POST a document to one node to the cluster, how does the change get propagated to the other nodes? Of course, I understand that the document has to be indexed, so I cannot expect a search query to find it immediately. Does ES use a Paxos
algorithm? Raft
algorithm?
UPDATE: A nice discussion of the split-brain problem in Elastic Search.