Suppose we have two peer nodes: the first node can send a connection request to the second one, but also the second one can send a connection request to the first one. How to avoid a double connection between the two nodes? To resolve this issue, it would be sufficient to make sequential the operations performed for creating inbound or outbound TCP connections.
This means that each node should process sequentially each new connection creation operation, both for incoming connections and for outgoing connections. In this way, maintaining a list of connected nodes, before accepting a new incoming connection from a node or before sending a connection request to a node, it will be sufficient to check if this node is already present in the list.
In order to make sequential the operations of creating connections, it is sufficient to perform a lock on the list of connected nodes: in fact, for each new connection, the identifier of the new connected node is added to this list. However, I wonder if this approach can cause distributed deadlock:
- the first node could send a connection request to the second one;
- the second node could send a connection request to the first one;
- assuming that the two connection requests are not asynchronous, both nodes lock any incoming connection requests.
How could I solve this problem?
UPDATE: However, I still have to lock on the list every time a new (incoming or outgoing) connection is created, since other threads may access this list, then the problem of deadlock would still remain.
UPDATE 2: Based on your advice I wrote an algorithm to prevent mutual acceptance of a login request. Since each node is a peer, it could have a client routine to send new connection requests and a server routine to accept incoming connections.
ClientSideLoginRoutine() {
for each (address in cache) {
lock (neighbors_table) {
if (neighbors_table.contains(address)) {
// there is already a neighbor with the same address
continue;
}
neighbors_table.add(address, status: CONNECTING);
} // end lock
// ...
// The node tries to establish a TCP connection with the remote address
// and perform the login procedure by sending its listening address (IP and port).
boolean login_result = // ...
// ...
if (login_result)
lock (neighbors_table)
neighbors_table.add(address, status: CONNECTED);
} // end for
}
ServerSideLoginRoutine(remoteListeningAddress) {
// ...
// initialization of data structures needed for communication (queues, etc)
// ...
lock(neighbors_table) {
if(neighbors_table.contains(remoteAddress) && its status is CONNECTING) {
// In this case, the client-side on the same node has already
// initiated the procedure of logging in to the remote node.
if (myListeningAddress < remoteListeningAddress) {
refusesLogin();
return;
}
}
neighbors_table.add(remoteListeningAddress, status: CONNECTED);
} // end lock
}
Example: The IP:port of node A is A:7001 - The IP:port of node B is B:8001.
Suppose that the node A has sent a login request to the node B:8001. In this case, the node A calls the login routine by sending by sending its own listening address (A:7001). As a consequence, the neighbors_table of node A contains the address of the remote node (B:8001): this address is associated with the CONNECTING state. Node A is waiting for node B accept or deny the login request.
Meanwhile, the node B also may have sent a connection request to the address of node A (A:7001), then node A may be processing the request of the node B. So, the neighbors_table of node B contains the address of the remote node (A:7001): this address is associated with the CONNECTING state. Node B is waiting for node A accept or deny the login request.
If the server side of node A rejects the request from B:8001, then I must be sure that server side of node B will accept the request from A:7001. Similarly, if the server side of node B rejects the request from A:7001, then I must be sure that server side of node A will accept the request from B:8001.
According to the "small address" rule, in this case the node A will reject the login request by the node B, while node B will accept the request from node A.
What do you think about that?
