I'm working on a distributed application for a graduate class I'm taking. There has been much discussion around implementing a way to maintain peer connection health statuses. Think Gossip. One of the primary themes for this project has been how we should rely on UDP communication to overcome the "excessive" bandwidth usage of TCP (wrt maintaining these lists of connection status between peers).
I don't know that I agree with this stance though. To me, the primary advantage of TCP isn't the reliability of the connection, but instead the congestion control it offers. It seems more likely that TCP connections between hosts is actually lighterweight than UDP because you get reliability, congestion control, and built-in connection failure notifications.
One potential scenario that could make adequate use of UDP packets is a backup mechanism for detecting link failures (which is what I've started to implement, though I'm not positive this is the right approach). The idea being that our primary connection to peers is via a TCP socket, whith built-in failure detection, but if we were to miss the disconnect message from this link, we could fall back on a gossipy-stle peer list we received via UDP from other hosts.
Am I looking at this all wrong? Keep in mind that this question is specific to handling peer connections and failures in a distributed application, across multiple nodes.