13

I need several duplex channels between two hosts. There are a number of advantages to establish only one TCP connection. But I doubt multiplexing would cause some inevitable problems. Will it harm performance or increase latency significantly? And what about memory usage and CPU usage? Is there any suggestion or caveat you'd like to give?

10

TLDR: The major drawback you might notice when multiplexing multiple channels on top of TCP (if you do it right) is an increased latency because of head-of-line blocking between the channels.

Corollary: If you don't care about latency you should be fine.

On the other hand using a single TCP connection “means less competition with other flows and longer-lived connections, which in turn lead to better utilization of available network capacity“.

Head-of-line blocking blocking over TCP

If you multiplex multiple channels on top of the same TCP stream, the channels might suffer of head-of-line blocking:

Head-of-line blocking (HOL) can occur when transport protocols offer ordered or partial-ordered service: If segments get lost, subsequent messages have to wait for the successful retransmission in the receiver queue and are thus delayed.

When you multiplex multiple streams on top of TCP you get HOL between the channels.

If channel A has filled up the TCP send buffer, you will have to wait before all of this data is received before any new data of channel B can effectively be transmitted to the remote application layer.

See "Multiplexing on top of TCP" for more details about multiplexing channels on top of TCP and the discussion on hackernews.

Examples of multiplexing over TCP

Channel multiplexing over SSH (over TCP)

A typical example of this is SSH. SSH can multiplex multiple channels (see ControlMaster, ControlPath and ControlPersist in OpenSSH). Using this reduces the cost of initializing a new SSH session (initial latency) but heavy transfer on one channel usually increases the latency/interactivity of the other ones (which does not happen if you use multiple TCP stream): if you are using a interactive sessions and start trigerring a heavy file transfer over the same channel, your session will start getting a lot less interactive.

Multiplexed HTTP/2 over TCP

HTTP/2 uses multiplexing of requests/responses over TCP in order to fix the HOL blocking. This feature is advertised in many articles and papers about HTTP/2. The HTTP/2 RFC claims:

HTTP/1.1 added request pipelining, but this only partially addressed request concurrency and still suffers from head-of-line blocking.

[...]

The resulting protocol is more friendly to the network because fewer TCP connections can be used in comparison to HTTP/1.x. This means less competition with other flows and longer-lived connections, which in turn lead to better utilization of available network capacity.

However what is not discussed is that HOL blocking is not resolved entirely. HTTP/2 over TCP still suffers) from TCP-level HOL blocking.

This is discussed in this LWN article about QUIC:

HTTP/2 was designed to address this problem using multiple "streams" built into a single connection. [...] it creates a new problem: the loss of a single packet will stall transmission of all of the streams at once, creating new latency issues. This variant on the head-of-line-blocking problem is built into TCP itself and cannot be fixed with more tweaks at the HTTP level.

Other multiplexing strategies

SCTP

That's one of the distinguishing features of SCTP (multistreaming), you can have multiple independent streams in the same SCTP association and each stream does not block the other ones.

See SSH over SCTP — Optimizing a Multi-Channel Protocol by Adapting It to SCTP for the effect of using SCTP in order to avoid cross-channel HOL blocking in SSH:

SCTP only preserves the order of the messages within a single stream to mitigate an effect known as head-of-line blocking. If a message is lost, the subsequent messages have to be delayed until the lost one is retransmitted to preserve the order. Since only messages of the same stream have to be delayed, the number of affected messages after a loss is reduced.

[...]

By mapping the channels of SSH onto SCTP’s streams, the benefit of multi-streaming is made available to SSH, which is the mitigation of head-of-line blocking.

SCTP is not necessarily easy to deploy (because of OS availability, middlebox interaction, etc.). A possibility is to implement it over UDP in userspace.

QUIC (multiplexing over UDP)

Another example, is the experimental QUIC protocol used for multiplexing HTTP over UDP (because multiplexing multiple streams on top of of TCP as HTTP/2 does suffer from HOL blocking):

QUIC is a new transport which reduces latency compared to that of TCP. On the surface, QUIC is very similar to TCP+TLS+HTTP/2 implemented on UDP.

[...]

Multiplexing without head of line blocking

Google’s QUIC protocol: moving the web from TCP to UDP presents a good overview of QUIC and HOL blocking when multiplexing channels on top of TCP.

A recent presentation claims that HTTP over QUIC improves latency but that the HOL-blocking improvement is a “smaller benefit“:

0-RTT, Over 50% of the latency improvement

[…]

Fewer timeout based retransmissions improve tail latency […]

Other, smaller benefits, e.g. head of line blocking

Note that while QUIC is described as “very similar to TCP+TLS+HTTP/2 implemented on UDP” it is in fact a general-purpose transport which can be used independently of HTTP/2 and might suit your needs.

Note: HTTP/QUIC si going to be standardized as HTTP/3.

  • I don't think HOL is a real problem, if there is a flow control mechanism. Open multiple TCP connections also create multiple window buffers, which may not be more memory efficient. – Sherwood Wang Jul 27 '16 at 8:37
  • I've considered SCTP. But it seems that SCTP hasn't yet been very portable and NAT devices handle it badly. – Sherwood Wang Jul 27 '16 at 8:39
  • @SherwoodWang, Having a control flow mechanism in your multiplexing protocol won't prevent HOL blocking to happen. – ysdx Aug 1 '16 at 11:47
  • When there is no data available at the sender, the sender may send a flow control frame to tell the receiver to switch to the next multiplexed channel and continue to send data frames of this channel. As of receiver is full, similar mechanism can also be used. It definitely prevents HOL blocking with just half of a round-trip latency. – Sherwood Wang Aug 2 '16 at 0:20
  • @SherwoodWang, the TCP HOL problem is really at the receiving side. If some packet was lost in transmission the receiver cannot give the following packets to userspace. It has to wait for the packet to be retransmitted and received. All the reamaining data (event from other multiplexed stream) is blocked until this packet is received. – ysdx Feb 10 '18 at 22:36
3

I'd say you need to read the ZeroMQ Guide, the patterns it gives with the reasons and disadvantages are essential reading.

But otherwise, there's no problem with disconnecting the network channel from your application data delivery. You will have to take on the muxing and demuxing of data packets sent (and I would recommend a packet-based architecture here, not streaming data in a continuous flow) and the buffering of them on each end.

There should be little impact otherwise, you may need more memory - but only slightly - to buffer data, and a little more CPU as you are handling more code to lock and parse packets, but nothing significant. (unless you are writing something specialist that require a large throughput and performance is critical).

2

Yes, I've built a client-server database system using precisely this principle.

The channels multiplexed onto the one TCP connection each send packets of data, which are then split out to the respective recipients at the other end.

Hogging of the connection by one garrulous channel is done by the TCP connection sender doing round-robin selection of which packet to transmit from among the channels that have data ready to send.

To handle the case of one channel deciding to send a 1GB packet and locking everyone else out, the sender can choose to split a packet into chunks and only send one chunk before it gives another channel a turn. At the receiving end the reassembly of chunks into a packet is done before the recipient sees the packet.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.