I'm involved in a project which involves implementing a binary protocol that will be transmitted over TCP. During our early discussions we have hit a brief snag on deciding whether byte stuffing is required if we decide to include the packet length in the field header.

For example. If our packet structure was like so:


Do we need to byte stuff? One of my colleagues seems to say that given we know the packet length then even if a special character i.e. STX, ETX appears in the bytes it will not matter as we are reading [Packet length] amount of bytes anyway.

Seems to make sense but I'm not 100% sure if that reasoning is quite right.

Please note that we are not using existing protocol handler type modules such as proto-buf but designing this protocol ourselves for our own use.

To confirm what I think I'm meaning when referring to byte stuffing:

Byte stuffing is a process that transforms a sequence of data bytes that may contain 'illegal' or 'reserved' values into a potentially longer sequence that contains no occurrences of those value

Taken from Wiki - Consistent Overhead Byte Stuffing


Your colleague is correct.

If you have total packet length, and the data payload is the only variable-length piece in the packet, then you do not need to byte stuff the payload. If you have more than one variable-length piece, you will have to think about how you will disambiguate the various chunks, so you know where one stops and the next begins.

Byte stuffing is used when it is not feasible to insert the packet length into the stream before the packet data payload. USUALLY, this is because messages are of varying lengths and the message length cannot be known a priori.

  • Cheers John. Yes although the payload length may vary we will know it prior to the actual sending. – dreza Apr 13 '14 at 8:08
  • The other situation where byte stuffing is useful is when sending data over an unreliable medium. Use of length-prefixed data with no escapes or byte stuffing is fine if every byte of data will get sent reliably, but loss of a byte may render everything afterward unintelligible. Use of byte stuffing and escapes will make it possible to resynchronize parsing following some corrupted data. – supercat Jun 1 '15 at 6:49
  • @supercat: The question was about a protocol for data to be transmitted via TCP. TCP provides a reliable medium for payload data. That was one of the specific requirements for its design, back in the 1970s. – John R. Strohm Jun 1 '15 at 15:01
  • @JohnR.Strohm: Indeed, TCP does guarantee that either the data will be delivered in order with no gaps, or the connection will die altogether. If the code which is responsible for generating and parsing packets will know when connections die and need to be re-established, there's no need for byte stuffing. If, however, data has to pass through an intermediate layer which is configured automatically to re-establish dropped connections, that layer will need to understand the packet protocol if byte stuffing is not used. – supercat Jun 1 '15 at 15:26

I had to look up "byte stuffing." I'm not sure its a common term so you might want to include a short definition of it.

Assuming that you mean the technique that modifies bytes so that you can have a terminating byte, such as zero, by escaping the data bytes that are zero...

Please, think about this before asking. Field length. Terminating character. Can you think of any situation that would require both?

The field length is almost always the best choice. After reading the length you can read the data. No need to check the data for special bytes.

One word of caution though: Don't blindly trust the length data. Always check that the value is in a valid range first.

  • Sorry, added what I thought of as byte stuffing. – dreza Apr 13 '14 at 8:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.