3

I'm composing communication lyrics for client-server and what am I thinking about:

  • "authme username passord" (maybe encrytped)
  • "accept"
  • "get archive of H2O from 03.02.2005 to 20.12.2064"
  • transferring binary structure or "error descrtiption"

why I always need to do something like

  • 0x0FA52FD + CRC
  • 0x0D34423 + CRC
  • ...

I can see some secure reasons but I think it's not the real reason so why I can't use strings in client-server communication?

2
  • 12
    "communication lyrics" - that needs to be made a technical term for something. Apr 4, 2012 at 9:25
  • 1
    Yes, the idea of computers singing to each other has a certain appeal.
    – Jaydee
    Apr 4, 2012 at 12:41

6 Answers 6

7

I would advocate a binary protocol based off something like Protocol Buffers. There's little benefit, in my opinion, in a human-readable protocol assuming the protocol isn't meant to be read by a human.

Using protocol buffers gives you the compactness of a binary protocol with the flexibility of a text-based protocol (e.g. with optional fields and the like). It also has the added benefit of being able to generate the serialization/deserialization code for you automatically, so you can use strong-typed objects in your program code.

4
  • I don't transfer data in strings / commands only.
    – cnd
    Apr 4, 2012 at 9:23
  • thank you for Protocol Buffers, maybe I will use it right now for my project. Seems interesting, and protobuf-net for client.
    – cnd
    Apr 4, 2012 at 9:27
  • 6
    Even if the protocol isn't meant to be read by a human, at some point somebody is going to have to hack / troubleshoot / understand it and it is loads easier if it is text . . . Apr 4, 2012 at 10:31
  • Although TCP/IP is reliable and 8-bit clean, some communications protocols may require sending data over a link which has neither property. Having a character or sequence which appears at the start of every message and nowhere else can be helpful when a link is unreliable, but would generally require that binary data which could contain such characters escape them. In some cases it may be easier to say that a protocol will always expand things by e.g. 33% than to say that data may expand by anywhere from 0% to 100%.
    – supercat
    Dec 4, 2014 at 17:53
5

you CAN use strings (look at the http, ftp, ... protocol) but then you'll likely bump into encoding problems (less now that the UTF8 has become the de facto standard but you'll still bump into stuff like UTF16 and will need to account for that)

you would also need to write a string parser that handles malformed input correctly (if only to drop the connection instead of crashing)

however parsing binary often comes down to a sequence of switch(in&MASK){case FIRST_CASE:...} which is much more efficient

however next to all these is the amount of data you are transmitting: binary can be much more compact than strings for comparable protocol

2
  • using UTF8 isn't a problem, writing normal parser / lexer is not a problem, same as parsing binary nodes, sure binary is more compact but I don't want to use strings for data, just foe commands. If I compare commands size with all other data it's no matters strings they are or binary symbols.
    – cnd
    Apr 4, 2012 at 8:34
  • +1 for thinking about encodings for data (which affects binary protocols too), but -1 for thinking that constants in textual protocols need to be in anything other than a fixed encoding (e.g., ASCII). Apr 5, 2012 at 21:06
4

There are actually very few good reasons to go binary. The one you mentioned -- security -- is a false promise. You are either encrypting your data or not; binary transmission in the clear is still in the clear it just requires a modicum more skill to read. Another reason mentioned -- text encoding -- is effectively a solved problem at this point. On the flip side, binary encoding can create interesting problems -- think big- versus little-endian.

The advantages of text-based protocols are that they are vastly easier to understand, build, debug, use and hack. You don't need nearly as much tooling to get anywhere. When you do need tooling there is an endless array of text-based tools at your disposal. With your custom binary protocol you get to define the toolset for better of for worse.

This is not to say that binary protocols don't have their place -- there are lots of applications that require that sort of compactness and pointedness. But binary should not be the default answer.

2

Do you need to maintain readability of the protocol itself? If so, why? Human-readable(that is, text strings) protocols like the example you gave have multiple downsides. First obvious one is the bandwidth requirements; it is surely much more compact to send out 0x01, 0x00 than something like "USER LOGIN\n", let alone longer phrases as you described. The second is security. For example what do the bytes 0x3C, 0x02 and 0x7F mean in terms of protocol? The only way to figure out is to start reverse-engineering the binaries. Much more tedious work, and not always that trivial.

Frankly, I see absolutely no real benefits of using human readable protocols, unless they are specifically meant to be human readable - something which I don't see happening that often. I would strongy advice against strings in protocols. Please, don't.

6
  • so because of size only? does size of commands such mutter? it's not easier for humans, it's easier for me to debug (just a bit).
    – cnd
    Apr 4, 2012 at 9:21
  • 1
    Size matters all the time. Just because bandwidth "is unlimited"(for now, and for some it might not be at all) it doesn't mean you shouldn't preserve it. It's just a matter of "good practices", why would you waste something which you could easily conserve? I think finese is important, especially when we deal with communications. Minimizing overhead and "bloat" should be something to thrive for, because it affects everyone using the network.
    – zxcdw
    Apr 4, 2012 at 11:09
  • 1
    It's been a while since I've worked in the guts of TCP/IP, but I don't think it's as simple as 'saving bandwidth'. As I recall TCP/IP is not going to waste a whole packet on transmitting a single byte or word unless specifically configured to do so (Nagle's algorithm: en.wikipedia.org/wiki/Nagle's_algorithm). Data is going to be buffered until it can fill up a packet. Using a more 'wasteful' representation may actually reduce command latency, as the buffers will fill up more quickly. Apr 4, 2012 at 23:13
  • @youngdood Tell that to the veterans of Y2K. There are plenty of good reasons to embrace "bloat". Apr 5, 2012 at 20:57
  • @CharlesE.Grant Unless, of course, you force it to (e.g., setting the PUSH flag). But your point is valid - a bit is pretty narrow at 1Gb/sec. Apr 5, 2012 at 20:59
2

The Internet has lots of experience with binary and textual protocols. Textual protocols are the norm, not because "we always do it that way", but because over 40 years experience has shown that textual protocols are significantly easier to work with, and because bandwidth considerations rarely pan out in the long run.

Don't conflate "binary" and "secure". There's nothing inherently secure about a binary protocol. Even undocumented binary protocols have been reverse-engineered. Similarly, there's nothing inherently insecure about a textual protocol. Security arises from well-thought-out plans against attacks, not from simple obscurity.

1

There are two reasons that spring to mind:

  1. Encoded requests and responses are usually shorter than "plain text" requests. If you are sending a lot of data this could become important. It's less so these days with faster connections and if you are only sending relatively small amounts of data.

  2. Security is a real concern. If requests and responses are sent in the clear they can be read while in transit (man in the middle attacks) and, given enough packets, your application could be compromised. People could send fake requests to get data out of your system, insert fake data into your system and generally make a mess of things. While this is also true for binary data, it's harder and less obvious what the data is.

5
  • 1
    about security: I understand but binary data is also hack-able in this way, just a bit more complex / harder.
    – cnd
    Apr 4, 2012 at 7:54
  • @Alessa - That is true, I assumed you were talking about some sort of encoding/encrypting.
    – ChrisF
    Apr 4, 2012 at 7:54
  • 12
    I'd phrase it more strongly: using a binary format vs. a text format does not improve security. It raises the clue barrier slightly, at most by providing obscurity. And as you surely have heard before: security through obscurity is not real security. Apr 4, 2012 at 8:22
  • @JoachimSauer I wish my management could understand this. I currently have to implement an information maze around a product for the sake of IP protection, with the main consequence of making things awfully complicated. Apr 4, 2012 at 8:33
  • 1
    a binary protocol does not offer any additional security by itself. You can send text over TLS just as easily as binary data. Apr 4, 2012 at 8:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.