24

For a company I used to work for, I had to implement a socket receiver that mostly took data in UDP form over a local connection from some specialized sensor hardware. The data in question was a well-formed UDP packet, but interestingly, the data payload always ended with a CRC16 checksum formed using the rest of the data.

I implemented the check on my end, as per the spec, but I always wondered if this was necessary. After all, doesn't the UDP protocol itself carry a 16-bit CRC? Therefore, although UDP packets can be lost or out-of-order, I was under the impression that they can not be corrupted without being discarded by the network hardware before they reach the OS's processes. Or is there some special use-case I'm missing?

It's worth adding that I was working in the defence industry, which as I'm sure you can imagine, likes to be super-explicit about everything like this, so I'm wondering whether it was just a case of "security OCD"...

Improve this question
3
  • 2
    If it's for security purposes, not just about preventing accidental corruption, you need to use a MAC, which is the keyed equivalent of a checksum.
    – CodesInChaos
    Oct 9, 2015 at 12:20
  • 1
    The UDP checksum is only good for the data that was injected into the UDP packet. What actually creates the checksum? What uses the checksum? Is it used to ensure integrity before the UDP packet is created or perhaps carried along with the packet to ensure that it maintains integrity as it flows through other systems? Without a broader understanding of the components of your system and how the data is created, transformed, and used, I'm not sure that your question is answerable.
    – Thomas Owens ♦
    Oct 9, 2015 at 12:22
  • @ThomasOwens The data was sent from the originating device back-to-back to the receiving hardware. No middle-men. The checksum was created by the originator as a last step before sending.
    – Xenoprimate
    Oct 9, 2015 at 13:05

3 Answers 3

Reset to default
31

The UDP protocol does not guarantee that messages are delivered in order or delivered at all, but it does ensure that those messages which do get delivered are complete and unchanged by automatically including a 16-bit checksum. That means adding another 16-bit checksum on the application layer is usually redundant.

...usually....

First, with IPv4 (not IPv6), the checksum is optional. That means you might be using an exotic configuration which doesn't do checksum generation and validation (but in that case you should rather fix your network stack instead of jury-rigging this on the application layer).

Second, with a 16bit checksum there is a one in 65536 chance that a completely random message happens to have a valid checksum. When this margin of error is too large for your use-case (and in the defense industry I could imagine several where it is), adding another CRC-16 checksum would further reduce it. But in that case you might consider to use a proper message digest like SHA-256 instead of CRC-16. Or go all the way and use a real cryptographic signature. This protects not just against random corruption but also intentional corruption by an attacker.

Third, depending on where the data comes from and where it goes to, it might be corrupted before or after being sent over the network. In that case the additional checksum inside the message might protect the integrity of the message further than just between the two network hosts.

Improve this answer
3
  • 1
    @AProgrammer I admit that the choice of words might have been misleading. I replaced it with "proper message digest". Message digest functions are far longer, making accidental collisions so unlikely that they can be assumed impossible for practical purposes.
    – Philipp
    Oct 9, 2015 at 13:47
  • 2
    It attempts to ensure that messages are unchanged, but the checksum used in UDP is rather weak. While the chance of a random message having a valid checksum is indeed 1 in 65536 for all 16-bit checksums, the more useful measures involve detectable number of bit flips arranged either randomly or in a burst, and all checksums do not perform equally according to this metric.
    – Ben Voigt
    Oct 9, 2015 at 16:27
  • 2
    @AProgrammer Cryptographic hashes (MD5, SHA-1/2/3, ...) aim to be as cheap as possible while ensuring security properties like collision resistance. Typically they can process several hundred MB per second, so they shouldn't be a bottleneck foranything less than Gbit connections. They are still slower than many non cryptographic ones which don't need to be collision resistance. Only password hashes (PBKDF2, bcrypt, scrypt, Argon,...) aim to be expensive to compute.
    – CodesInChaos
    Nov 16, 2015 at 11:14
18

UDP does provide a checksum, however.

  1. The UDP checksum is only 16 bits. That means a 1 in 65536 chance of a corrupt packet passing the checksum.
  2. in UDP over IPv4 the checksum is optional, so a sender could theoretically end up sending a packet without a checksum.
  3. The checksum covers the IP/port information as well as the data. While this is useful in dropping packets with corrupt addresses it means that if the packet passes through a NAT the checksum must be recalculated by the NAT.
  4. The checksum only protects the data while it is traveling in the UDP packet. An application level checksum can protect the data end to end as it passes through a more complex system.
  5. The UDP checksum clealy only tells you that the packet was generated by a UDP implementation. It doesn't tell you that it came from your sensor. An application level checksum on the other hand may help reject packets that are valid UDP but came from some other source.

So I can see legitimate reasons for not trusting the UDP checksum but equally not trusting the UDP checksum and then implementing a similarly weak checksum at the application level seems strange.

There is the possibility that the person desinging the protocol simply didn't know that UDP provided checksums or that the protocol is actually a slight variant of one designed to run over a medium that does not provided checksums.

P.S. since this post is tagged security be aware that the checksums in question are designed to protect against inadvertant changes. Protecting against deliberate modification or spoofing requires both the use of cryptographic hash functions that are resistant to deliberate collisions/preimages and the use of some mechanism (e.g. signatures made using a public key) to verify the hashes themselves have not been modified.

Improve this answer
0

Another possible reason why the original author wanted to include the checksum in the message itself is to support multiple transport protocols. While UDP includes a CRC16 checksum, I2C or SPI might not, nor does RS232.

Think about where the sensor data originates from. It might originate from a microchip that is connected via I2C or serial interface to some other chip that acts as a gateway to an IP network (UDP). In that case it might make sense to already add a level of protection against bit-errors from the beginning on.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.