0

In the context of a software update system, suppose we have the following:

  • a.tar: an archive file containing an old version of an application
  • b-a.patch: a file containing the binary difference of the old a.tar and the new b.tar (from bsdiff)

Both files are present on the same system, and the authenticity and integrity of both files have been verified using sha256 hashes from a trusted source.

If we now reconstruct b.tar from a.tar and b-a.patch, can we trust b.tar to be correct, or is it still necessary to verify the integrity of the result, e.g. by checking the file hash?

In response to the comments:

I would consider it necessary to verify the integrity of the result, if there is a "reasonable" probability, under normal operating conditions, that the reconstruction process (using bsdiff) could produce a corrupted file while exiting normally.

What is "reasonable?" Not sure. Let's say more than 1 in 100k?

As this concerns a software update system, the scenario we would like to protect against is inadvertently replacing a working application by a new version that has been corrupted during reconstruction.

9
  • 2
    Please define "necessary", in particular including the exact scenarios you are trying to protect against and with reference to the requirements for your system. Commented Feb 5 at 9:26
  • 3
    This depends on whether you distrust your patch mechanism. If you do that, you should check the file hash on the result. However, do you trust your hashing code? If not, you need to somehow verify that it hashed the files correctly. At the end of the day, you need to trust your system to do things as you programmed them to do, unless you can afford to run 3 independent implementations and do a voting to decide which results to accept. Of course, then you need to trust your voting implementation... Commented Feb 5 at 10:04
  • 1
    What threats are you worried about? e.g. this is under "normal operating" conditions, conditions where your hardware may be failing, conditions where your update procedure is being deliberately attacked by a skript kiddie, conditions where your update procedure is being deliberately attacked by a nation state actor etc? (Spoiler: in the last one, you're screwed) Commented Feb 5 at 10:52
  • 1
    I've experienced one case where such paranoid validation uncovered a problem – unrelated file system corruption due to use of faulty non-ECC memory. If you have reasonably high integrity requirements I'd just check the hash of file B for peace of mind, but generally this isn't needed and should be handled by other layers of the software stack.
    – amon
    Commented Feb 5 at 13:06
  • 2
    Given all that, it seems to me your biggest risk is a bug in bsdiff. Is there a more than 1 in 10^5 chance of that? Without having done an audit of the code, I'd say "yes" - but you should of course do your own risk assessment. Commented Feb 5 at 13:19

2 Answers 2

3

The key thing to consider is: How will you handle the situation when this unlikely scenario occurs?

If you just get and error and can regenerate b some other way, well the impact is going to be very low, you can skip the check.

If it means sending the wrong data to mars and your billion dollar robot exploding. then you should check right?

In your scenario, it seems like the error will be detected when the software fails to launch, and if it does then you can probably make the patch again and reinstall before any damage is done.

But maybe you are updating high frequency trading systems, or BIOS on remote devices or something where the cost of failure is high. Running a check to make sure seems like a sensible thing to do.

In either case the probability of the error doesn't really matter as much as whether you can handle, or have a plan to handle, the failure if it occurs.

1

release process

At a bare minimum, you must produce b.tar from binary patches in CI/CD and automatically verify that its hash matched the desired hash.

This mitigates the risk of seldom-triggered bugs in your tool chain producing a bad output file. For example a rare buffer size alignment could tickle OBOB bugs. Or an unusually large number of zeros could have some interesting interaction with compression. The bsdiff code is less widely used than NIST hashing libraries, so is less well tested. There is some interesting level of complexity to its internal branching and datastructures.

mitigation strategy: Release process is halted until bug is diagnosed and fixed.

deployment process

Things went well in the development lab, but they might not in the field.

Your tool chain may depend on dynamic linking to glibc and other core libraries, which independently rev in the field in ways which differ from what happens in your lab.

OS versions change, again in ways that differ between lab and field.

Well meaning sysads and OS vendors may arrange for surprising behavior of fundamental primitives, usually in the name of security. So for example an Anti Virus checker may intervene when it sees you writing a byte sequence that matches some signature, and decides to neutralize a perceived threat by altering write() behavior.

As product software grows, it may now exceed filesystem (or other) limits that it previously fit within, leading to modified output.

It's hard to say what may possibly happen in the field. Why do we ship production code with asserts enabled? Because we want to learn of surprising field behavior so we can fix our code. Same reason it's worth routinely verifying the b.tar hash at deployment time.

mitigation strategy 1: A background thread immediately starts verifying b.tar as it is being written, and asynchronously reports its completion status. It could report to local human operator and/or to the software vendor's webserver. This report could happen during or even after a software install attempt in a foreground process, and it might modify the installation. Total install time is not much impacted.

mitigation strategy 2: Foreground installer sequentially (1.) patches, (2.) verifies b.tar, (3.) proceeds with normal install on successful validation. Total install time is lengthened by time it takes to read and hash the archive file. Notice that sequential reads can be very fast, and multiple cores can be simultaneously hashing multiple stripes of the archive. Rather than producing a single golden hash, you may wish to publish one hash per archive chunk. Or publish single hash of several chunk hashes.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.