-3

The Steam Client application allows one to download games purchased through the associated Steam account. A typical modern game uses quite a bit of disk space (in the order of 50-70 GB), so it is not unusual for users to download the files in several Steam sessions.

However, when resuming a download that has been stopped or paused, it resumes almost instantaneously from where it was stopped before. This indicates that no partial hashing is performed on the local machine (which would actually take a lot of time in order to hash tens of GB). How can Steam ensure the integrity of the files if it does not compute a partial hash when resuming the data transfer? Note that even when the download is interrupted due to a system failure, the process resumes almost immediately when the Steam Client is started.

Even if Steam performs a partial hash "by blocks", i.e., by hashing the last block instead of the entire file, how can it make sure that the files already on disk are indeed there, or not corrupted?

Improve this question
11
  • Guess: It runs the verification of blocks that are already present in parallel to the download, and only let you run things when both are finished
    – Caleth
    Jan 2, 2018 at 11:41
  • @Caleth But there is no "verification step": once the download is done 100% it is immediately runnable.
    – PleaseEditUsername
    Jan 2, 2018 at 11:43
  • Let's say you interrupt steam when something is 50% downloaded: when you resume, as long as the verification of existing files is at least as fast as the download, the "verify existing blocks" is done first. Second guess: It writes an "I verified this block" log note somewhere with a filesystem timestamp, and trusts that things haven't changed if they match. (There does exist a "re-verify files" action in the properties, to fix things if that goes wrong)
    – Caleth
    Jan 2, 2018 at 11:47
  • 2
    In the current form, your question will probably be closed soon, because we do not make guessing games here about third party software. But if you reword your question like "how to design a downloader with [add your requirements here]", and add the Steam Client just as an example (not pretending you want to know how exactly that program works internally), then your question has a better chance to survive.
    – Doc Brown
    Jan 2, 2018 at 12:46
  • 1
    You have those issues whether this is a resumable transfer or not. If you understand how to solve it for an uninterrupted download, you can do so for each chunk of the file.
    – Mat
    Jan 2, 2018 at 13:12

1 Answer 1

Reset to default
2

I've never tried to dissect it, but yes the Steam download management is impressive in its ability to pause/resume and even integrity check an installed app and it finds the files missing or corrupted files and fixes. But the question is not on the latter.

I'm sure they transfer a compressed archive, not file by file, and in blocks or chunks, with both client and server knowing where the transfer is so that it can resume if cutoff.

You can look into file transfer protocols and how they evolved. ZModem (https://en.wikipedia.org/wiki/ZMODEM) was the first widely used protocol to support resuming. I'd be surprised if that methodology isn't still present in modern download managers.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.