1

I'm looking to come up with a standard for how to implement a system that will automatically update software. As such, there is no actual programming (yet), but I still want to make sure my standard is robust before moving on to the stage where I build a proof-of-concept prototype.

Please reference:

As a question, are the implorements, core technologies, and response ideas in my idea complete enough andor reasonable? Also, are there any currently-apparent problems that might arise during implementation and execution of what I've outlined?

12
  • 6
    xkcd.com/927
    – Philipp
    Commented Jan 17, 2015 at 2:17
  • 1
    Just installing software to begin with is a hard enough problem. Making an auto-update "standard" will be at least as difficult if not more. Do you have many tens or hundreds of thousands of dollars to commit to this? Commented Jan 17, 2015 at 6:42
  • 3
    Following @Philipp there is no margin in re-inventing the wheel; it is a project not worth doing. Because they are open source and de facto standards, you should stop your activities until you've learned Debian apt and RedHat rpm deeply. These are the best of breed and knowing their internals should be sufficiently scary to turn you away from this endeavor. Do not reinvent Windows MSI because you won't even want to use it yourself.
    – msw
    Commented Jan 17, 2015 at 13:51
  • 1
    @msw thanks for pointing those out! However, the purpose of this is not to be a package manager, but instead a standard to follow when you want to implement a system that silently, automatically updates your software in the background. I'll look through apt and rpm!
    – Ky -
    Commented Jan 17, 2015 at 16:17
  • 3
    There is a lot of literature on that. Read first wikipage on dynamic software updating Commented Jan 17, 2015 at 17:20

3 Answers 3

7

However, the purpose of this is not to be a package manager, but instead a standard to follow when you want to implement a system that silently, automatically updates your software in the background. I'll look through apt and rpm!

The problem is that what you want to do is building a package manager whether you like it or not.

Your current proposed system seems to revolve around simply downloading installers at some unknown interval, and running them. There is a lot more you'll have to do.

That model depends on each and every package being able to have a silent installer. While having a silent installer is great, that will not work for a great many software packages. You'll either have to ignore them, making the 'standard' pretty much useless, or have to define exceptions that can be channeled through your software for when the user has to make decisions.

Say some package depends on the .NET framework. For Windows, that's going to be a lot of packages. Since the installers won't know whether .NET is installed, they'll have to include it, every time. Are you going to want the system to download that massive redistributable over and over and over? And if you don't, then you'll have to have a way for softwares to indicate that's what they need, so your system can download it for them. And having the initial software won't save you, as it is not uncommon for a given software product to change .net framework versions between 'updates'.

Pretty soon, different packages will need to have different versions, and so your software will have to keep track of all that.

How is your software going to handle updates that can't be done successfully? It is unacceptable for your auto-update to randomly break an existing software package. You'll have to implement a system for handling that, and rolling back the changes. Windows Installer provides some of that, but not all packages use windows installer to install themselves, and Windows installer is obviously not available on any other popular platforms.

I don't think you realize the complexity this task requires if it is going to solve real problems for many people.

3
  • 2
    +1 "I don't think you realize the complexity…"
    – msw
    Commented Jan 17, 2015 at 18:22
  • I never intended for the downloaded executables to be run; I intended for the software implementing The BARC to, to put it simply, download new versions of its outdated files and use those to update itself.
    – Ky -
    Commented Jan 18, 2015 at 16:38
  • On your example about the .NET framework: I thought that if a program needed it, and it wasn't installed, Windows itself would prompt the user to install it. Also, if this is not the case and the program must include its own .NET framework, then why can't it tell if it's already downloaded it in the past? Certainly a program can track its own files and see if they're there.
    – Ky -
    Commented Jan 18, 2015 at 16:41
3

The core idea would be to find the difference between the set of files currently installed on someone's computer and the newest set of files that should be installed on that person's specific computer; and (once these differences are determined) download whatever new files are missing and discard whatever old files are obsolete; hopefully in an "atomic" way (that makes it impossible for someone to start the software at the wrong time and get a mixture of old and new).

Note that determining "should be installed on that person's specific computer" may involve an amount of logic. It can depend on which OS the user actually has (32-bit Windows XP, 64-bit Windows 8?), or which CPU they have (with/without AVX?), or which version of DirectX or Java or whatever they have, or whether they're a registered user or not (e.g. only using a free version with restricted features), or whether their video card can handle high detail textures or not, or whether they've enabled various optional features, or...

Also note that the same system could (should?) be used for installing and uninstalling software. For installing software, the set of files currently installed is "nothing" (so the auto-updater knows it needs to download everything in the set of new files). For uninstalling software, you can just pretend that the newest set of files that should be installed is "nothing" and discard the set of currently installed files.

In addition; I'd want some security and/or fault tolerance. For example, maybe the files are encrypted with the provider's private key, and the auto-updater uses the provider's public key to decrypt the files. Maybe every file downloaded has an MD5 checksum and (after decrypting?) the auto-updater makes sure the file's data actually does match the MD5 checksum.

Let's talk about bandwidth. Is the data compressed? If there's a 123 MiB file and the new version of that file only has 12 bytes that are different; do you download the entire 123 MiB file, or do you only download something that describes the differences ("insert 4 bytes here, replace 6 bytes there, then append 2 bytes on the end")?

What does the server store? Does it store plain files only (and compress, encrypt, etc. every single time the file is downloaded); or does the server store pre-compressed and pre-encrypted files, plus store pre-computed descriptions of how to convert old files into new files (so if we're updating that 123 MiB file the server can quickly provide the "insert 4 bytes here, replace 6 bytes there, then append 2 bytes on the end" description)?

Now; let's look at your "refine.txt". Do the technical details within "refine.txt" that describe how it will actually work, cover anything important at all?

4
  • Thanks for all this! You brought up plenty that I never considered. As for your last question, no; that's not until much later. Right now I'm refining my ideas (mainly the roadmap). Hopefully I'll be able to use all this new information to better refine my plan going in and not have to change too much when I'm deep into the project!
    – Ky -
    Commented Jan 17, 2015 at 6:44
  • 1
    @Supuhstar: I'd also strongly recommend splitting it into 3 or more completely separate projects - one project to define and maintain an (open) specification (e.g. like WC3 does for HTML), one project to implement the "server side" parts of that specification (e.g. like Apache does for HTML), then one or more projects to implement the "client side" parts of that specification (e.g. like Mozilla does for HTML).
    – Brendan
    Commented Jan 17, 2015 at 8:13
  • 1
    @Supuhstar: In addition to this answer, will your standard address OS quirks (versioning of core libraries, inability to overwrite executables in use, etc.)? Commented Jan 17, 2015 at 9:53
  • @BartvanIngenSchenau I plan to, yes. I've been watching how my file structures change when Chrome auto-updates, and taking mental notes on such behavior.
    – Ky -
    Commented Jan 17, 2015 at 16:14
3

The original poster said:

However, the purpose of this is not to be a package manager, but instead a standard to follow when you want to implement a system that silently, automatically updates your software in the background.

While you can write an auto-upgrader it is hard to get it right without the package-manager ability to check inter-package dependencies.

For example, old (current?) Windows installers would depend upon a certain version of the Visual C RunTime library (msvcrt.dll) so they'd ship with their idiosyncratic version of that core system file put it in the core system directory and break other applications that needed a different version. The Unix equivalent is known as "dependency hell" where upgrading X would entail upgrading Y which might require downgrading Z. Modern distributions strive mightily to prevent this hell and use package managers to do it.

Finally, auto-upgraders are trivial wrappers around package managers: for every installed package, see if the server has a newer version and ask that the package-manager install the new. As @Brendan implied, if you think you can get away with a single, monolithic "total application image" for every machine, you'll be sorely disappointed because different machines will have annoyingly different requirements. You'll also encounter the legacy code requirement of "I need to keep groovy_app.1.1 on this machine because groovy_app.2.1 breaks my mission critical thingy".

Organizations resist change generally. Organizational IT resists change even more because each change is a discrete, jarring jump for most everyone involved. Google Chrome upgrade is a poor model to follow as it is its own complete monolithic system. Yep, new revisions of Chrome will occasionally break existing extensions (including those "by Google") but they simply don't care. They can get away with it because they are the GOOG, you likely can't.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.