Is my auto-update standard reasonable?

Question

I'm looking to come up with a standard for how to implement a system that will automatically update software. As such, there is no actual programming (yet), but I still want to make sure my standard is robust before moving on to the stage where I build a proof-of-concept prototype.

Please reference:

• The GitHub repo in which I've placed my ideas is
  • Refine.txt, which outlines all my ideas for the standard
  • update-response-idea.json, which gives an example of what a server response might look like

As a question, are the implorements, core technologies, and response ideas in my idea complete enough andor reasonable? Also, are there any currently-apparent problems that might arise during implementation and execution of what I've outlined?

Answer 1

However, the purpose of this is not to be a package manager, but instead a standard to follow when you want to implement a system that silently, automatically updates your software in the background. I'll look through apt and rpm!

The problem is that what you want to do is building a package manager whether you like it or not.

Your current proposed system seems to revolve around simply downloading installers at some unknown interval, and running them. There is a lot more you'll have to do.

That model depends on each and every package being able to have a silent installer. While having a silent installer is great, that will not work for a great many software packages. You'll either have to ignore them, making the 'standard' pretty much useless, or have to define exceptions that can be channeled through your software for when the user has to make decisions.

Say some package depends on the .NET framework. For Windows, that's going to be a lot of packages. Since the installers won't know whether .NET is installed, they'll have to include it, every time. Are you going to want the system to download that massive redistributable over and over and over? And if you don't, then you'll have to have a way for softwares to indicate that's what they need, so your system can download it for them. And having the initial software won't save you, as it is not uncommon for a given software product to change .net framework versions between 'updates'.

Pretty soon, different packages will need to have different versions, and so your software will have to keep track of all that.

How is your software going to handle updates that can't be done successfully? It is unacceptable for your auto-update to randomly break an existing software package. You'll have to implement a system for handling that, and rolling back the changes. Windows Installer provides some of that, but not all packages use windows installer to install themselves, and Windows installer is obviously not available on any other popular platforms.

I don't think you realize the complexity this task requires if it is going to solve real problems for many people.

Answer 2

The core idea would be to find the difference between the set of files currently installed on someone's computer and the newest set of files that should be installed on that person's specific computer; and (once these differences are determined) download whatever new files are missing and discard whatever old files are obsolete; hopefully in an "atomic" way (that makes it impossible for someone to start the software at the wrong time and get a mixture of old and new).

Note that determining "should be installed on that person's specific computer" may involve an amount of logic. It can depend on which OS the user actually has (32-bit Windows XP, 64-bit Windows 8?), or which CPU they have (with/without AVX?), or which version of DirectX or Java or whatever they have, or whether they're a registered user or not (e.g. only using a free version with restricted features), or whether their video card can handle high detail textures or not, or whether they've enabled various optional features, or...

Also note that the same system could (should?) be used for installing and uninstalling software. For installing software, the set of files currently installed is "nothing" (so the auto-updater knows it needs to download everything in the set of new files). For uninstalling software, you can just pretend that the newest set of files that should be installed is "nothing" and discard the set of currently installed files.

In addition; I'd want some security and/or fault tolerance. For example, maybe the files are encrypted with the provider's private key, and the auto-updater uses the provider's public key to decrypt the files. Maybe every file downloaded has an MD5 checksum and (after decrypting?) the auto-updater makes sure the file's data actually does match the MD5 checksum.

Let's talk about bandwidth. Is the data compressed? If there's a 123 MiB file and the new version of that file only has 12 bytes that are different; do you download the entire 123 MiB file, or do you only download something that describes the differences ("insert 4 bytes here, replace 6 bytes there, then append 2 bytes on the end")?

What does the server store? Does it store plain files only (and compress, encrypt, etc. every single time the file is downloaded); or does the server store pre-compressed and pre-encrypted files, plus store pre-computed descriptions of how to convert old files into new files (so if we're updating that 123 MiB file the server can quickly provide the "insert 4 bytes here, replace 6 bytes there, then append 2 bytes on the end" description)?

Now; let's look at your "refine.txt". Do the technical details within "refine.txt" that describe how it will actually work, cover anything important at all?

Answer 3

The original poster said:

However, the purpose of this is not to be a package manager, but instead a standard to follow when you want to implement a system that silently, automatically updates your software in the background.

While you can write an auto-upgrader it is hard to get it right without the package-manager ability to check inter-package dependencies.

For example, old (current?) Windows installers would depend upon a certain version of the Visual C RunTime library (msvcrt.dll) so they'd ship with their idiosyncratic version of that core system file put it in the core system directory and break other applications that needed a different version. The Unix equivalent is known as "dependency hell" where upgrading X would entail upgrading Y which might require downgrading Z. Modern distributions strive mightily to prevent this hell and use package managers to do it.

Finally, auto-upgraders are trivial wrappers around package managers: for every installed package, see if the server has a newer version and ask that the package-manager install the new. As @Brendan implied, if you think you can get away with a single, monolithic "total application image" for every machine, you'll be sorely disappointed because different machines will have annoyingly different requirements. You'll also encounter the legacy code requirement of "I need to keep groovy_app.1.1 on this machine because groovy_app.2.1 breaks my mission critical thingy".

Organizations resist change generally. Organizational IT resists change even more because each change is a discrete, jarring jump for most everyone involved. Google Chrome upgrade is a poor model to follow as it is its own complete monolithic system. Yep, new revisions of Chrome will occasionally break existing extensions (including those "by Google") but they simply don't care. They can get away with it because they are the GOOG, you likely can't.