How can I develop a Package Manager for Embedded Systems?

Question

I need to develop/use a package manager for an embedded Linux system with the following properties:

• A master fully controls which packages need to be installed (the slave has no possibility to ask the master anything)
• Download speed on the device is slow that not just everything from the master distribution can be download to the target
• Upgrading/Downgrading of packages must be possible
• It should be possible to replace the package manager on the system itself
• Possibility to replace the operating system itself
• It's easily to recover from failures (power loss, ...), especially relevant when considering the two points above
• Locks on the slave which have to be deleted manually are by no means allowed
• If a new (meta?) package replaces an old (meta) package and there are some unneeded packages left, they're removed, to save space on the device
• Define some form of order or dependency to install packages
• Perhaps some form of a recovery system is need (how to detect that it is needed, how to boot it, ...)

Some points are just repeating other points, and some points are not necessarily something the package manager itself has to do.

I started to copy ideas from dpkg (especially how it install files and renames them later to reduce the time frame where something bad can happen) and use the deb format for packages itself (not using all of the fields).

I'm not sure myself, if this questions belongs to Programmers, but instead asking for a solution or anything like that, I'm more interested if other programmers faced the same problem and if there are any notable resources how this kind of problem is solved. Personally it happened to me too many times that embedded devices could be almost destroyed by a software update without any possibility (without special hardware) for recovery.

Some links that may be relevant for others are the following ones:

• http://en.wikipedia.org/wiki/List_of_software_package_management_systems
• http://en.wikipedia.org/wiki/Ipkg

I could as well provide a pseudo code alike description for what I intend to do.

Answer 1

As a rule with embedded systems you need a boot loader to manage OS updates. Typically you have a primary boot loader that boot straps the minimum amount of hardware needed to install a secondary boot loader.

Usually the primary boot loader will be able to install a basic program from a physically connected store such as USB or serial, which is usually the secondary boot loader. The idea of the primary boot loader is to prevent "bricking" of the device under any kind of software failure - so the primary boot loader can never be field upgraded, if at all.

As you appear to want to perform complex "Over the Air" updates, the primary boot loader is not a suitable place for it, but you want to be able to recover from a failed operation OTA. This will require a secondary boot loader that can manage this complexity, and install OS and OS updates as needed.

If you can tolerate the risk of a device update failing, and having to resort to a primary boot recovery, then you will not need the secondary boot loader.

Think of a normal PC - Primary is the BIOS recovery/ rollback jumper, secondary is the BIOS (Which can boot from a USB/DVD/NETWORK etc, and is upgradeable) and then the OS. The OS can install it's own updates, but if one of those fails badly, the BIOS can do a PXE Network boot and reinstall the OS.

You can make the Secondary as complex as needed, provided you have the time, as it can be updated, but KISS is essential.

What you are asking for is not trivial to build from scratch, Find someone that's done it before. I am not familiar enough with Embeded Linux systems to know what might be out there for you to build on. The best advise I can give is : It will fail, you need a recovery mechanism, once you have a recovery mechanistic, it does not matter if it fails because you can recover, so focus on recovery, not prevention of failure. The focus on how many simultaneous failures do you need to recover from, just 1, 2, more...... More than 1 is extremely hard, more than 3 not even NASA goes there.