How to manage configuration file updates when deploying new application version?

Question

Given an application having:

• Configuration Files (xml or json)
• An Automated Deployment Process

After the deployment the configuration files can be modified by users.

When deploying a new version of the application, the new version can include new configuration properties, one would like to add the new configuration properties and preserve the old properties values.

For Example:

<!-- OLD -->
<config>
  <path value="C:\App" />
</config>

<!-- NEW -->
<config>
  <path value="F:\Ppa" />
  <errorMessage value="ERROR!" />
</config>

<!-- EXPECTED -->
<config>
  <path value="C:\App" />
  <errorMessage value="ERROR!" />
</config>

How to manage this properly given considerations such as:

• the application is deployed in multiple production environments
• the config files can have arbitrary structure and nesting

So far we considered several solution but each have a considerable drawback:

• Use Xml Semantic Diff Libraries -> Does not handle well some edge cases (detects add/remove as change)
• Create format specific logic -> Requires changes whenever the structure is modified
• Store the configuration property/values in the database (SQL) -> Would require substantial rework in the application

Answer 1

Automatically editing files that are also manually edited is hard and prone to error, just avoid it if possible at all.

Use two layers of configurations which are merged by the config loader: Application-provided and user- or site-provided. The former must never be changed by users, but the latter have precedence when loading config values. This allows users to quickly see what they have configured themselves, and also enables them to revert a configuration to the application defined value by simply deleting or commenting out their local definition.

Answer 2

Before looking for a tool to handle this for you automatically, I would recommend to think about a strategy how to handle this, then look for a tool afterwards. Here is a strategy which can handle a lot of sensible changes over a certain period of time.

Inside the application, there is probably a configuration class, and the Xml file is a persistent representation of that class. The class can have properties of simple types (which map directly to certain Xml elements), and some of complex types (which map to a subtree in the Xml file). The initial configuration values correspond to the default values of the class properties.

• The Xml loader should be tolerant against changes in the Xml file. That means, it should load Xml files even if they don't contain all properties of the configuration class (so the properties keep their default value), and it should ignore Xml elements which don't correspond to any property of the class.

• Care for backwards compatibility: between two versions of the applications, never delete any property from the class, never rename an existing property, and never replace the type of a property by another type with a smaller range of valid values.

• When it seems to become necessary to replace a property by a different type or rename it, introduce the changed one as new, keep the old one alive for backwards compatible reads, and provide a migration step which can initialize the new property from the old one during loading. The deprecated properties should be marked as such, and once they are migrated, they should probably not be written back again to the new Xml file version. This migration is nothing any tool can do automatically - those tools don't know the semantics of the properties, one has to implement this step manually.

• Make sure whenever the application is started and detects an old configuration file, it writes it back immediately with the new default values, or at least when the configuration is edited inside the inbuilt config editor of the application.

• When, after some time (think in years), you think the config class has gotten too many deprecated values, you can freeze the old configuration class and provide a new one which contains all you need in a cleaned, new structure. When the app detects an old file version, it does not do the migration not "property by property", but once for the whole file, by utilizing the older, freezed config class for reading, but not for writing.

In C#, all this can be done with the standard XmlSerializer class (as I mentioned in a comment, we did this successfully with long-term supported application). But you can also do this with your own Xml loading mechanism or utilizing other tools. It only requires additional migration logic whenver one needs more than just adding new properties (so trivial changes like the ones in the example from your question do not require something sophisticated).