35

Our project uses a user-specific configuration file. This file is currently not in version control, since it is different for each user. The problem is, whenever a developer adds a new module that requires configuration, or changes the name of an existing modules, the other developers get errors because their private configuration files are not updated.

To solve the problem, we thought of working with two configuration files: a default/global configuration file that will be in version control and will be updated regularly by each developer that adds a new module, and a private configuration file that will be kept out of version control and will contain only the user-specific changes.

However, this still seems like an ad-hoc solution.

Can you propose a better solution?

What do the professionals do?

6
  • 5
    see stackoverflow.com/questions/6009/…
    – jk.
    Mar 27, 2012 at 11:47
  • 4
    Boggle... Why on earth are you even allowing developers to rename modules and break customer configuration at any point other than major upgrade?
    – Mark Booth
    Mar 27, 2012 at 12:07
  • @jk yes, there is even a better match: stackoverflow.com/questions/1974886/… Mar 27, 2012 at 12:26
  • I'm puzzled. How is this not a problem when you install upgrades.
    – Joshua
    Mar 27, 2012 at 16:31
  • 6
    It is not an ad-hoc solution at all. The primary configuration file lives in version control, overridden as required by user-specific config files. Of course, the amount of user-specific configuration needs to be minimized, but some small amount may be unavoidable. May 23, 2012 at 21:55

9 Answers 9

22

Though you already got some good answers here, most of them miss the root cause of your problem: your user config files seem to contain more than just user-specific information, they also contain (perhaps redundant) information which is under version control somewhere else, probably in different files, like module names.

I can think of two possible solutions here:

  • try to separate that information rigorously. For example, don't use any module names in your user config. Use id numbers (for example, GUIDs) to refer to modules, and let those id numbers never change after they have been assigned to a module. Of course, that probably has the drawback that your user config files lose some of their simplicity they might have now. You will perhaps need to create a GUI tool to edit your config files instead of using a plain text editor.

  • give your config file format a version number, and whenever something like a module name is changed, assign them a new version number. Then you can provide an upgrade script which checks the version numbers, and if the config file is not up-to-date, it changes all module names it finds within the file and increases the version number afterwards. This can be automated, so the process of upgrading won't disturb your team mates in their daily work.

EDIT: after reading your posting again, I think your supposed solution is reasonable, as long as new modules are just added, but not renamed. What I wrote above will allow to change module names or the structure of the configuration of existing modules afterwards. But if you don't need that, I would stick to the most simple solution.

11

It's a reasonable solution.

You need a way of specifying the initial value(s) of any new configuration element(s). These have to be stored somewhere and a global, read-only, configuration file is the obvious choice.

Then when each user changes their personal configuration you write these changes to their local copy.

Your code will need to read the global configuration first and the the user specific one to overwrite any changed values. This will be far simpler than reading the local one and then trying to work out which ones haven't been set and hence need reading from the global settings file.

If you use something like XML for the storage then you don't have to worry about handling the case where you remove settings. They won't get requested from the users copy of the file and if you recreate the file on save they'll get removed the first time the application is used after the change.

3

We have a somewhat interesting solution, we're primarily PHP developers, so we use Phing which allows you to create automated tasks written in PHP, so instead of doing our normal svn update, we do a "phing update" which calls svn update, and then replaces our configs with the proper variables, for example, a config:

$db_user = "${test.db_user}";

So the configuration files are all versioned with that interesting syntax, and then we generate an adhoc, unversioned config file for my specific instance, which replaces those "variables" with unversioned settings specified in unversioned ini files. This way we can modify any user-specific files and have the changes populate throughout other working copies.

2

The program should have a default setting in code for when the value is not found in the configuration file. This way as new things are added it won't break, your upgrade path would be smoother, and your users will have fallback for when they mess up the configuration file as well.

Another more complex example would be at program startup or some other key point, open the configuration file using an initialization module and add any defaults that are missing, but this seems pretty heavy.

1
  • +1 for mentioning that this is not just a problem for developers, but a general deployment problem.
    – sleske
    Mar 28, 2012 at 9:03
2

Put a version number into the personal configuration file (the version number of the config file format).

Make the code that processes the personal configuration file check the version number, and if it is out-of-date, run through an update procedure. So, basically, anyone who makes a change that will break existing config files needs to bump the version number of the config file format, and write a procedure to update config files of the previous version (renaming sections, etc.) and re-save them.

You'll probably want some process like this for end-users anyway, so you might as well use it to make your developers' lives easier.

1

Typically how I've seen it done is to have a configuration file with default values checked into the repository. These could be, as an example, the values needed on the testing server. Then, when a developer checks out the file, they will have all of the values. If a new field is added or a field is removed, this is handled in a merge. A developer will check in the needed value for the target server and not check in any other changes to fields that are for his personal development environment.

It takes care to ensure that the merge is done right, but it seems pretty safe to me.

2
  • That seems rather error-prone; I've seen this done, but devs kept accidentally checking in personal settings, which then overwrote the settings other devs used :-(. Also, this means the whole tree will always show up as "modified" in VCS tools, which is very inconvenient.
    – sleske
    Mar 28, 2012 at 9:05
  • @sleske It does require a certain amount of discipline. But honestly, I would expect the ability to do this just fine from most developers.
    – Thomas Owens
    Mar 28, 2012 at 10:28
1

What we do here is create blocks of settings. This can be done in Zend like this:

[production]
key1: parameter1
key2: parameter2

[testing: production]
key1: parameter2
key3: parameter4

This means that testing inherits production and extends it with key3. All each developer then needs to do is set his environment (testing or production in this case)

4
  • The settings are more user-specific. They include such things as installation folders of applications, personal preferences, etc. So, it is not enough just to select between two pre-defined environment. Mar 27, 2012 at 12:08
  • Depending on how many settings and how they are used you can use resources in the ini file for Zend. Not sure if this is also applicable for your problem.
    – Agilix
    Mar 27, 2012 at 12:10
  • I needed a more general solution, that can handle all types of user-specific preferences, not just resources. Mar 27, 2012 at 12:29
  • Just a thought here but wouldn't it be better to store those in a database? At least the preferences. And then you can couple those the a user. If not, it was just a thought ;)
    – Agilix
    Mar 27, 2012 at 12:50
0

This is a useful solution based on the post: Keeping passwords in source control

In summary the strategy is to "keep an encrypted version of the configuration file in source control and then provide a means through which the user can encrypt and decrypt that data."

  1. Create a dummy comfig file and .gitignore it.
  2. Create a makefile that can encrypt and decrypt the config file
  3. Store the makefile and the encrypted config file in the repository
  4. The makefile asks for a password, and how to contact the author for the password.
  5. When building the project, if the makefile has not run, advise the user with console.error("Config file [conf/settings.json] missing! Did you forget to run make decrypt_conf?");
  6. A check is described to ensure the config file is up to date.
2
  • 1
    -1 since this doesn't answer the original question at all. Also, answers that are little more than a link and no explanation aren't useful for the Stack Exchange Q/A format, since external links may disappear at some point, leaving no reference to what the suggested answer was.
    – Derek
    Jun 28, 2013 at 18:22
  • 1
    This is an interesting post though. Jun 29, 2013 at 18:50
0

We built a tool called Config that handles configuration issues like this. What you will do is create (or import) 1 master configuration file. Then create an environment called Local. In the Local environment, create multiple instances, 1 instance per user. If you need to make a change that is common across the board, like adding a new configuration entry or modifying the module name, just make the change, and it will be applied across the board. If you want to make an instance/user change, make that configuration value a variable, then change the variable. This change will only be applied to your instance/user. All of these are under version control. You deploy the configuration files via push or pull. The pull option is similar to git pull, but for that specific instance/user.

Config gives you additional features like comparing configurations between users, search, tagging, validation, and workflow. It is SaaS so not really for those not yet ready for the cloud, but we do have an on-premises plan.

2
  • I think including a SaaS to manage developers' configuration would be an overkill... But that's just, like, my opinion. Also, if the config contains sensitive data (unlikely, since it's probably for testing on their own workstations) it's an instant no-go.
    – Mael
    Feb 23, 2018 at 8:06
  • I don't think Config is an overkill if you consider how easy it is to setup versus the time spent trying to figure out the solution, asking the community, implementing, and maintaining it. Config works on any platform, format, language, library, so you only learn it once. On sensitive data, there's a client-side encryption option, a local vault if you will. Users who go all in can install in-house or use a VPC. Feb 23, 2018 at 15:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.