Background info and what I've tried

Within each of two mostly distinct environments, say 'Research' and 'Production', there is a need for a structured parameter file. The file contains things like database connections, tables, and some other parameters, nothing unusual.

The current approach is to have two distinct parameter files, say "params.txt" and "params_prod.txt" (for production). (Also, in case it helps, we're not actually using .txt for this, we're using a markup language, but the details of that don't really matter for this question.)

Most of the information is duplicated between these two files, leading to lots of copy/paste. Plus, in version control, everyone has to manually make changes to two files and we have to trust that they are propagating changes if checking things in.

Needless to say, this leads to headaches to resolve when some divergence happens between the files. We have tried workarounds, like writing tests to check that the files are identical in all the ways they need to be, but this is not a perfect science since the structure of the files can change and the definition of what, exactly, must be identical between them changes too.

I have had one idea:

Just use a single file, but create sub-sections within the file. There can be a sub-section for general parameters that are always shared, another section for things that should be treated as the 'default' parameters (the 'Research' environment settings) and then another section for the 'Production' environment settings.

We already have code that parses the parameter files and instantiates objects, loads data, etc. etc., based on the parameters. We could go back and add some options to that code, such that it loads data according to which sub-section of the parameter file it is told to use and ignores parameters from the other section.

This has some benefits: (1) everything is in one file, so there's no "just trusting" that people are propagating changes. (2) It also does not require copy/paste code since anything that is shared between both environments needs to only appear in one section at the top of one parameter file. (3) If anything, this should make the parameter files themselves more modular and easier to use. (4) We save time/cost that would have been spent creating complicated test-based work-arounds that check whether the files are being propagated together during check-ins with version control. We don't need to do that in this case.

It does have costs too: (1) time spent making the newly formatted parameter file specifications in XSD so we can validate it and ensure it is backwards compatible; (2) costs if we do need to re-code and make our software-that-interfaces-with-parameter-files have options for whether to use 'Research' or 'Production'. (3) If any of the properties that are currently considered to be in the sub-section that is shared, but which suddenly become things where we want to have different options between 'Research' and 'Production', we would then have to refactor all parameter files to move that item down into the other sections.

(3) Seems like the biggest worry, but it also forces us to constantly re-factor these parameter files, which is a good thing in my view. Besides, if we want flexibility about re-assigning a parameter from one sub-section to another, there ought to be better ways to achieve it than duplicating the entire file.


Are there any significant pitfalls that I am failing to realize about the proposed idea regarding don't-repeat-yourself vs. build-in-flexibility trade-offs in parameter file design?


1 Answer 1


another option is to use a hierarchy of files:

there is a param_prod.txt where all parameters specific to production is set

then there is a param.txt where general parameters are set

to find a parameter you first go into param_prod.txt and if you don't find it there you then check param.txt (if not found there then either use arbitrary default or error)

pros: DRY, easily create another branch, extensible to multiple levels by keeping a list of which files to search

con: not immediately clear where a certain parameter is defined, need to create new format where parameters can be left out (maybe), searching through all files can be slow (use caching for speedup)

  • This is a clever idea, and it prevents the need to duplicate changes. I'll definitely think more about it and include it when I compile my suggestions on this.
    – user103181
    Commented Sep 25, 2013 at 13:17
  • @rachetfreak at once of my previous jobs, we used m4 to build a single property file out of the appropriate set of files that was then deployed for that site. The code only looks it up in one spot - no hierarchy. It was the build process that built and understood the hierarchy of files. It also gave you nice macros for expanding things without having that in the code either.
    – user40980
    Commented Sep 26, 2013 at 17:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.