1

I just wonder how do you deal with this scenario. Let's say you have an application that uses configuration files that are loaded right after the application starts. Configuration files store - lets say 100 - configuration parameters. For all clients the application uses the same parameters except one (let it be one file named circuit.cfg), so for clients A and B the difference is in just one configuration file.

If the application is under GIT control how do you handle differences (different files) of such file for all clients?

My guesses are:

  1. keep different branches (develop-client-a, develop-client-b) all up to date to develop which stores some version of circuit.cfg and when application is deployed for client a it is done from develop-client-a branch,
  2. store circuit-client-a.cfg and circuit-client-b.cfg in develop branch

I think that 2) is a little bit harder to maintain since you need to have knowledge somewhere which of the configuration files have to be chosen when deploying for particular client.

On the other hand having multiple branches can be a problem for CI server if the configuration of it is that you always make a release from develop branch (but it can probably be customized if such scenario exists).

Do you have any suggestions for this? How do you deal with it?

3 Answers 3

3

First, I would heavily recommend against using branches for customization, even when the kind of customization will be as simple as providing a specific config file. The problem is, this will lead to long-living branches, which have a huge potential of becoming a maintenenance issue. In the worst case, you end up like the asker of this older SE question, but even with only one custom branch, you have to make sure you don't forget to merge new features from your main dev branch into the custom branch, which has a certain risk of producing merge conflicts.

Or, lets say you want to add configuration validator to your automated tests, which makes sure all different custom configuration files have a valid structure. This is way easier when all config files live in the same branch.

So you should focus on a solution with just one branch, like your option #2. You wrote

I think that 2) is a little bit harder to maintain since you need to have knowledge somewhere which of the configuration files have to be chosen when deploying for particular client.

Well, you are correct that somewhere in your deployment process you need to have to make sure each client gets the correct configuration file - which is always the case, regardless of the technical approach. However, going with #2 is not necessarily "harder to maintain" than your approach #1. The trick is to build the knowledge "which client gets what" explicitly into your build & deployment process.

Now you did not mention how your deployment process looks like or what type of system you are developing, so I cannot tell you the exact steps. But since you mentioned a CI server, I guess you have most of your build and deployment steps automated, maybe with some build or deployment scripts. There is where you should approach the problem: make sure you build the system automatically for each client (or for each type of client) individually, for example, in a folder of their own. One last step in the build process then should be to copy the correct config file for the specific client to the output folder. If you use Git repos for releasing/deploying, this can be implemented in an analogous manner.

And yes, this can also be accomplished with Hans-Martin Mosner's suggestion of using separate repos for those config files. If you think it will become simpler that way, use it, however it may not really be worth the effort to setup individual repos for a few files maintained by the same group of devs.

3
  • I did not mention about build system at all because I thought its not important. Since you asked I will tell you. We use maven for dependency management and starting build process. So all dependencies are build separately and artifacts are stored in the server, so when application is built all dependencies are retrieved first and then we start building app, as you probably know that since its very common. My guess about branch was just an idea that I don't especially like because of what you have said that it can be a problem for maintaining long living branch. Nov 16, 2021 at 6:58
  • After I have written the question I have realized that I can do this the same as we already do with dependencies. I mean I can delete my resources from application and put them in new maven projects that will produce an artifacts that I will depend on in the application (one of them). Lets say res-client-a, res-client-b and when maven starts the build process this specific artifact will be retrieved and placed where it has to be. The only think I have to do to make automated process correct is to change dependency of this client resource in the pom.xml of the application. Nov 16, 2021 at 7:00
  • Not to make such change permanent I will depend on some maven execution argument that will change dependency name and retrieve appropriate res artifact. This is a little bit different from what I said in the first place that I want to change only one configuration file, here I have to have a copy of whole resource directory but it is not a problem though. Nov 16, 2021 at 7:01
1

Software development repositories should be separate from client specific configuration data repositories. If you have many configuration parameters that are normally unchanged for each client then it might be best to keep them under version control with the application, and manage the client configurations separately.

If a new software version requires a new parameter that should be different for some clients, then you would need to update those client configurations but probably not much more.

8
  • Storing client specific config data in the same repo as the main application can be perfectly justified, especially when the same devs who maintain the application are responsible for maintaining the client specific configs. One has to make sure deployment isn't done directly, in an unfiltered manner from the devs repo, of course.
    – Doc Brown
    Nov 15, 2021 at 14:38
  • Of course it can be done, just as you can keep completely unrelated projects in one repo if you're careful :-) If you have multiple clients with different release cadences and configurations it makes sense to view the deployments as projects separate from the core development. In this case the OP seems to have (hypothetical IIUC) issues with keeping them in one repo, possibly with different branches, and using separate repos eases that pain. Nov 15, 2021 at 15:18
  • 1
    See my answer, different branches is IMHO the very wrong approach here.
    – Doc Brown
    Nov 15, 2021 at 15:19
  • 1
    My interpretation of above: Put all client configurations together in a separate repository, differentiated by name, e.g., IBM-Federal.conf, Honeywell-Thermostats.conf, JPMorgan-AssetManagement.conf. Or, possibly, by named subdirectories within which are (all) the configuration file(s) for that client with their proper name, e.g., IBM-Federal/global.conf. Then, you CI/CD system (or whatever you use for deployment) deploys both your application out of its repository and the specific client configuration out of the configuration repo.
    – davidbak
    Nov 15, 2021 at 15:51
  • @davidbak: yep, that's how I read this answer, too. Still the deployment process has to differentiate between which config goes where, and I don't see how this gets easier or harder as if those config files would live in the same repo as the application.
    – Doc Brown
    Nov 15, 2021 at 16:10
0

It looks from the comment as if you are using Maven. If so, I would suggest something like :

First create your application completely without configuration files as one or more Maven artifacts.

Then for each client you create a separate Maven project/module responsible for packing the application in the form expected by the client. (Could be a WAR for web apps in a Java EE container, or or a Spring Boot standalone executable). In that process you also add the configuration files needed - either inside the archive, or next to it - and they can be easily put in the repository as each client is separate.

You still have the issue with configuration files being different between test and production. Write them so the differences can come in from the environment.

This approach is easily scriptable so your CI/CD system can build for every client every time without any manual steps.

1
  • This is basically what I did so far. My comments in Doc Brown's answer presented first idea but it was not the best as well. Now, I have an application (artifact) without resources and I have more maven projects that depend on application artifact and add specific configuration and produce complete client specific artifact. This is so far the best what I could do and its very simple to achieve. Jan 2, 2022 at 20:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.