1

Until now for each of our applications we store application configuration in the same repository as the code that uses the configuration. We have environment specific yml files (dev, test, .., prod) that contain plain text as well as encrypted configuration values. Those get deployed for example as "Application Settings" in Azure. Depending on the type of application (Java Tomcat, .NET webapp, etc.) The configuration values are fetched at runtime from environment variables or configuration files directly.

Now, we would like to decouple the release of the code from release of the configuration. A configuration change should not require us to rebuild and redeploy the application (process, approval, etc). This would be for patches for instance where the application version x.y will fetch the latest configuration for x.y.*.

The natural way to do this is to move the configuration values into a dedicated service that the applications will fetch them from, a REST API, backed by a database for instance. The question is where that external repository gets populated from, where the configuration lives from then on. Two options I see are keeping the configuration in the repository of the application it configures or having a "Configuration" repository holding the configuration of all applications.

I am a big proponent of keeping the code and configuration together in one repository. I just can't really quantify the upsides. Here are a few I can think of.

  • Keeping configuration isolated from other applications'. Other application's configuration changes won't affect each other. (Changing a seemingly unshared configuration)
  • Review of code and configuration at the same time in the same merge request.

I feel very strongly about this but I don't know why.

I don't see any downsides of keeping the configuration together with the code and no upsides of moving all configuration into the same repository.

Why is it better to store the configuration alongside the code? Why is it better to store all configuration in the same repository?

  • 2
    What is the stated purpose of decoupling the release of the code from the release of the configuration, and what benefits will it provide? – Robert Harvey Jun 21 '18 at 21:25
  • It's not 100% clear to me. One argument is that if there's anything wrong with the configuration then we'll be able to react quicker. The artifact doesn't have to be approved etc. that's just the process we're dealing with, I'm not saying it's my preferred way to do it. – Tobias Jun 21 '18 at 22:22
  • I don't think moving the configuration into an external store will help us much. We're just working around an issue instead of fixing the root cause. But that's what I have to deal with so I'm trying to make the best out of it. – Tobias Jun 21 '18 at 22:44
3

The configuration values are fetched at runtime from environment variables or configuration files directly.

It sounds like you already have the possibility of decoupled deployments because you say that you pass in configuration at runtime. It sounds like your process is the thing that's getting in the way of decoupled config deployments. Configuration is only coupled to the software if it's part of the artifact at build time and cannot be changed at runtime. I think you're conflating repositories with deployments. Not everything that's in the same repository must be deployed together, and things that are in separate repositories can be deployed together.

This would be for patches for instance where the application version x.y will fetch the latest configuration for x.y.*.

What do you mean when you say application version x.y will fetch configuration for x.y.*? The application shouldn't be fetching config. The config should be passed in by whatever starts the application (through environment variables or command line arguments).

I think it's pretty important to keep the "developer's" config with the code. Your application code should have a config that is appropriate for a developer to clone the repository, build, and run the software on their local system. I think the dev, test, and prod configs should not be in the same repository for a couple reasons:

  1. Security - Even though you've encrypted sensitive information in the config, it's still not a great idea to keep this in VCS with the code. E.g. you wouldn't want to keep it in an open-source repository. Such a practice also encourages the key to be shared among developers so that anybody on the team can update the config when it changes.
  2. Flexibility - Your codebase should be designed to run with any configuration, it shouldn't try to predict how a specific production deployment will be configured. There can be infinite variations of the config, why should your code repository contain specific ones? If you want to create more environments you'll end up with a proliferation of config files.
  3. Separation of Responsibility - In many companies the personnel who write the software are not the same people who set up the infrastructure it will run on. I think it's easier for the team that manages the infrastructure and deployment of software to manage the configuration of the software. Instead I think the software engineers should create and maintain the description of the configuration and the operations team should manage which hosts/databases/passwords etc. get filled in.

The natural way to do this is to move the configuration values into a dedicated service that the applications will fetch them from, a REST API, backed by a database for instance

I don't see this as much of an improvement of your current setup. Indeed it is much more complex. Now you have to configure your original system to talk to the configuration service. Also running the application locally would now require running an additional service. In the spirit of dependency-injection, software should not configure itself--configuration should be passed in. If you were to develop a service like you're describing it would be better to fetch the configuration from the service then start the application with the fetched config instead of the application fetching its own config.

I am a big proponent of keeping the code and configuration together in one repository. I just can't really quantify the upsides. Here are a few I can think of.

  • Keeping configuration isolated from other applications'. Other application's configuration changes won't affect each other. (Changing a seemingly unshared configuration)

There is nothing stopping you from keeping applications' configurations separate from each other and certainly deploying one configuration shouldn't necessitate redeploying all configurations.

  • Review of code and configuration at the same time in the same merge request.

I think this is odd. Does this mean before the code is reviewed, you must decide the production configuration? If my change requires a new database, do I have to go to the infrastructure team, create all databases for all environments, fill in the hostnames, usernames, and passwords? Again if the description of the configuration is in the source repository then that can be reviewed without dealing with the production configuration.

  • I think you make many good points. I think my problem is that I previously worked in an environment where individual dev teams were very autonomous and had pretty much all the responsibility. The devops layer was very thin and was more there to help us get started. Separation of responsibilities is a very good point! I'll try to view configuration as the responsibility of that team and I think it should make things easier for me. Thank you very much! – Tobias Jun 22 '18 at 16:15
0

A change to a config file shouldn't require a big process. It shouldn't be super easy either, because a bad change could lock users out or cause other malfunctions.

Storing configs in a web service just moves the problem around, and then doubles your maintenance effort.

You have a process to work in, so the challenge is working with The Powers That Be to develop a streamlined process for config changes.

Barring that, identify the config settings that change often and add that to your database instead. You'll need one app push to deploy the new code, but from then on you should have maintenance screens in your application to handle those changes.

A web service is like hitting a thumb tack with a sledge hammer in this case. Just regular ol' database tables and forms should do the trick.

  • The services don't have a UI – Tobias Jun 22 '18 at 1:31
0

Ive worked on a variety of ERP, CRM and Billing systems. If i understand what you mean by configuration correctly... In my experience anything that describes the instance enough that the software can start up is held in files. Beyond that application configuration is commonly held within the database.

The main goal being that you can backup and restore a db from one environment to another and it can be started without risk of pointing at the wrong folder paths etc and making a mess.

The advantage of table driven application config is not only can you pass the maintenance to the business users, but it can be queried, and compared easily.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.