Put very simply, my company's product is a webapp with a database backend. We have a stage environment and a production environment for each of roughly 100 clients. There is a separate database for each environment. The database contents come from a number of CSV files that are created and maintained by our (non-technical) data team. Data is client specific and must also be updated when dev adds new features to the product. However, not all clients use the same version of the product, so not only the contents but also the structure of the CSVs varies by client.

Currently, the CSV files are versioned in SVN. However, there is no separation between tested and untested data. My team is responsible for loading new data to customer environments, but we've gotten burned before by loading untested data to a production environment. The clear answer to this is to mimic a development setup and create a trunk, branches, and tags, where trunk is always stable (tested data), branches contain experimental/untested changes, and tags are created every time new data is loaded to an environment.

My question involves the dependency between data and code...when dev makes a release that includes new features, the data has to be updated to accommodate those features before we can upgrade the code on an environment. How can we track whether the data for a particular client is compatible with a particular version of code?

My first instinct is to create a branch in the data repo every time a code upgrade is planned for a given customer. The branch name would correspond to the release number. However, because the code version on our production boxes usually lags significantly behind that on our stage environments, the data team would have to work in two branches simultaneously, ensuring that new features for the latest and greatest code only go in the newest branch, but fixes continue to go in the older branch as well. Of course, once the code on stage and production diverge, we cannot be 100% sure that the data fixes we test on stage will work on the older code installed in production. Does anyone have any ideas about how we can manage this process better?

2 Answers 2


Wow that sounds like a complicated mess. I have worked with these types of systems before and having clients on different versions really complicates things.

I highly recommend using the branch method. You need to treat these csv files as code since in practice they are. They should be tested and promoted just like you would code.

  • Complicated mess is right. +1 for pointing out that the CSVs should be treated as code. May 27, 2011 at 3:00

Have a table that includes the data version. Have an upgrade process that involves updating tables, and getting rid of/inserting data that changed between versions.

I would also strongly recommend that you develop to a standard where data can be updated independently of the code release. It will make releases much smoother if you can prep the database then do the release independently, though admittedly it makes development somewhat trickier.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.