3

I was reading the book "Continuous delivery, reliable software deployments through build, tests and deployment automation ". And the author mentions that one should be able to rollback to any version of a software with one click.

  1. I am wondering if this is really useful, and especially in the following scenarios :

    • You are working on web projects. Sounds to me that in this case, you don't really need the ability to rollback to any previous version. Being able to rollback the current release should be enough. Once it's deployed and working, why would you ever want to go back to a previous version ?

    • And actually this argument holds true if you are working on desktop applications.

    Perhaps I misunderstood what he means by "being able to rollback to any version".

  2. Does he mean the production environment or the development environment ? If it's the later that would make more sense - even if I still find it kind of unecessary for the web, source control branching should do the trick for desktop applications.

  3. And still if it's the later, do you think that using tools like Entity Framework Code First Migration that allow you to recreate the database from the source code is enough to say that we have reach this standard ? Because then, with source control, we'd even have the referential data that need to be seeded for this particular version after the database is created.

  • 5
    for 1: Having deployed your software does not mean it is working:) – Philipp Apr 1 '15 at 11:33
  • What is your definition of a version? It may no longer apply to the way the author thought about it back in the day. – JeffO Apr 1 '15 at 15:30
  • @JeffO My definition is that a version is a release – tobiak777 Apr 1 '15 at 15:43
8

In short:

  1. Since you don't know whether a build contains a critical bug at the time of deployment, you may need to roll back more than one version when you're deploying at a relatively high frequency. If your setup doesn't allow you to do so, your only alternative is to roll forward as quickly as you can.

  2. As soon as you have an application that has 3rd party dependencies (say an API) or storage (say a database with schema), rolling back may be a lot harder than just deploying the same bits again. Source control is not enough, your schema needs to be reselient. You may need to build forward and backward compatibility into the design of your code and database schema to be able to pull that off.

  3. Code First Migrations will allow you to easily move forward, but it doesn't fix your schema if you accidentally delete a vital column, or populate a default value with the wrong value or something else that may happen during a migration. being able to roll back would mean being able to restore the accidental changes, while retaining important (business) data. Patterns like CQRS provide you with a form of record and playback of changes, which will allow you to reconstruct the data in it's original form if needed.

Your interpretation probably holds when you're able to take down the site/application, back it up, install the new version, test it (can you absolutely be certain you did not introduce a critical bug?) and see that it works. If you want to take a step forward, do rolling deployments, canary builds/releases, dark launching, live failovers and other advanced deployment techniques, you need to take a step back and take in the whole picture.


Say you have Continuous Integration and you want to deploy every build when it passes all tests, you may already be 2 or 3 deployments ahead when you may find a critical defect which you weren't testing for.

If you can only go back one deployment, you're stuck and will have to fix the defect and roll forward. This puts a number of demands on your ability to come up, test and distribute a patch in short order.

If you're able to turn back to the last known good build, you're in much safer water.

You'll see that teams who have the ability to roll over at any point int time, will be less strict about this rule, than teams who need the ability to roll back.

  • Thanks a lot for your comprehensive explanation. The point I was trying to make, and you helped me to see it more clearly, is that depending on the project - for example web ones - you are not really interested by rolling back, but rather by rolling over. Because if a bug is found you don't need to patch the previous version since nobody is using it, you only need to fix the bux in the next release. And I think that since new projects are becoming mainly SaaS and web projects, it is important to outline it. Thank you for addressing all the points I made. – tobiak777 Apr 1 '15 at 11:19
  • 2
    At the time the book was written, not many companies were able to do 75+ deploymeents to production a day. Nowadays companies and products like Etsy, Facebook, Visual Studio Online, all are able to roll forward, instead of roll back. It requires a hefty investment in infrastructure though, especially if the site isn't relatively static and self-contained. – jessehouwing Apr 1 '15 at 11:21
  • That makes sense, thank you so much I've been puzzled by this question for quite a while. – tobiak777 Apr 1 '15 at 11:23
  • @jessehouwing yep and that's why the top 10% of companies do this and all others (especially in Canada) are only able to do deploys at most twice a day and that's only to staging environments, production deploys are still in a weekly or monthly cycle. – Rudolf Olah Apr 1 '15 at 21:08
  • 1
    It may take you 4 days to solve the issue. Or your build and verification may take half a day... – jessehouwing Apr 2 '15 at 11:04
1

Being able to rollback to the current version is a different way of saying "rollback to the previous version", now to do that you have 2 ways of doing this:

  • You can simply make a copy of the current version before deployment, and if anything goes wrong, just replace the new version with the backup copy.
  • You can create a system where you can deploy any version at all, so deploying a new release means running the deploy task with the newer version, and rolling back is just the same process - only you specify the previous version.

The 2nd way is obviously more work, but it is far superior - it is the "right" way of doing this. You shift the problem of rolling back to one of deployment, so you do not need special work to make the backup and re-deploy the backup. Saving yourself this extra work means you can spend it on creating the correct deployment mechanism - which you have to do anyway as its what you use for deployment of new versions!

So, setting up your CI system to deploy is easier, better and also lets you deploy very old versions for free. In some businesses this is essential as you may be called upon to debug a problem a customer has in their version - so you'll need a way to re-create the version they are running. You may not need this for your single-deployment website so you aren't seeing what the benefit might be, but trust me - its very real.

Set up your CI system like this and you'll be set for the future, and any future roles you may have where you do need a fully working CI system, you'll have experience setting it up correctly.

0

"Once it's deployed and working, why would you ever want to go back to a previous version ?" The key words here being "and working". In a perfect world, every new version would be thoroughly tested and completely accurate before it is deployed. But in real life, that doesn't always happen. What if you discover a critical bug that was introduced, not in the current version, but in the previous version? You many need to roll back two versions. Lots of applications these days, especially web sites, have a fairly high deployment rate. If you're putting out a new release every day, it wouldn't be shocking if a bug that was introduced three days ago wasn't discovered until today.

Even with a relatively slow release schedule, bugs could go undetected for a long period of time. Like many applications have "year end reporting". If we released a new version in January that created a bug in the year end reporting, it might go unnoticed for almost a year. (Whether rolling back a year's worth of releases is a practical solution, well, depends on the nature of the app and how much it is changing. Presumably any rollback is temporary, just until you get the current version fixed and can redeploy.)

  • You may be able to roll back just the report module... If your architecture would allow that ;)> – jessehouwing Apr 1 '15 at 14:33
0

Never mind the current deployment, if you (or your customers) run a previous version you may need to get a snapshot of that version going to reproduce an issue for an incident or compare current undesirable behaviour to a previous build. That includes not just the application, but test data, and test details as well.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.