Return to Answer

This question boils down to how to handle failures in db and application upgrades, and which one should be done first, in a situation where there are lots of application instances and one shared DB.

If it is critical to keep the downtimes as small as possible, I would try to split the deployment into the following steps:

1. Set the applications into "read-only" mode for a short time to prevent data loss (you may have to implement such a feature).

2. Make only changes to the DB like adding columns and tables which are "backwards compatible enough" so the old application version can work with them (at least in read-only-mode). Even an operation like "split a column into two" with some data migration is possible here, as long as the original column is not deleted right now. If this step fails, roll back the DB.

3. Then, upgrade the applications. Any application which was updated successfully can be switched to read-write-mode again. If one application server upgrade fails, fix the issue (or take the server offline). Don't proceed until all running applications are upgraded.

4. Make the non-backwards compatible changes to the database, like deleting tables or columns which are not needed any more. (I guess often there are no such changes).

That way, you won't need a rollback system for the applications, and you don't have the risk of silent data corruption.

This question boils down to how to handle failures in db and application upgrades, and which one should be done first, in a situation where there are lots of application instances and one shared DB.

If it is critical to keep the downtimes as small as possible, I would try to split the deployment into the following steps:

1. Set the applications into "read-only" mode for a short time to prevent data loss (you may have to implement such a feature).

2. Upgrade the DB (schema and data) to an intermediate version, whery only changes to the DB are applied like adding columns and tables which are "backwards compatible". So the old application version can work with them at least in read-only-mode. Even an operation like "split a column into two" with some data migration is possible here, as long as the original column is not deleted right now. If this step fails, roll back the DB.

3. Then, upgrade the applications. Any application which was updated successfully can be switched to read-write-mode again. If one application server upgrade fails, fix the issue (or take the server offline). Don't proceed until all running applications are upgraded.

4. Upgrade the DB to a final version, applying any non-backwards compatible changes to the database, like deleting tables or columns which are not required any more by the new application version.

That way, you won't need a rollback system for the applications, and you don't have the risk of silent data corruption.

This question boils down to how to handle failures in db and application upgrades, and which one should be done first.

If it is critical to keep the downtimes as small as possible, I would try to split the deployment into the following steps:

1. Set the applications into "read-only" mode for a short time to prevent data loss (you may have to implement such a feature).

2. Make only changes to the DB like adding columns and tables which are "backwards compatible enough" so the old application version can work with them (at least in read-only-mode). Even an operation like "split a column into two" with some data migration is possible here, as long as the original column is not deleted right now. If this step fails, roll back the DB.

3. Then, upgrade the applications. Any application which was updated successfully can be switched to read-write-mode again. If one application server upgrade fails, fix the issue (or take the server offline). Don't proceed until all running applications are upgraded.

4. Make the non-backwards compatible changes to the database, like deleting tables or columns which are not needed any more. (I guess often there are no such changes).

That way, you won't need a rollback system for the applications, and you don't have the risk of silent data corruption.

This question boils down to how to handle failures in db and application upgrades, and which one should be done first, in a situation where there are lots of application instances and one shared DB.

If it is critical to keep the downtimes as small as possible, I would try to split the deployment into the following steps:

1. Set the applications into "read-only" mode for a short time to prevent data loss (you may have to implement such a feature).

2. Make only changes to the DB like adding columns and tables which are "backwards compatible enough" so the old application version can work with them (at least in read-only-mode). Even an operation like "split a column into two" with some data migration is possible here, as long as the original column is not deleted right now. If this step fails, roll back the DB.

3. Then, upgrade the applications. Any application which was updated successfully can be switched to read-write-mode again. If one application server upgrade fails, fix the issue (or take the server offline). Don't proceed until all running applications are upgraded.

4. Make the non-backwards compatible changes to the database, like deleting tables or columns which are not needed any more. (I guess often there are no such changes).

That way, you won't need a rollback system for the applications, and you don't have the risk of silent data corruption.

This question boils down to how to handle failures in db and application upgrades, and which one should be done first.

If it is critical to keep the downtimes as small as possible, I would try to split the deployment into the following steps:

1. Set the applications into "read-only" mode for a short time to prevent data loss (you may have to implement such a feature).

2. Make only changes to the DB like adding columns and tables which are "backwards compatible enough" so the old application version can work with them (at least in read-only-mode). Even an operation like "split a column into two" with some data migration is possible here, as long as the original column is not deleted right now.

3. Then, upgrade the applications. Any application which was updated successfully can be switched to read-write-mode again. If one application server upgrade fails, fix the issue (or take the server offline). Don't proceed until all running applications are upgraded.

4. Make the non-backwards compatible changes to the database, like deleting tables or columns which are not needed any more.

This question boils down to how to handle failures in db and application upgrades, and which one should be done first.

If it is critical to keep the downtimes as small as possible, I would try to split the deployment into the following steps:

1. Set the applications into "read-only" mode for a short time to prevent data loss (you may have to implement such a feature).

2. Make only changes to the DB like adding columns and tables which are "backwards compatible enough" so the old application version can work with them (at least in read-only-mode). Even an operation like "split a column into two" with some data migration is possible here, as long as the original column is not deleted right now. If this step fails, roll back the DB.

3. Then, upgrade the applications. Any application which was updated successfully can be switched to read-write-mode again. If one application server upgrade fails, fix the issue (or take the server offline). Don't proceed until all running applications are upgraded.

4. Make the non-backwards compatible changes to the database, like deleting tables or columns which are not needed any more. (I guess often there are no such changes).

That way, you won't need a rollback system for the applications, and you don't have the risk of silent data corruption.