How can I organise a better deployment process? [closed]

Question

I work in a company of about 10 developers. When we start a project, each of the developers involved in the project have their dev environment. Once they complete a task, it is then pushed to phase two which is called "Staging", where QA takes place. Once bugs are cleared, it is then set to "Ready for UAT" and UAT (User Acceptance Testing) is done on the same phase (Staging). If the client is happy with the work it is pushed to the last stage which is called "Live".

Detailed development overflow

1. Specification
2. Development (Once specification is approved by client)
3. Staging QA
4. Staging UAT
5. Go Live Deployment
6. Stage of Support (1 month of free support, this is to give the client a change so point out things we/they didn't notice on the QA and UAT stage)

Once the stage of support ends and the client wants to change something to the software, they need to rise a CR (Change Request), and it all start's from stage one again (Big change requests, things such as new functionality to the current system)

If the change request is small then no specification is needed. It will start from development stage to go Live deployment. If the client spots something wrong after the go live for this CR, then he will have to rise a support case and it starts all over again from the development stage.

My question is, could there be a better way of doing this?

Answer 1

I'd suggest continuous integration of any useful incremental work from any developer in a single integration branch, with automated and manual QA done as often as possible on the integration branch. Everybody is on the same page, no room for "wandering off" or wasting effort and resources polishing changes (in individual dev environments) which may be completely invalidated when integrating with other similarly polished code from other individual dev environments.

In such environment you can apply in a much more effective manner other useful software development strategies like Agile, TDD, etc.

For example, applying Agile:

• steps 1-4 all happen on the same/main branch, simultaneously. The specifications are broken down in many small pieces which can be developed, QA'd, UAT'd, and delivered incrementally (in max a few days). The client can see the incremental results early, incrementally adjust the small specs and re-prioritize them as needed. By working closer with the client in steps 1-4 QA and UAT can really become one, which is ideal IMHO. TDD can also be thrown in the mix here.
• Steps 5-6 are done on short-lived (1-4 weeks, the shorter the better) versions (ideally just selected CI/CD labels/tags on the main branch), mostly producing new incremental specs for subsequent such versions (instead of producing support requests for longer life versions each in their own branches), until the final client-satisfying product is achieved (much faster than through the repeated longer iterations you have now).
• Steps 5-6 for one version are done in parallel with steps 1-4 for the next version.

The team's focus and resources are mostly directed towards reaching the final product version the customer desires instead of "wandering off" supporting non-final versions, which can easily lead to exploding costs.

Answer 2

Your user acceptance tests are way too late. You risk expending much effort on conforming exactly to requirements which then turn out not to be what the customer needs. If at all possible, involve the actual users of the software in the quality control process as soon as there is something to test - don't wait until after your testers think it's perfect.

Answer 3

There are lots of questions that need to be answered here:

• Are you using a vcs?
• What is your branching strategy?
• Can you deploy using a single button?
• Do you have automated tests?
• When do you run the tests?
• Do you have any continuous integration?
• What happens with your team while the job is on the Staging area? Are you working on something else? Or just waiting?
• What happens with the QA team while the UAT part is going on?
• How long it takes you to deploy?
• Does deployment mean that your user cannot work on the system?
• Do you have a database? Can you undo a deployment on your database?

The basics of what you are doing is fine, mostly because of that luxurious trait of having a fully closed specification (never happened to me). But the basics don't make your life easier, nor provide the best value for your company.

I am supposing you have a vcs. Which branching strategy are you using? Those strategies will change based on the vcs. I would choose a different branching strategy if I am using Subversion/TFS Source Control or if I am using distributed ones like git or hg.

Have tests(unit, integration, requirement), and make sure that you can run them automatically. Tests should be run on a CI environment that goes between the dev and the staging environments. Any bug encountered by the QA team should become an automated test where possible (depending on your UI technology that could not be always possible). These actions will: reduce the time spend on "Staging", reduce the number of bugs that appear, reduce the number of regression errors that will appear.

The CI environment should automatically compile and run all the tests (it could be well possible that you want to run unit tests all the time, but integration tests only at one specific point of the day). This action will provide your team with assurance that either you are not breaking anything or that you will know pretty much immediatly. Tools for CI? Jenkins, Team City, TFS, ...

At some point I was working on a team on which a deployment of our application meant usually a two hour manual process that needed all users to be logged out. We managed to improve it, though still manual, to less than 5 minutes with only a restart for the users if there was no database change. If there was a database change, it still required half hour of the users being out. But the time I left the company we were looking at using tools (Octopus on this case) to make it even easier and a green/blue strategy on our database to eliminate completely downtime (except for the restart of the client terminals).

I would completely separate the staging environment for internal QA from the staging environment for UAT. My previous experience is that is very messy to have them together.

I would have a prelive environment just to test the deployment on the exact same conditions as there are in live. I had suprises before because our system was depending on some library or OS patch that was present on all stages but Live.

Which then will mean that there is another question, can your IT team replicate the setup of live? If they cannot they should be working on it. The provisioning of a machine/environment should be completely automated once you have the hardware.

You can always improve your process. At the beginning you usually can take some big strides, later one, is more fine tuning. You should ask yourself (and your team) the question that you have posted every month.

Answer 4

When anyone asks any question of the form "Is there a better way to x," the answer is always yes. ^_^

In general, though, if your team is meeting targets and generally satisfied with the current process, the cost of rocking that boat is often not worthwhile.