0

We just migrated to git from svn and are trying to define a process that would help us achieve the following:

  1. Deployment to dev env without requiring code review.
  2. Have a pull request / code review process when merging into the release branch.

Environment:

  1. We have a team of 3 developers.
  2. There are 3 envs - Dev, Test and Prod.

Proposed Process:

  1. We have 3 git branches.

    • dev: Used to merge all features and deploy to Dev env.
    • release: Used to merge all features and deploy to Test and Prod envs
    • master: Merged from release
  2. Developers work on individual feature branches off created off dev branch

  3. Developers cannot test their changes locally and hence are required to move their code to Dev env to test. So they merge their changes into the dev branch when they want to test.

  4. There are frequent such merges to dev branch from different feature branches as everyone tests out their features and fix the bugs.

  5. A week before prod release date, developers create Pull requests to merge individual features into the Release branch for features which are ready to go live to Prod.

  6. Their pull requests will be reviewed by peers and merged into the release branch.

  7. After all PR are merged, the artifact is tested by QA team in Test env.

  8. Assuming all is well this artifact will be deployed to Prod and the release branch will be merged into master. Marking end of release

I would like to get feedback on the proposed process. Also I have below concerns around this process: In point 6 of Proposed process - would developers need to rebase / merge their feature branches with release branch and again push the changes in the feature branch in order to be able to merge? And would it matter in what order the peers review each feature branch and merge into the release branch?

  • 5
    Honestly? This sounds like a nightmare. I would never trust the development environment if features are going to be independently merged onto an entirely separate branch for release. This whole thing sounds like a band-aid over a more fundamental problem - why can't you make smaller, more frequently merged changes and have the code review occur before deployment to dev? Why can't you have local development environments? – Ant P Jul 26 at 13:58
  • 2
    Your proposed branching model seems too complicated for a team of 3 developers. So is Gitflow, @Theraot. – Robert Harvey Jul 26 at 16:15
  • 1
    We tried to implement the above today and faced a road block. On Point 5 when the developers try to merge their feature branches into the Release branch - we noticed that the feature branch does not only have its own change but other changes from merging the dev branch back into the feature branch to enable the feature branch to be merged into the feature branch So it seems likes there is no other option for us but to adopt gitflow only. We will need to do PR for every change going into the Dev branch and Merge entire dev branch into the release branch when ready for release. – amit9921 Jul 26 at 18:52
  • 1
    @amit9921: I looked at your question again, and I see a lot of detail about the process that you're using that doesn't work, but not a lot of detail about what you intend to do about the problems that you do have. Have you given some thought to solving the biggest problem that you have first, which is that your developers cannot test locally? Have you given some thought to vastly simplifying your branching model? You can always complicate it when your team gets larger, if you want to. – Robert Harvey Jul 27 at 0:02
  • 1
    Testing locally may be prohibitively hard, but making it possible to test individual feature branches looks as a reasonable mitigation. – Alex Cohn Jul 27 at 6:37
5

Developers cannot test their changes locally

This is a gigantic problem, all your other issues stem from this problem and it should be your top priority to solve this. How you solve this will depend heavily on your set up and is worth a separate question. Therefore everything else I'm going to say is irrelevant, but:

I think you have a nasty mix of git flow and Trunk based development, git flow basically isn't going to work for you with the "can't test locally" problem.

Developers work on individual feature branches off created off dev branch

This is a problem*, this means my branch for feature A contains all the in progress work of everyone else's, none of which has been code reviewed. This means when I put in a pull request it contains everyone else's unfinished work which must then all be reviewed together. Your work shouldn't be branched off any branch that hasn't been code reviewed [Exceptions of course apply, but as a general rule]

*Other workflows may do this with a branch called dev and be fine, but in this case dev is a branch that just contains a grab bag of everyone in-progress stuff.

Developers cannot test their changes locally and hence are required to move their code to Dev env to test. So they merge their changes into the dev branch when they want to test.

Fix this, the whole process will be broken while this is the case

A week before prod release date, developers create Pull requests to merge individual features into the Release branch for features which are ready to go live to Prod.

I'm assuming you mean "At least a week before .." not "A week before ...". Completed work should be code reviewed and merged as soon as possible. Having many feature branches hanging around just leads to merge conflicts.

After all PR are merged, the artifact is tested by QA team in Test env.

This is a bit waterfall but not fatal. Ideally the features should be tested as soon as they are finished, so that when bugs are found people still remember what all the pieces were supposed to do

Summary

All that said, what you effectively have is "Everyone commits directly to master and code reviews happen on a per commit basis after the fact, release branches taken for release to prod" but with extra steps (like calling master dev, branching in and out of dev for no reason and merging from your branch that contains all of dev into a release branch). If you're going to effectively do that, just do that. [I personally don't like that, but for a small team like you have it can work].

  • Thanks for taking time to understand and question and provide your input. I agree with your observation that the problem stems from the fact that Developers are not able to test locally. We invoke spark jobs which use resources on cluster which we were unable to run from local with earlier version of spark and CDH. We recently upgraded and we can give that a try again now. Regarding your summary - We do not commit directly to master as we want it to be used for prod tagging. Also Dev branch could have more changes in it which decide not to release. – amit9921 Jul 26 at 18:48
  • dev branch having more changes that you decide not to release requires discipline. I mean, when a release branch is created, it should point at an existing commit on the Dev branch (probably, this commit will be on the master branch, too). From now on, some changes in dev may be propagated (cherry picked) to release branch. – Alex Cohn Jul 26 at 19:55
  • @amit9921 As a slightly painful version could you have 3 "dev" environments (one for each developer) that can be pointed at arbitrary branches to allow each developer to have their own "local in the cloud"? – Richard Tingle Jul 26 at 20:37
  • @AlexCohn The proposed plan was to merge individual feature branches into the Release branch and not the dev branch (as mentioned in point 5) - however as we have realized today (pls see my comment in the question) this is not feasible as the feature branches have code changes merged from the dev branches too. – amit9921 Jul 26 at 21:31
  • @RichardTingle having and maintaining 3 different locals will be quite costly. plus its not scalable too. This leaves us with the option to follow git flow only. If we are able to resolve local testing we should be OK. Although I do find git flow also too onerous for a small dev team. – amit9921 Jul 26 at 21:34
1

It would be worthwhile to have feature branches testable on the server, just like dev branch is testable in your proposed setup. With only 3 developers, this won't be an unsustainable multiplication of testing environments; you could probably force Jenkins or whatever CI you use to reuse same testing environment for all feature branches (this would mean that developers will not be able to test their changes in parallel, but that's what you already have if they all have to merge to dev branch).

Having performed tests, a developer would start a pull request to dev branch. In the ideal world, such PR would automatically trigger an automated test, but waiting for code review won't delay further development, possibly of a different feature, likely branched from the dev branch.

Sometimes, new feature branch will continue from a pending-on-code-review feature branch. This may require an extra merge to the new feature branch when the PR gets finally accepted, but it should not be a terrible merge.

At any rate, the feature branch must get merge from the dev branch every time the latter is advanced (one of its pending PRs getting accepted). If a PR lingers for a long time, this can get painful. But if the PR is marked WIP, work in progress, it can afford to be not up-to-date vis-a-vis the dev branch.

  • In proposed flow the developers should be able to test their changes in parallel as they merge their changes into the dev branch which has other developers changes and the merged code gets deployed to dev env. Having deployments done from individual feature branches would mean developers will need to co-ordinate their testing - and in our case commits as they the deployment to dev is auto deploy. – amit9921 Jul 26 at 21:41
  • 1
    Exactly, this is the core issue IMO. If I am forced to merge with others' untested code to test my change, I must co-ordinate with them much more than time-sharing the test servers. – Alex Cohn Jul 27 at 6:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.