Let's say in an effort to improve quality, your team agrees on the following policy:
all commits to the master branch (direct or via pull request) have to be categorized as either bugfixes or feature changes; never both at the same time.
to easily distinguish the two, commit messages have to start with e.g.
[bug 123]
or[feature 456]
, respectivelyin feature commits, all code changes have to be feature-flagged (using some sort of application settings tool such as NFig). The idea is that while the feature flag is turned off, the commit is not supposed to change app behavior in any way.
EDIT: feature-flagged are supposed to be short-lived and only for release purposes. Martin Fowler calls those release toggles.
So for example, a feature commit like this...
function existingCode() {
// existing stuff
+ doSomethingNew(); // <--- NEW
// more exiting stuffstuff
}
... would be in violation of the policy because doSomethingNew()
introduces an immediate change in behavior. Feature-flagging the change like this...
function existingCode() {
// existing stuff
+ if (Config.FeatureFlags.NewFeature456)
+ {
+ doSomethingNew(); // <--- NEW
+ }
// more exiting stuffstuff
}
... would be compliant with the policy: there is no change in behavior until the NewFeature456
feature flag gets enabled (and therefore no risk of accidental breakages, feature leaks etc).
My question is: would it be conceivable to enforce the feature flag policy with an automated test?
The way I would imagine a test like this to work is, roughly:
- inputs: git repo, commit hash, name of feature flag
- build repo with and without commit (feature flag disabled in both)
- check if both builds behave the same
That last "behaving the same" part is obviously the tricky part. A number of things come to mind here, from running a number of other automated tests to code analysis - something like comparing the ASTs of both builds (that might get difficult).
Could that possibly work? Has it been done before?
all commits to the master branch
there is your problem. NEVER commit to the master branch. ONLY EVER merge to master from a branch that, prior to merging, has undergone a full test-cycle.