0

If a feature that was done in 2, 3 months and before it was merged into the main branch, and both branches were under constant changes every day, should integration tests be written at this stage?

Will it be too early, because both branches are changing, the UI were changing, and even the specs were changing. If the integration tests are asked to be developed at this stage, you can wait for the test to pass the Jenkins or if somebody already merged, then you have to re-merged and go through the Continuous Integration tests for another 2 hours, and in one day or two, the test will already fail due to things changing so fast.

Would that be too soon? Would it be more appropriate when coding have all finished, and with the feature branch merged into the main branch, that the integration tests start to be written?

Some moderator close this question saying: Possible duplicate of How big does my project need to be for me to unit test it? It is not a duplicate. This question is about when. That question is asking about what size.

  • Sounds like your team needs to have a discussion about this to determine what the best way forward is. – Robert Harvey Sep 8 '17 at 6:51
  • 1
    There is no too soon. But they should exist in the same branch as the feature is developed in. This way the yet failing tests do not break the stable branch. You may have to setup an additional job in your jenkins to execute the test on the feature branch... – Timothy Truckle Sep 8 '17 at 6:55
  • @TimothyTruckle there is no too soon? How do you reason for that? So if there is no too soon, even before the specs are written, you should write the integration tests, because "there is no too soon"? – 太極者無極而生 Sep 8 '17 at 7:07
  • Possible duplicate of How big does my project need to be for me to unit test it? – gnat Sep 8 '17 at 7:07
  • @gnat totally unrelated – 太極者無極而生 Sep 8 '17 at 7:07
4

Too early to start writing the tests is when the specification hasn't been written yet. Once you have the specification you should start writing the tests waiting for the code is much too late and you will never catch up.

Ideally your coders should be writing code which can be tested to verify it satisfies the design requirements as soon as they think that they are finished and before they start thinking about moving on to other things.

In practice you will still end up modifying and supplementing tests during the coding but you will have something to work with.

Read up on Test Driven Development but the same principles apply to just about every other workflow.

To clarify - it is entirely possible that the specification writers, coders & test writers may be the same people or even the same single person, (as pointed out by Doc Brown but:

  • If you don't have a fixed, issued, specification you don't know where you are going, when you have got there or even if you have got there - writing or even finalising the specification after the project is documentation not specification.
  • Without a clear specification you are prone to feature creep.
  • Of course during the course of almost any project the specification will change but if you have good traceability between specification and tests, you mean you don't know which requirement this test is for & which tests demonstrate this requirement is satisfied, then you know which tests need updating, the same applies to the code - you also have a clear route to pointing out to management when specification dithering and late changes are costing money (thanks to Bart van Ingen Schenau for mentioning this) and should be left to version 2,3...
  • The other argument for good traceability is so that when a part of the code doesn't work, as demonstrated by your tests, you will know which features will be missing if you simply disable it for this release.
  • One auditor once told me that whenever they were presented with the specification and it was at Issue 1 they knew the project as going to be a nightmare.
  • With change control on the specification you have a clear record of which requirements were changed, when, by whom and hopefully why and with how much impact on the coding and testing - when the project overruns or overspends this can be the developer and testers best friend.

As a result of all of the above I personally think that it is a great idea to have a specification even for solo projects and certainly for anything complex enough to require formal tests.

The other thing that is would like to point out, (in answer to 太極者無極而生) is that if you wait to formalise the specification and write the tests until the code is done then the specification may never get done, the testing probably will never be done and any problems found will possibly never get addressed because the budget and time frame will both have been used up and there will be a lot of pressure to get things out of the door. I speak from experience as I am, not for the first time, having to rewrite someone else's code that has been a maintenance nightmare for years because:

  • The original specification was never released, never updated, became completely out of step with the delivered system, (to the point where the physical system only meets the specifications for the external dimensions but in no other respect.
  • There were never any formal tests developed for the software, (after all without a specification that resembled the system there was no way to write meaningful tests).
  • The budget was gone so some "does it look like it works" testing was done and the systems shipped so that the customer could be billed.
  • Patching one problem caused new or old problems to appear because there were no tests available to check the system still worked properly.
  • 1
    I actually quite disagree with written integration tests before you write the code. The specs can constantly change. Management can say this has to change and that has to change throughout several months, and you end up using two times the effort to modify the tests and modify the code. Maybe somebody likes it, but I don't. – 太極者無極而生 Sep 8 '17 at 7:20
  • 3
    @太極者無極而生, Management can say at any time that something has to change. If you wait with writing your tests until the specs are completely stable, you can't write them until the product is decommissioned or discontinued. Everybody in the organization should be aware that changing specs after implementing them has started incurs a cost that the tests also need to be adapted. – Bart van Ingen Schenau Sep 8 '17 at 10:25
  • In this answer there are "spec writers", "coders" and "test writers" seen as different people. That is probably not the way every team works. – Doc Brown Sep 8 '17 at 11:34
  • @太極者無極而生 - I have addressed your points in the main question. – Steve Barnes Sep 8 '17 at 17:40
  • @DocBrown - Good point I hope I have addressed with an edit. – Steve Barnes Sep 8 '17 at 17:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.