Deploying code without unit tests to give testers more time

Question

So we work in a structure with three week sprints, and we want to give testers time to test.

Currently our process is, write functionality > write unit tests > code review > deploy to testing environment.

What I thought of is that when we have functional code, why are we not deploying it to give testers more time to test things, of course still have a code review, but often writing unit tests can take another day or a few if it's something complex.

So the idea being, write functionality > code review > deploy > write unit tests (and fix bugs) > code review > deploy to testing environment.

The only thing here being that you've got two separate code reviews which takes more time from the code reviewer. Additionally, there is the risk a developer just doesn't write the unit tests afterwards.

Perhaps the answer to this question is not take in work that has the potential to run over, or get better at estimating etc. but when this situation does occur, what's the problem with using this as a backstop?

Answer 1

I cannot imagine you really meant to shift any kind of unit testing to the point in time after the time of a deployment. I can imagine, however, the Sprint process you really had in mind looks more like this:

1. Loop several times per day:

   • write functionality
   • maybe do some manual testing
   • write and run unit tests

   (not necessarily in that order, especially not when doing TDD)

2. Deploy intermediate release to the testers

3. Loop again

   • Write additional automated tests
   • fix bugs reports from the testers, or from your own tests
   • refactor

4. Deploy release candidate to the testers

That can be a reasonable approach (I left the code review phases out for simplicity, add them where you think they work best in your team). Maybe that is what you meant, not sure, the process description in your question is a little bit terse. Note also in step 3, when those tests take a day to write, they are most probably not unit tests.

Passing intermediate releases to testers can be a good idea, to get earlier feedback, but you have to make sure you know what you are doing:

• For example, I would not pass anything completely untested to the testers. You burden the testers with issues you could have identified way more easily and way more quickly by yourself beforehand. Even worse, there is a certain risk the code might so broken in certain areas testers cannot even start with testing the functionality they want to see, or the bugs in the code will shadow other bugs.

• When you hand too many half-baked features over to the testers, there is a certain risk that the testers have to invest a lot of extra work into things which will never make it into the final product.

  (However, sometimes it can be the right thing to stop delivering a new feature just because the testers found too many issues during intermediate tests).

Answer 2

Let me put it this way: it is a lot less hassle to find a bug with a unit test than with a tester. When a unit test fails during development, it typically takes a couple minutes or less to fix and they move on. No paperwork.

When a tester finds a bug, they first have to verify it's actually a bug. Are they using this new feature correctly? Are the understanding the UI correctly? This often involves lengthy discussions with a developer, who finally agrees it's a bug. Then a jira must be created. Then the developer goes through the entire process of fixing the bug, manually testing it, getting the code reviewed, making a release, and sending the jira to retest. In the meantime, the tester may be blocked from further testing, or they may feel they have to redo the testing once the bug is fixed. That cycle typically takes a few days at least.

I want my testers to find tricky important bugs, not spend all their time on stupid little ones.

Answer 3

What I thought of is that when we have functional code, why are we not deploying it to give testers more time to test things

If you haven't tested it, how do you know it's functional?

If you actually know it's functional, it must either be entirely trivial, or already have unit tests that have passed. Right?

Unless you just tested it manually, in which case you could have spent that time writing a unit test that would automate the check on every future build.

... often writing unit tests can take another day or a few if it's something complex

That doesn't sound like a unit test. Are you sure it's not something higher-level?

... isn't it better than testers sitting around doing nothing?

Perhaps the testers can work on automating integration tests (and perhaps the unit tests that take you more than a day are actually integration tests they could take over). Maybe they can work on scripting some of their integration tests to run in your CI setup.

Testers are often sat around waiting for us to deploy code which is pretty bad

Why don't you give them the spec, so they can figure out how to test it while you're figuring out how to write it?

This is extra useful because you have an extra set of eyes looking for ambiguity or inconsistency in the spec (similar to the way you often see problems differently while writing unit tests than when coding with your "optimistic path" glasses on).

Answer 4

You should hand a new version to testers when all stupid little bugs that you should have fixed are fixed. If you are so good at writing code that your unit tests never find any problems, then hand it over before you write unit tests. But if your unit tests tend to find bugs, its much more efficient to pass on new versions after writing tests and fixing problems.

Basically, pass your work on when you are reasonably confident that QA wont find any problems.

PS. There’s an exception when you fix a blocking issue. If your application requires login, and the login functionality is broken, and QA can’t test anything because of this, then you fix it ASAP without caring about any bugs in the login code.

Answer 5

The world today is inching closer to "developers write all tests". With respect to functional testing; the check in should not allow developers to check in code without the proper level of tests included. Angular is a great example in that it includes testing stubs for all new projects.

The idea is that the developer code also contains a full suite of new and regression tests which provide the assurance of product quality. If the QA team is truly embedded with the development team then they are working together on the same sprint using the same language. Following this pattern works for both front and back end developers, but no lag in sprint is needed for this, as the tests are a requirement for each sprint.

E2E Testing is a different story, as it tests the interfaces between components. It is much larger in scope and requires it's own sprint planning and deliveries. This could be a QA Team only responsibility; as often, the tools are different and developers are too busy holding down their own responsibilities.

Anyway we look at this, manual testing is still relevant, but in diminishing fashion as today's tools cover a lot of what manual testing did in the past. It's my opinion that everyone working in software must be a programmer.

Answer 6

There is a reason that the industry is moving to wiring the tests first, before you write the code, and it's pretty simple - if you write the tests first you end up structuring the code to make it easy to test.

It's very easy to write code that is difficult to test. And generally speaking, if you're not thinking about the tests when you are writing the code you will end up with code that isn't easy to test.

Write tests first.

Answer 7

Other answers have explained why unit tests should be considered an essential part of your definition of done, but I want to comment on one alternative that may help with giving your tests more time to test: using shorter iterations.

It's possible that your testers need more time because their time isn't being used efficiently. With three-week sprints, I'd imagine that they don't have a whole lot to test for the first couple of weeks (or possibly the middle, if they spend the first week testing the previous sprint). Then at the end, all of the tests for the sprint fall into their lap at once.

If you can break the work into smaller iterations (even if some tasks have to be continued across iterations without deliverables), you may be able to hand tests off to your testers at a more consistent pace.