33

So we work in a structure with three week sprints, and we want to give testers time to test.

Currently our process is, write functionality > write unit tests > code review > deploy to testing environment.

What I thought of is that when we have functional code, why are we not deploying it to give testers more time to test things, of course still have a code review, but often writing unit tests can take another day or a few if it's something complex.

So the idea being, write functionality > code review > deploy > write unit tests (and fix bugs) > code review > deploy to testing environment.

The only thing here being that you've got two separate code reviews which takes more time from the code reviewer. Additionally, there is the risk a developer just doesn't write the unit tests afterwards.

Perhaps the answer to this question is not take in work that has the potential to run over, or get better at estimating etc. but when this situation does occur, what's the problem with using this as a backstop?

16
  • 21
    ... note TDD people would probably say that your order of working is utterly wrong, it should be loop several times over "write unit tests -> write functionality", then comes code review and deploy.
    – Doc Brown
    Aug 27 at 10:52
  • 6
    You suggest to ignore code best practices in favor of artificially imposed limits. Another example scrum mainly benefits management, developers don’t need sprints to deliver working software.
    – Rik D
    Aug 27 at 11:40
  • 63
    Downvotes are for bad questions, not for bad ideas!
    – Doc Brown
    Aug 27 at 11:58
  • 5
    Does "deploy" mean deploy to production, or to an internal staring server where only testers would see it?
    – max630
    Aug 27 at 14:00
  • 3
    Yes internal testing server, I should probably elaborate a bit more.
    – DubDub
    Aug 27 at 14:30
23

I cannot imagine you really meant to shift any kind of unit testing to the point in time after the time of a deployment. I can imagine, however, the Sprint process you really had in mind looks more like this:

  1. Loop several times per day:

    • write functionality
    • maybe do some manual testing
    • write and run unit tests

    (not necessarily in that order, especially not when doing TDD)

  2. Deploy intermediate release to the testers

  3. Loop again

    • Write additional automated tests
    • fix bugs reports from the testers, or from your own tests
    • refactor
  4. Deploy release candidate to the testers

That can be a reasonable approach (I left the code review phases out for simplicity, add them where you think they work best in your team). Maybe that is what you meant, not sure, the process description in your question is a little bit terse. Note also in step 3, when those tests take a day to write, they are most probably not unit tests.

Passing intermediate releases to testers can be a good idea, to get earlier feedback, but you have to make sure you know what you are doing:

  • For example, I would not pass anything completely untested to the testers. You burden the testers with issues you could have identified way more easily and way more quickly by yourself beforehand. Even worse, there is a certain risk the code might so broken in certain areas testers cannot even start with testing the functionality they want to see, or the bugs in the code will shadow other bugs.

  • When you hand too many half-baked features over to the testers, there is a certain risk that the testers have to invest a lot of extra work into things which will never make it into the final product.

    (However, sometimes it can be the right thing to stop delivering a new feature just because the testers found too many issues during intermediate tests).

7
  • 7
    You should consider that you think that you are giving more time to the testers to do testing, but was is likely to happen is that there will be more bugs in the final code which will result in more work for the tester as they have to raise more bugs. One of the benefits of unit testing is that you might come up with scenarios that you didn't realize before.
    – JCalcines
    Aug 27 at 12:27
  • I know, my comment is just an extension. I was writing an answer in the same lines as yours. I just wanted to highlight the risk of going down that road.
    – JCalcines
    Aug 27 at 12:36
  • 1
    @JCalcines: made some final additions to my answer. Your first comment, however, reminded me of this: One of the benefits of handing early versions over to the testers is that they might come up with scenarios that I didn't realize before,
    – Doc Brown
    Aug 27 at 13:00
  • So this might be turning into another question. While the problems you've raised are true, isn't it better than testers sitting around doing nothing? Just looked at this answer and thinking this is a completely different thing: pm.stackexchange.com/questions/16361/…
    – DubDub
    Aug 27 at 14:55
  • I think the more I'm thinking it through that's a separate problem and they both need to be addressed individually. Testers are often sat around waiting for us to deploy code which is pretty bad, and I'm looking at this and thinking it's better they're doing something rather than the problems you've described, where as ideally neither should be happening, thanks for the answer.
    – DubDub
    Aug 27 at 14:59
44

Let me put it this way: it is a lot less hassle to find a bug with a unit test than with a tester. When a unit test fails during development, it typically takes a couple minutes or less to fix and they move on. No paperwork.

When a tester finds a bug, they first have to verify it's actually a bug. Are they using this new feature correctly? Are the understanding the UI correctly? This often involves lengthy discussions with a developer, who finally agrees it's a bug. Then a jira must be created. Then the developer goes through the entire process of fixing the bug, manually testing it, getting the code reviewed, making a release, and sending the jira to retest. In the meantime, the tester may be blocked from further testing, or they may feel they have to redo the testing once the bug is fixed. That cycle typically takes a few days at least.

I want my testers to find tricky important bugs, not spend all their time on stupid little ones.

1
10

What I thought of is that when we have functional code, why are we not deploying it to give testers more time to test things

If you haven't tested it, how do you know it's functional?

If you actually know it's functional, it must either be entirely trivial, or already have unit tests that have passed. Right?

Unless you just tested it manually, in which case you could have spent that time writing a unit test that would automate the check on every future build.

... often writing unit tests can take another day or a few if it's something complex

That doesn't sound like a unit test. Are you sure it's not something higher-level?

... isn't it better than testers sitting around doing nothing?

Perhaps the testers can work on automating integration tests (and perhaps the unit tests that take you more than a day are actually integration tests they could take over). Maybe they can work on scripting some of their integration tests to run in your CI setup.

Testers are often sat around waiting for us to deploy code which is pretty bad

Why don't you give them the spec, so they can figure out how to test it while you're figuring out how to write it?

This is extra useful because you have an extra set of eyes looking for ambiguity or inconsistency in the spec (similar to the way you often see problems differently while writing unit tests than when coding with your "optimistic path" glasses on).

1
  • 3
    Some unit tests are non-trivial. For example, if you're implementing a third-party specification (eg. EXIF metadata), it's not enough to confirm that you can round-trip the data to and from the file. You need to verify that the data written is correct, which means finding or writing an additional implementation of the spec, which takes time.
    – Mark
    Aug 28 at 1:13
7

You should hand a new version to testers when all stupid little bugs that you should have fixed are fixed. If you are so good at writing code that your unit tests never find any problems, then hand it over before you write unit tests. But if your unit tests tend to find bugs, its much more efficient to pass on new versions after writing tests and fixing problems.

Basically, pass your work on when you are reasonably confident that QA wont find any problems.

PS. There’s an exception when you fix a blocking issue. If your application requires login, and the login functionality is broken, and QA can’t test anything because of this, then you fix it ASAP without caring about any bugs in the login code.

2

The world today is inching closer to "developers write all tests". With respect to functional testing; the check in should not allow developers to check in code without the proper level of tests included. Angular is a great example in that it includes testing stubs for all new projects.

The idea is that the developer code also contains a full suite of new and regression tests which provide the assurance of product quality. If the QA team is truly embedded with the development team then they are working together on the same sprint using the same language. Following this pattern works for both front and back end developers, but no lag in sprint is needed for this, as the tests are a requirement for each sprint.

E2E Testing is a different story, as it tests the interfaces between components. It is much larger in scope and requires it's own sprint planning and deliveries. This could be a QA Team only responsibility; as often, the tools are different and developers are too busy holding down their own responsibilities.

Anyway we look at this, manual testing is still relevant, but in diminishing fashion as today's tools cover a lot of what manual testing did in the past. It's my opinion that everyone working in software must be a programmer.

2
  • Automated vs manual testing is orthogonal from developer vs QA testing (both developers and QA testers may write automated tests). However developers cannot perform QA or User testing for the same reason that students cannot mark their own (or each others') homework assignments. If fully developer-tested code (with full automated unit and integration test coverage) is pushed straight into production; the "QA Test" capability essentially ends up on users and a lot of things will be missed because developers will never have the same perspective on testing as a QA tester. Aug 29 at 12:37
  • Agreed. My only point is the now popular, "developers must write functional test for every check-in" is the right prescription for focusing on quality first. As to all other types of testing all are legitimate in their own concern. However; I do believe all QA team members must know how to program. Aug 29 at 18:33
1

There is a reason that the industry is moving to wiring the tests first, before you write the code, and it's pretty simple - if you write the tests first you end up structuring the code to make it easy to test.

It's very easy to write code that is difficult to test. And generally speaking, if you're not thinking about the tests when you are writing the code you will end up with code that isn't easy to test.

Write tests first.

2
  • 9
    The industry isn't moving to this. It's moving to extensive test suites, but I have yet to see any company write tests firsts. TDD gets a lot of talk, but almost no implementation. And that's because it doesn't actually work well in the real world. Aug 28 at 15:56
  • 2
    Real TDD isn't "write all the tests first, before starting the implementation". It's "write tests as you go, just one test ahead of the implementation". Once you're used to it, any other style feels dangerous; it's a similar feeling to driving without a seat-belt, even for those of us old enough to remember the days before they were universal. Aug 30 at 10:40
0

Other answers have explained why unit tests should be considered an essential part of your definition of done, but I want to comment on one alternative that may help with giving your tests more time to test: using shorter iterations.

It's possible that your testers need more time because their time isn't being used efficiently. With three-week sprints, I'd imagine that they don't have a whole lot to test for the first couple of weeks (or possibly the middle, if they spend the first week testing the previous sprint). Then at the end, all of the tests for the sprint fall into their lap at once.

If you can break the work into smaller iterations (even if some tasks have to be continued across iterations without deliverables), you may be able to hand tests off to your testers at a more consistent pace.

Not the answer you're looking for? Browse other questions tagged or ask your own question.