19

When creating time estimates for tickets should the time taken for testers (QAs) be included in a tickets estimate? We have previously always estimated without the testers time but we are talking about always including it. It makes sense for our current sprint, the last before a release, as we need to know the total time tickets will take with one week to go.

I always understood estimation was just for developer time as that tends to be the limiting resource in teams. A colleague is saying that wherever they have worked before tester time has also been included.

To be clear, this is for a process where developers are writing unit, integration and UI tests with good coverage.

Improve this question
4
  • What about time for bugfixes resutlting from issues the tester finds? That's a really tough thing to estimate :).
    – Frank Puffer
    Jan 10, 2017 at 17:08
  • 3
    Is testing part of your definition of done, or are we talking about a whole other team/department?
    – nvoigt
    Jan 11, 2017 at 7:08
  • 2
    It's perfectly possible for tester effort to be the vast majority of the time spent on a "ticket". So, IMO; yes.
    – Grimm The Opiner
    Jan 11, 2017 at 9:05
  • @nvoigt Testing is part of our definition of done.
    – TTransmit
    Jan 12, 2017 at 19:18

9 Answers 9

Reset to default
34

My recommendation: You either include testing time in the ticket, or add a ticket to represent the testing task itself. Any other approach causes you to underestimate the real work needed.

While developer time is often a bottleneck, in my experience, there are many teams constrained on test. Assuming the limiting resource is one or the other without evidence, can bite you.

As your colleague, I haven't seen a successful organization that doesn't take testing time into account.

Addendum per your clarification: Even if devs write automated tests, particularly unit tests (integration tests do better), they are insufficient to properly test.

If there is QA people involved, their time need to be estimated, one way or another. Only if you are deciding to remove QA people from payroll, then their work time has effectively vanished and you can remove it from the estimation. But this would have side-effects that are easy to ignore. And you may still be missing performance, stress, security and acceptance testing.

Improve this answer
5
  • 6
    The bottleneck location depends on the company. At mine, we have 8 developers feeding one QA resource. QA is obviously the bottleneck
    – Marshall Tigerus
    Jan 10, 2017 at 19:21
  • 2
    I agree that adding a ticket for testing is a good option here. It sounds like OP has no control over QA and it is done by a separate team. If they find something wrong then this could be considered a bug and a new ticket created for the fix / change.
    – My Head Hurts
    Jan 10, 2017 at 20:55
  • @MarshallTigerus: I think it's generally the case that it's easier to co-ordinate the people needed to provide QA for N developers (exact number depends on the product), than it is to co-ordinate N developers. So in some sense QA "shouldn't" be the bottleneck, you "should" hire another QA (and fire a developer to make salary and desk space available, even, but let's hope it doesn't come to that). Of course not everything is always as it should be.
    – Steve Jessop
    Jan 11, 2017 at 1:42
  • 1
    +1 for another ticket, makes it much easier to know where things get stuck.
    – Matthieu M.
    Jan 11, 2017 at 8:08
  • 1
    @SteveJessop Lots of things "Should" happen :)
    – Marshall Tigerus
    Jan 11, 2017 at 18:14
19

Emphatically, Yes

Testing is part of the development process. If your team actually spends time testing the software, the time spent testing needs to be part of the estimate.

Improve this answer
5

If this is agile, I would include the testing effort as part of the total story points. For example, dev effort maybe 1 day and testing 1 day so that would be a 2 point story.

It depends what your definition of done is, but usually its dev complete, business acceptance, and testing sign off in the iteration. If the DoD is just business acceptance then testing effort does not necessary have to be included in the story points, but it still should be tracked.

Improve this answer
2
  • 1
    1 day != 1 point.
    – Rafael Sales
    Feb 21, 2022 at 23:21
  • Excellent point @Rafael. Many organizations try equate 1 point to 1 day. :( I am not saying this is correct, but it does happen. Personally, for points, my school of thought is that points are just an arbitrary unit of measurement to account for complexity. For example a 1 would trivial or small amount of complexity. A 1 point story could just take a few hours or several days to complete. But many want to use points as a tracking or planning mechanism against resources like development and QA.
    – Jon Raynor
    Feb 22, 2022 at 17:54
2

The estimate should account for all of the work necessary to complete the ticket. If testing is a required part of the ticket, then it should be included in the estimate. If testing is a separate ticket, then it shouldn't.

That of course can get all fuzzy once you start using story points, since the difference between a dev-only 5 and a dev-only 8 will be pretty proportional to a dev-and-QA 5 versus a dev-and-QA 8.

I've seen including tester time work. I've seen separate stories work. I've seen separate tasks, each estimated by the group doing them work. Do what makes sense for you, after all the process is there to serve you, not vice versa.

Improve this answer
2

The fact that you can't answer this strongly suggests you don't know why you're writing estimates (or at least that you disagree with your colleague why you're writing estimates). This is a bigger problem than whether the estimates should or shouldn't include testing.

Find out, or reach agreement, why you're writing estimates. If it's to predict what a particular team will achieve in a particular time, then the answer simply depends on whether that team, the one you're estimating for, does the testing or not. If your QA team is separate and has its own scheduling then they might be interested to know how much testing time you (the developers) think is needed from them on a given ticket. They might ignore your numbers and put their own in. Either way they can track that separately from the dev time estimates.

On the other hand, if one team is doing all the dev, testing, and QA, and the purpose of the time estimates is to predict and plan what that team is doing in a particular time frame, then of course the time estimates must include QA, along with any other tasks that it is necessary for that team to do in order to achieve the stated goal. For that matter if you have to have a kick-off meeting for every ticket, or fill in some bit of paperwork at completion, then the time for the admin needs to be in there somewhere. You can't just ignore it.

If it's all one team but with separated roles "developers" and "testers", then that might mean you have a lot of tickets that only one side of the divide is capable of working on, and your (perhaps entirely hypothetical) Gantt chart looks exactly like the chart for two separate teams would look. This fact will upset some methodologies more than others, and you might be better off splitting the planning because of that, but if you don't split it then you have to ticket and estimate everything the team needs to do or your predictions will be hopeless.

If the purpose of the estimates is something other than prediction and planning, for example "because we mindlessly follow an empty ritual that includes them", or "because management uses them as a stick to beat us with to get overtime out of us", or "because we have to make a fixed-price bid and the numbers go into an enormous formula" (thanks John Wu), then it might be harder to figure out what they should include ;-)

Improve this answer
1

Always estimate all the work that needs to be done to make a user-story/feature/ticket really done. We call this DoneDone.

We're done when we're production-ready.

This includes any manual and exploratory testing, but even user-acceptance testing.

An Agile team should be able to release a new part of finished work at any moment. As:

Working software is the primary measure of progress.

How do you know if it works, if you haven't tested it? Now you write that development time is the bottleneck of your time. As a QA-engineer I think that most teams have their bottleneck in testing capacity or they are just taking short-cuts.

To make a long story short, also estimate the testing effort. Keep in mind this could influence your velocity. If you have been doing relative estimates in story-points the testing might already be incorporated into your average velocity.

Improve this answer
1

Here is something very important: All estimates should be accompanied by assumptions and exclusions.

This includes specifying what is included: development only, design and development, dev and unit testing, coverage for acceptance testing, buildout of infrastructure, etc.

If you are providing an estimation document to a project manager, they are going to convert that document into a work order or statement of work for a client or (if an internal project) for the PMO. They may have set formulas for adding overhead (for example, some projects may add X% to cover QA, then add Y% to cover governance and project management) that are set by contract or set by experience. And you don't want to double count. On the other hand, they might not add those automatically.

Practices differ. Whoever is using these numbers will need to know what the numbers mean, and you should be explicit about whether you are including test time or not.

Improve this answer
1

The time should be included in the estimate but you should not estimate the testers time, instead the testers should estimate their time.

Not including the testing time is a false estimate of the total time it will take, but developer time and tester time are not interchangeable (not least because you presumably work in parallel, with the testers an iteration behind) so you should estimate them separately. Moreover, you are not in a position to estimate the time testers will need to test any changes, only the test crew themselves should do that.

Improve this answer
1
  • 1
    Given that that it is the you that fills in the ticket, and that the test time should be included, then the dev should include a 'guestimate' for testing, for later refinement. It's all to easy to create a catch 22 estimation black hole with certain rules... These holes happen in many form-filling tasks.
    – Philip Oakley
    Jan 11, 2017 at 11:57
0

Encapsulation

I'm going to approach this from a software development point of view and language.

  • You are a small cog in a large machine.
  • From the outside of your team, your ticketing system acts as an interface/API to your team
  • Business users who use the ticketing system aren't developers

From good software design, you should simplify and encapsulate as much as possible.

So to look at the process from the point of view of the Business Users, they really only care about 2 things.

  1. How much will it cost?
  2. Are we done yet?

Allowing the Business User to know about your team's internal process is bad management; akin to giving public access to internal state.

Failure to protect your team's internal state, is inviting other teams to manage your resources and mess with your internal state.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.