4

Imagine we have a Sprint Backlog defined, consisting of business features to code. When a feature is developed and delivered (deployed to a test server), QA can start manual testing (we don't have automated tests for a reason). Testing takes several days (we cannot decrease the time needed).

Is it ok for the development team to complete development right at the end of the Sprint, leaving the QA team with no time to test all features by the end of the Sprint?

If it is, then QA team can plan test features in next sprint. Additionally, I have concern that new planned dev features depend on QA test results

5
  • 2
    A sprint end is not a release date. QA can work in their own pace on actual release candidates. There is no issue here. Commented Jul 18, 2021 at 19:42
  • its an issue if you are trying to follow the scrum rule of "deliverable features" at the end of a sprint and dont allow yourself some wiggle room
    – Ewan
    Commented Jul 18, 2021 at 22:29
  • @Ewan I don't think "deliverable" in a scrum sense ever meant stable, tested, commercial grade, ready to ship piece of software. It is an iteration, it is supposed to build and run and be demo-able. Commented Jul 19, 2021 at 6:05
  • scrum has always had the idea of a deployment sprint, but I think the expectation is that the features have been tested to some extent
    – Ewan
    Commented Jul 19, 2021 at 8:02
  • 2
    I suspect Scrum would say why the heck do you have a separate QA team? Commented Jul 19, 2021 at 11:04

6 Answers 6

0

No, but it means you can't release after every sprint.

Everyone will tell you that the answer to this is automated testing and that will reduce your testing time to zero. But lets just go with 1 week of manual testing needed before a release and that week will throw up 2 or 3 bugs which will need to be fixed. And that you do 1 week dev sprints which produce 2 or 3 features.

What you need to do is have a much longer release cycle than your sprint cycle. For week long sprints I would suggest monthly releases and expect to miss or roll back every other one. For an actual release cadence of every 2 months.

When you finish a sprint you put those features into a release branch and start testing. The Developers immediately get started on the next set of features on the dev branch.

When A bug is found in the release branch, that moves to the top of the sprint backlog and is worked on as a priority, fixed on the release branch and merged into dev.

After, lets say 2 weeks testing and fixing you have a tested and releasable version of the software on your release branch and two dev sprints minus bug fixing time of feature work in the dev branch.

Release the release branch, move the next set of completed features from the dev branch to a new release branch and continue the cycle.

This way all employees have a steady supply of work. Beware of pushing to meet the deadline though.

When you put on pressure to have bugs fixed quickly, you don't really slow down the feature work as much as cause short cuts to be taken and hence more bugs in the next release.

If you have manual testing, bugs are guaranteed. Missing deadlines is guaranteed, rolling back releases is guaranteed.

If you find you are developing features faster than you can test and finding too many bugs in testing, then don't pause development, don't move everyone onto bug fixing. Work harder on the specifications of those features. better specs = less bugs found in testing and faster development

11
  • Thanks for comprehensive answer. So, bottom line is, dev team should not stop doing(and delivering) features throughout the sprint. If QA cannot catch up then QA should continue testing current features in next sprint. Am I right ?
    – voipp
    Commented Jul 18, 2021 at 18:19
  • yeah. Much as i would love it, no-one is going to pay developers not to work
    – Ewan
    Commented Jul 18, 2021 at 18:51
  • Also, I wouldnt say "catch up" you can regression test a dozen features in the same time as you can regression test 1 naturally the test cycle will be slower than the dev one
    – Ewan
    Commented Jul 18, 2021 at 19:00
  • Btw, Imagine Dev team developed feature F1, handed it off to QA team. Next feature to do is F2 and it coupled with F1. So, QA finds bug in F1 and dev team almost finished F2. Now you need to fix F1 and also make additional changes to F2, it seems ineffective to start F2 before F1 is tested
    – voipp
    Commented Jul 18, 2021 at 20:11
  • 3
    For an actual release cadence of every 2 months. I'm not sure that I would consider this Agile. In 2001, when the Manifesto was written, the delivery cadence was "from a couple of weeks to a couple of months, with a preference to the shorter timescale". In 2021, with all of the advancements over the last two decades, it's possible for large-scale systems to be deployed to production multiple times a day. A 2-month cadence may be current state, but I would not consider the organization to Agile unless they were working to get that down significantly.
    – Thomas Owens
    Commented Jul 18, 2021 at 20:50
2

The fact that you have a hand-off between the people who develop the functionality and the people who test the functionality implies the existence of a sequential set of steps in the iteration, either for the system as a whole or on a workunit-by-workunit level. Most of the agile methods aren't built to support this kind of approach.

Different methodologies and frameworks have a slightly different way of looking at this, however the idea of a cross-functional, highly collaborative team that gets work done. In methodologies that have timeboxed iterations (Scrum is a good example of this), the expectation is that a team regularly gets work done within an iteration timebox. Done means designed, implemented, integrated, and tested so that way the integrated product is ready for release and use.

As an outsider, I would challenge your reliance on manual testing and the ability to reduce the time. Although there may be certain types of testing that are best performed manually, there's plenty of opportunities to automate testing. Perhaps you don't have the right tools or infrastructure. Perhaps the design of the system doesn't lend itself well to automation. These are problems that can be solved, though, with appropriate investments.

Even with opportunities for improvement, those will take time and money. They don't address what to do now. This is really a choice for the organization to make, based on the tradeoffs.

One option is to stop development with enough time to test, fix any issues, and retest the fixes. If testing takes "several days", then you will need at least more than "several days" to triage, fix, and retest. Without any better numbers, I'll assume that it will take 5 days (1 standard work week) of testing. If your timebox is 2 weeks, you have one week of development and one week of testing/fixing. In the end, you have a deliverable product that can be demonstrated to and inspected by the stakeholders as part of a process to determine the next steps.

Another option would be to continue development right up to the end of the timebox and take the first several days of the next timebox for triaging and fixing issues. This makes it extremely difficult to plan your iteration with any meaningful level of confidence since it depends greatly on the number and type of issues found. You may opt to leave buffer space in and sometimes greatly underpromise what you can deliver. You may also opt to not leave in buffer space and miss your objectives. Either way, your stakeholders probably aren't gaining confidence and trust in the team. In addition, if your new development is depending on things that haven't been tested yet, you're risking building on a defective design or implementation and introducing a lot of rework into your process.

My preference would be to always have a done increment at the end of the timebox, where done includes tested. This lets you get valuable feedback from stakeholders and build future iterations on a more stable, reliable base. However, you should still work to reduce the amount of manual testing required and move testing throughout the iteration rather than something that happens at the end.

10
  • 1
    The problem with your answer in my view is that with both your options you have a team of developers sitting idle while the manual testing takes place. Its the same as my answer, but with developers who get every other week off
    – Ewan
    Commented Jul 18, 2021 at 22:13
  • 1
    @Ewan Why do you think that developers would ever be sitting idle? In Option 1, the developers should be involved in testing, triaging, fixing, and retesting issues. Ideally, they'd also be paying down the technical debt that is requiring extensive manual testing in the first place. In Option 2, the developers are handing off at a Sprint boundary and starting the next Sprint - no downtime there.
    – Thomas Owens
    Commented Jul 18, 2021 at 22:20
  • 3
    @Ewan I would hope that any person developing a system could follow a pre-written test case for that system. You're right that finding a bug late in the test cycle would cause work to roll over, but the same can be said for any other method. Which is why if the organization really wants to be agile, they'd fix these problems.
    – Thomas Owens
    Commented Jul 18, 2021 at 22:50
  • 1
    @ThomasOwens I can assure you that our QA people are a lot better at QA than I am. And I can assure you that I'm a lot better at development than they are. If the organisation wants me to achieve half of would I could achieve, that's up to them, but they'll soon have to do it without me.
    – gnasher729
    Commented Jul 19, 2021 at 12:30
  • 1
    If there's a pile of manual testing to be done by the team, then our QA guys do it. I can much better support my team's objectives by enhancing my skills inside my specialty.
    – gnasher729
    Commented Jul 19, 2021 at 17:41
0

People do Scrum or other agile methods in different ways at different companies and organizations. This is how we work at the client that I'm current working for:

We have a backlog (in Jira) of features to implement and bugs to fix. Before we start a sprint, the dev team, together with the client (who has the role of product owner) plan which Jira issues we'll work on in the sprint.

For each issue, we create a feature branch. When a developer has finished working on a feature or fixing a bug, that developer creates a pull request, and asks two other developers to review their changes.

Before someone creates a pull request, that person must have done more than just make their code compile - the author is expected to have written unit tests and test their new feature themselves first before creating a pull request.

The reviewers inspect the code and also test the new feature. If the reviewers approve, then the feature is merged into the development branch.

At the end of the sprint, we merge the development branch to the master branch and we build a new release of the software from the master branch.

Here is where your question comes in: When a feature isn't ready at the end of the sprint, we simply don't merge it to the development branch, the feature branch remains and the issue is carried to the next sprint.

We are working with a development / test / acceptance / production approach. That means that we (developers) work on the development and test environments; QA works on the acceptance environment. Whenever we make a new release, it's installed on the acceptance environment, and that's where QA tests our work.

You could say that the QA teams is always a sprint behind the developers - while were are working on sprint X, they are testing the release of sprint X - 1.

When the QA team approves a release, it's deployed to production.

Don't take this as the only way, or the "right" way to do things. There are other approaches, ofcourse, such as continuous delivery. But I hope this gives you some insight into one way to do software development.

0

So, rather explicitly, if you're doing Scrum as the the tag suggests, you cannot finish the iterations with untested, unrealisable work. The guide tells us we must produce a "done" (tested, usable, releasable) increment at least once per iteration. Also, there isn't a QA team a dev team, there's just a team. So, my suggestions will involve people of different specialisations working together.

But is Scrum correct to demand we work together and prioritize creating releasable software on a regular basis? I think it is. For four reasons.

  1. Untested work isn't done, and not done work is likely to need the attention of the team as they are working on the next iteration. This is disruptive when the not-done code turns out not to be working. It's also kind of a bummer if you enjoy the satisfaction of shipping things.

  2. Working out of sequence with each other inherently reduces cooperation.

  3. One of the reasons we work in iterations is that we can ship things quickly and adjust our plans based on our customer's reaction to that software. This practice delays that feedback.

  4. The idea that "we must always keep developers busy with features" and "our testing process is long and manual" seem to come together quite often. The dirty secret of agile (or tbh software development in general) is that technical excellence is required for things to go well. If an organisation values keeping people busy over more meaningful things, like delivering useful software at quick but sustainable pace, short-term thinking can erode that technical excellence and grind even a busy team to a halt. I have seen this happen personally. There are no special cases, and no system to so complex it cannot be improved step by step.

I suggest you improve your testing process. Change your goal from keeping people busy to releasing frequently. Have the developers and testers working together closely throughout the whole iteration. During any slack-time have developers working to automate and speedup the testing process. I have once again, personal experience of this approach working.

1
  • What is slack time?
    – gnasher729
    Commented Jul 19, 2021 at 17:43
0

Is it ok for the development team to complete development right at the end of the Sprint, leaving the QA team with no time to test all features by the end of the Sprint?

Is it ok to release untested features? When you do this to QA that's what you're attempting to do. Some shops do this regularly and their users do their QA for them. And for them it's fine. It's not that this is ok or not ok. But it's something you shouldn't kid yourself about. You decided to do this.

That's really a business decision. One you shouldn't take away from the boss who hired a QA team thinking you'd use them. You can sit her down and explain why you think QA isn't really needed but that's not your call. If the boss wants QA then work with QA. Not against them. Don't eat their time box.

-1

Graphical, your setup seems to look like this:

Development  |------------|   ------------|
Testing              |----|          |----|
Acceptance                 |               |
Delivery                    |               |

To tell the truth, i would avoid that like hell. The reason is it means to deliver untested code into production. Even with a good set of Unit-Tests i would be very careful to do that.

Instead, if i need that time for manual testing,

To tell the truth, i would avoid that like hell. The reason is it means to deliver untested code into production. Even with a good set of Unit-Tests i would try my best to avoid this.

Instead, if i need that time for manual testing, i would switch to the following setup:

Development  |------------|   ------------|   ------------|
Testing                   |----|          |----|
Acceptance                      |               |
Delivery                         |               |

That way we only deliver code into production which is tested.
Some managers ask me if we could move Testing a bit and at least test and develop partially in paralel. They use the argument, that the last changes will take place only in a small part of the application while the rest stays unchanged.
My Counter-argument is normally, that, if we could foresee the unwanted results of our changes, then we would not need testing at all.
Yes, if have a good architecure with cleanly seperated building blocks, it is not very likely that a change in building block A will result in a problem in building block B. But it is not impossible.

It ends nearly always with the question how risky the development should be. If a problem in production is a killer, then you should avoid all kind of risks.
If your production is treated as a kind of second testing system, then do as you like.

Normaly it is something in between.
As a developer and as an architect, i fight for the possibility to deliver high quality software. That means i would never accept a "deliver untested code" without questioning the necessity.
Because if i allow it once, and it works without problems, then they want it a second time. If it works twice, it gets the "standard approach". And when it explodes, guess who´s fault is it?

4
  • But now where are your timeboxes? You can't say development+testing is a sprint because it overlaps with the next sprint's development. Commented Jul 19, 2021 at 11:04
  • :-) yes i can. In a sprint you also do refinements regarding future sprints. So what does speak against starting the development for the next sprint? The problem here is that you have two different timeframes. The development-frame and the delivery frame and they are not in sync. It makes life miserable, but if there is the restrictions because of manuel testing, you have to life with it. The alternative would be to not develop at all while testing...
    – JanRecker
    Commented Jul 19, 2021 at 11:35
  • Testing is not a refinement and development is not a refinement... Commented Jul 19, 2021 at 14:32
  • Doesn´t really change the point. If the restriction is, that you have to do manual tests, then you must 1. sit around lazy (not developing while testing), or 2. take the risk (by continueing developing while testing) or 3. life with two timeframes which overlap (developing as one and testing and delivering as the second). Not my favored solution but if the context does not allow a different testing approach, then the possibilites are limited.
    – JanRecker
    Commented Jul 20, 2021 at 11:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.