Scrum - how to carry over a partially complete User Story to the next Sprint without skewing the backlog

Question

We're using Scrum and occasionally find that we can't quite finish a User Story in the sprint in which it was planned. In true Scrum style, we ship the software anyway and consider including the User Story in the next sprint during the next Sprint Planning session. Given that the User Story we are carrying over is partially complete, how do we estimate for it correctly in the next Sprint Planning session? We have considered:

a) Adjusting the number of Story Points down to reflect just the work which remains to complete the User Story. Unfortunately this will mess up reporting the Product Backlog.

b) Close the partially-completed User Story and raise a new one to implement the remainder of that feature, which will have fewer Story Points. This will affect our ability to retrospectively see what we didn't complete in that sprint and seems a bit time consuming.

c) Not bother with either a or b and continue to guess during Sprint Planning saying things like "Well that User Story may be X story points, but I know it's 95% finished so I'm sure we can fit it in."

Answer 1

In my current team we do c).

The velocity should account for things the team really finished in the sprint. Something that was not delivered has no value for the customer, so we don't count any points for it, it's all or nothing.

So we shift the whole unfinished story on to the next sprint and all its story points will be added to the next sprint's velocity. Velocities even out over time and we take an average of the few previous sprints as a reference to determine estimate future velocities.

Answer 2

Option A is a commonly recommended course of action. You do not award points for the completion of the story for the previous sprint and to move the story back into the product backlog, where it is reprioritized. You compute your velocity (and other metrics) based on the completed user stories and value-added. When you begin planning for the next sprint, you take the highest priority stories based on their value (which might or might not include the unfinished story, if business priorities have changed). When you estimate the story, include the work that has already been done when factoring in the new amount of points for the story.

Of course, an alternative option (and my preference) would be to use the original estimated number of story points, which is something that I've done in the past. This makes the assumption that your estimate was good and well-founded, but you pulled down too much work for the sprint (eg - the story was actually worth 3 points, but the problem lies in the fact that you pulled down 15 story points when you should have only pulled down 13). It's a potentially dangerous assumption if you aren't confident in your ability to perform the estimates, but it might work based on your team and process.

However, you mention that this will "mess up the reporting of the Product Backlog". The Product Backlog should be dynamic anyway, with the ordering and estimations of each story changing as more is understood about the technology, the system, the requirements, and the customer. Typically, what gets reported out of the Product Backlog is the number of user stories and the total number of story points. These should be expected to change as priorities change, new requirements are added, invalid requirements are removed, and more information is learned.

Answer 3

Thinking too much about this makes it seem more complicated than it is... It's actually pretty simple:

Option C

Incomplete stories go back into the product backlog, without the points being changed. When planning the next sprint and what can get done, the discussion should include the fact that much of the work is already accomplished. If the team decides they can fit it into the sprint, then it goes in, with its original number of points. If not, it stays in the backlog. Done. The points get awarded to the sprint in which the story was completed.

If it helps, you could track a separate "work remaining" metric per user story, so that if, by the end of the sprint, the story isn't complete, the estimated work remaining can be noted on the story and factored in when planning its inclusion in a subsequent sprint. But even then, don't alter the original story's point value.

Answer 4

Given that the User Story we are carrying over is partially complete, how do we estimate for it correctly in the next Sprint Planning session?

I don't think options A through C are good, mainly because what (I think) should be most important regarding a team's velocity is it's average velocity and not whether the velocity of any given sprint went up or down.

When a user story is defined, it should have acceptance criteria. If anything in the acceptance criteria is not done, then the team simply does not earn any of the points. If the story is done-done (i.e. coded, tested, and accepted by the P.O.) then the team gets all the points.

This works well when the team is focused on its average velocity rather than on a given sprint's velocity.

Like M. Cohn in his book, I tend to have a preference for an all-or-nothing scenario. After all, trying to estimate whether you've completed 5 points out of an 8 point story, or maybe just 6 or 7 is just going to end up being another guessing game... and don't forget that you already got the initial estimate way off. It's probably better to just go with the simplest method and when get all the points after it has really been completed.

Quoting M. Cohn from his book¹ (my emphasis):

I’m generally in favor of an all-or-nothing stance toward counting velocity: if a story is done (coded, tested, and accepted by the product owner), the team earns all the points, but if anything on the story isn’t done, they earn nothing. At the end of an iteration, this is the easiest case to assess: If everything is done, they get all the points; if anything is missing, they get no points. If the team is likely to take on the remaining portion of the story in the next iteration, this works well. Their velocity in the first iteration is a bit lower than expected because they got no credit for partially completing a story. In the second iteration, however, their velocity will be higher than expected because they’ll get all of the points, even though some work had been completed prior to the start of the iteration. This works well as long as everyone remembers that we’re mostly interested in the team’s average velocity over time, not in whether velocity jumped up or down in a given iteration.

¹ Agile Estimation and Planning, Re-Estimating Partially Completed Stories, p.66

My team had previously attempted to assign partial points, despite some objections, and I don't think it worked well at all. (We don't do it anymore...go figure) This is especially the case because stories are supposed to get estimated as a team, yet if only one person is working on it, it'll be more difficult for the team to know how much an individual has actually completed. Agile is more interested in a team's average velocity rather than on how "nice" a particular sprint looks.

That being said, the author does mention that assigning partial points can be considered if the team is unlikely to tackle the remaining work in the next iteration. In this case, the team would estimate the work that remains and break it down into new user stories with whatever size they feel they should have. As the author mentions²:

The combined estimates would not need to equal the original estimate...

² Ditto, p.66

The better recommendation for the team is to break stories down to a sufficiently small size to avoid this kind of problem³:

However, the two best solutions to allocating points for incomplete stories are not to have any incomplete stories and to use sufficiently small stories that partial credit isn’t an issue.

³ Ditto, p.67

Hope this helps.

Answer 5

If your reporting tool cannot handle option A accurately then I would go with option B unless you have no hope of ever using your metrics.

In some cases you can do option B AND not skew what closed means by narrowing the scope of the old task you are about to close and only creating a new task for the scope that remains. This makes the history make more sense semantically and usually indicates places where you should have considered breaking the task down further. Sometimes this is not possible or logical and you simply have a task that is that large or runs into that level of issues.

edit: This assumes (as I believe the OP was assuming) that nearly complete work has not been knocked down in priority and that getting to the payoff on the previous effort keeps it high enough in the list to continue working. I know some shops do not do this, but most places I have worked consider finishing a near-complete leftover task to be valuable enough to simple continue unless something has changed dramatically. The penalty of changing context is often not worth changing the order.

Answer 6

I'm surprised there doesn't seem to be a straight answer to this. I was expecting a chorus of "option D, dummy"!

As we haven't seemed to be missing anything obvious, we figured this is one of those decisions each team needs to make for themselves based on how they want to work and what metrics are most important to them.

We decided that keeping accurate records of the user stories we have implemented is essential, as these form the basis for our testing, support documentation and release notes - that rules out option B.

Next we figured that being able to perform sprint planning accurately was essential - that rules out option C.

We have therefore concluded that option A is right for us. We're going to re-estimate the story points for any stories we partially complete in a sprint in order that we can plan for them properly in subsequent sprints. Over time the product backlog will slightly under report the amount of story points we have implemented, but this is going to be less of a problem than any of the other options and could possibly be resolved by changing our record keeping and reporting.

Answer 7

We track the time spent in the sprint iteration for capitalization purposes, (hours burnt on tasks related to a user story), and move the pointed user story along if the goal of the PO is to keep trucking on it during the next sprint, leaving points the same.

If the PO's goal is to move something else up in its place, then we'd simply put the unfinished story into the backlog and do whatever they wanted. Leaving the original estimate of points is my recommendation, because if you had a habit of biting off more than you could chew, every sprint, you'd be "completing" story points in a sprint that weren't fully-done and clean-tested items.

If you did want to leave something in the sprint, pointed, or you had to ship, I'd think you'd determine a slice point within the original story, to which you did reach the done-point, and commit that smaller piece for your iteration. Then the remainder could be re-pointed, and dropped into the backlog. That would be an opportunity to re-estimate because you agreed with your team to break the story into slices.

Answer 8

The first question to ask is What does a Story Point mean? If you define a story point as "How much work engineering gets done", then any answer will do. However if you define a story point as "How much value is delivered to the business", then it becomes important to not give credit until a story is 100% completed, accepted and delivered 100%. Modifying story points after a sprint based on what was completed will only hide the real problem: a) there wasn't a clear understanding of the story, b) the story was too coarsely defined, c) the story lacked measurable acceptance criteria, d) not a deep enough understanding of the tasks or dependencies to complete the story, e) understated the effort to test the story, f) pulled down too many story points, or g) ...insert your reason here...

The goal of agile is to be flexible while being predictable. The best way to be consistent, in my opinion, is to figure out what went wrong without modifying original story estimates.