Our version of Agile isn't working. Tips?

Question

I work on a small team of 4 developers. We are implementing a version of Agile that seems to continuously provide us with the same difficulties, week after week, and I'm looking for suggestions that can help us improve our process.

The background:

We generally do 2 week sprints, and each sprint we tend to underestimate our work, and we get in trouble with our manager because we're behind schedule.

We start off each sprint by tasking out the stories that our manager creates for us. Sometimes he throws in the tasks too and we estimate them. We don't use story points. We use the software Urban Turtle to "manage our sprints", which is essentially just stories and tasks, and the associated burn down. We don't plan for a release at the end of a sprint.

The most common issue that occurs is that we plan for a task in the beginning of a sprint only to discover it's much larger in scope, but still high in the priority, so we need to work additional hours on it. The second most common issue is that one of us runs into a technical problem that slows down the hours burned, causing a roadblock.

The only suggestion provided to us is to be more proactive in adjusting our estimates and providing updates during standups in the morning so we can adjust for the extra time needed.

However, there seems to be something fundamentally wrong with the way we're doing things. Perhaps there's a disconnect between the expectations of the manager on a project level and the expectations on a sprint level. Because we're making these sprint iterations according to a project plan, and therefore extending a sprint or deferring items screws up the project plan. So as developers we are being encouraged to perform Agile by extending estimates when necessary but also complete the sprint on time, which is confusing.

This can't be an uncommon problem, so I'm hoping those wiser than I out there have a suggestion or two on how we can stop running into this same problem every sprint. It's frustrating.

Answer 1

because we're behind schedule

This type of thinking is your problem. You're not behind the schedule, the schedule is too tight. You should start estimating stories in abstract points as opposed to hours and then over the course of 2-3 iterations figure out your velocity. Your velocity is how many points you usually do every iteration, not how many points your manager wants to fit in.

After this it doesn't matter if you consistently underestimate tasks - your velocity already accounts for that.

Obviously, this is impossible if you use hours instead of points.

Answer 2

It seems like the problems are your team's inability to make accurate estimates, and inability to forsee the problems that inevitability arise.

Small tasks are much easier to estimate accurately than large tasks, so try and break your tasks into much smaller chunks.

Don't allow anyone to make an estimate for any task, unless they know EXACTLY how they're going to do it. For any task that the developer doesn't know what to complete, put some time into THIS sprint's schedule for the developer to do some investigation and design, and come up with an accurate estimate. Never less than half a day. But move the task off to NEXT sprint. Then, when you come to the planning for next sprint, you'll have a good estimate. Note that this extra time isn't wasted, because it's time that the developer would end up spending in any case.

And don't be afraid of going back to the project manager and telling him/her that you'll need more sprints to get through the list of tasks. It's better to do that, than to commit yourself to impossible targets.

Answer 3

Are you trying to allocate 100% of your time? If so, stop doing that. Begin by adding up all the hours your team has to contribute during the sprint. Do this by assuming each worker will at best put 6 hours a day toward the project. This is considered an "ideal day". Those other two hours? Sucked up by meetings, breaks, administrative tasks, that time in the morning when you're reading email and planning your day, etc. This isn't "hedging your bets" or "sandbagging", it's being realistic.

Second, multiply that 6 hours/day by 80%. Why? Because as humans we suck at predicting how much time a task will take. This accounts for errors in our judgement. Again, not sandbagging, it's being realistic.

Now you have a number representing a realistic number of hours you expect to apply directly toward your tasks. When you are estimating, stop adding stories when the next story would put you over.

Finally, don't let the product owner add tasks. Scrum planning is for the team, the PO is not part of the team doing the work. Of course, in the real world if the PO is more knowledgeable than anyone on the team her input can be very useful. Still, if the team is taking the heat for being behind, the team needs to take ownership of exactly what tasks they will do. Your goal is to be able to meet the acceptance criteria; if a task doesn't lead directly to that, don't do it.

Remember, Scrum isn't about being more productive. It's about being more open and communicative. The work will take whatever it takes to get done. Scrum is there to make it easier to estimate, easier to communicate with stakeholders, and easier for your team to make a commitment.

Answer 4

Acceptance criteria poorly defined at start of sprint?

Initial estimates are often too low because story cards have poor acceptance criteria (if any) when they are estimated. What about shifting to Acceptance-Test-Driven Development (ATDD) aka storytesting to help the team get really clear on what the card is?

Stories too big?

Another reason you find out mid-sprint that your stories take longer than expected may be that they are too large. Have you seen the Agile in a Flash flashcard deck? They have a flashcard called "Shrink XL stories to Fit". It gives strategies to spliting stories like deferring edge cases, side effects, nonfunctional aspects, or error handling to later stories.

Can't estimate because you don't have sufficient information?

@sleske makes a good suggestion about spikes. Try and identify technical unknowns at estimation time. If there are any, see if you can defer the story into a later sprint and instead do a time-boxed investigation (spike) this sprint to try and learn what it would take to be able to estimate. Don't get carried away and solve the original story - the spike is done when you know enough to estimate the story.

Fail faster

And I agree with @Patrick Hughes - consider moving to 1-week sprints so you can see problems faster.

Answer 5

In addition to the good suggestions by @Kevin and @Patrick ...

Agile approaches are not "one size fits all" but this comment got my attention:

We are implementing a version of Agile that seems to continuously provide us with the same difficulties

You are better off starting with a methodology "by the book" (Scrum is dominant these days it seems) - and do EXACTLY what some other successful team did ... Do that for a few sprints... And only then begin to consider changes needed to optimize for local conditions.

Renting an experienced Scrum coach (for a few iterations) may be a real difference maker. There is subtlety in getting agile right.

Answer 6

First I recommend following book scrum-xp-from-the-trenches. Look at page 26 the point about Velocity calculations. The idea is to define a focus factor, and to say that for the next Sprint:

available man-days * focus factor = estimated velocity

estimated velocity is the sum of the estimates of the stories you plan to implement during next Sprint.

And after one Sprint you calculate the lasts Sprint focus factor as:

focus factor = actual velocity / available man-days

where actual velocity is the sum of the estimates of the stories you have implemented during the sprint.

Then you reuse this actual focus factor for the next Sprint, and after a few Sprint you will be able to be more precise with how much you really can achieve during a Sprint...

Answer 7

Until you get your estimations in order shorten your sprints to one week, this way you will recognize the overage faster and be able to react in smaller increments.

Spend more time up front when designing tasks to get some breathing room to recognize side effects that are probably what's causing the scope to bleed all over. It could also be that the tasks themselves are too long for a proper agile estimation, see if tasks can be broken down into shorter steps that will be easier to swallow.

Get everyone comfortable with sending up a red flag as soon as they hit a snag instead of getting stuck on a problem for a few hours. Try and detach ego and blame from this process, that's easier for some than others =)

And like @Kevin brought up, you never really schedule 100% directly to tasks because there is always overhead like meetings and so on that you may not recognize as time eaters.

When everything is cool on the scheduling front then go back to two weeks, you'll magically pick up a little time from the fewer meetings again.