How do you cope with a team who tends to underestimate time needed to complete tasks?

Question

How do you cope with a team who tends to underestimate time needed to complete tasks and haven't been improving the accuracy of their estimates?

Details: I work in a scrum team (7 engineers) in a FANG company, at the end of every sprint, we vote to estimate how many hours we need to spend on each user story for next sprint. Then we assign these stories to each one of us according to our available capacity.

I've been here for a year and we have a very persistent problem: almost everybody can't finish planned work in almost every sprint. We have huge carryovers in every sprint.

I tend to vote for larger estimates, but my teammates almost never learn from their past mistakes and persistently vote low in these estimations.

I'm the kind of person who just want work for 40 hours/week, chill and avoid burnout. I believe in 'underestimate and over-deliver'. I know some of my teammates work long hours all the time. Our scrum master works extra hours almost everyday yet she still vote very low all the time. She's been around for quite a while so we respect her opinions.

They might each have different incentives, like to impress the management or conform with the others? Maybe they want fast promotion? I don't know and I don't care. I try to cope with it by taking a strong lead in my own project and voice my concerns in planning meetings. But sometimes I get assigned to user stories that are estimated by the team. And they usually have ridiculous expectations, like launching a new small production service from scratch in a week. Remember it's a big company which has a lot of internal processes, and a teammate have told me it takes at least 3 weeks to launch a bare born service. I was on vacation when this estimation happened.

Also I would look bad if I have big carryover points too often.

My manager is kind of a people pleaser and tend to accept ridiculous deadlines from other team or upper management. Thankfully my manager listens to me.

Sorry for the long rant. I actually like my team and manager, so I don't want leave. I know we are doing agile all wrong but they don't change, and don't seem to care about working long hours.

Answer 1

You do have sprint retrospectives, right? That is the time to question: "How do we explain we did not burn down much? Again?" There may be excuses that won't make much sense to you. Just take note. Do this after each sprint. A time will come at which no one can argue any longer with the explanation you regard as obvious.

In any planning session you could ask "What is different now, what makes us believe this time we will meet this low estimate? Or do we not really believe it ourselves? Whom do we think we are kidding this time?" You are going to be real popular .

On the practical side, others may just not foresee the things they may or most surely will have to deal with. You may be able to point out these things.

Then we assign these stories to each one of us according to our available capacity

This is not very Scrum-like. Team members typically pick and choose their own issues from the set selected for the sprint.

First and foremost, this is the team's problem, not your personal one. The team comes out unreliable every time. So technically it is not a matter of you dealing with the team, you are part of the team. If it doesn't feel that way, perhaps it's not worth your while.

Answer 2

The 'official' answer for aglie development is to estimate in 'points' rather than hours and adjust your expected progress based on the 'velocity' or points achieved in the last sprint.

However. You don't really need to change much. You know how many 'estimated hours' you achieved in the last sprint vs the actual number of person hours there were.

So you know by how much you underestimate. Simply put less hours worth of tasks in the sprint to compensate.

The points approach is designed to de-emphasise estimation, which is never very accurate under the best of conditions, and just get you to achieve the goal of completing what you put in the next sprint. Rather than worrying about how long each task will actually take.

I would also recommend 1 week sprints to improve estimation.

Answer 3

I would consider this persistent behaviour also a sign of the software being much too hard to maintain with respect to professional standards. The general feeling is, that things should go faster.

Besides all procedural techniques, like discussing difference in opinion on scrum points, there are some possible issues here:

• The different aspects of the problem are not elaborated enough in detail; or peer discussed. Better peer programming, written designs and such might help - if feasible.
• The software system contains technical debts: refactoring is needed. When there is too much parallel inheritance, business data and logic is interspersed with taking care of visual components. Hence changes come hard.
• The software system contains technical debts: the code relates to undocumented business behaviour. Regression errors happen. Then business code must be totally integrated with documentation on business requirements. That documentation must be in readable business language and the code must refer to the documentation and comment the code design.

In general it is hard to reform an existing organisation, without large consensus or organisational power.

But you may add your own measures, like written designs, peer communication, and especially evaluation of past work in technical detail.

Answer 4

I think that before you can figure out what to do about the situation, you need to determine why it's happening. Is the problem really that the programmers are terrible at estimating? (It's certainly plausible. Estimates are hard.) Or is the problem that the tasks are underspecified or poorly specified? Or is there some other problem such as constant interruptions, or tasks not on the schedule that need to be addressed immediately?

Once you know why, then you can think about the what of the problem.

Answer 5

It sounds like part of the problem is that your team thinks that points and stories belong to individuals. I say that because you said:

Then we assign these stories to each one of us according to our available capacity.

and

I would look bad if I have big carryover points too often.

Stories and points belong to the team, not the individual.

In Scrum, you don't have carryover points, the team does. Also, you shouldn't be assigning stories to individuals. At least, you shouldn't be assigning all stories. You decide which stories to work on first, but you only pick stories that you are certain can be completed. Only when those are completed should you pick up the next story in the sprint backlog.

I would suggest that you stop thinking in terms of individual contributors and start thinking in terms of the team as a whole. If a story is started but not completed, it is the fault of the entire team. If an individual is in danger of not finishing a story, the rest of the team needs to pitch in to help.

At its core, scrum is very simple. Do a sprint, and figure out how many story points you completed. That's how many points you pull into the next sprint. You don't need to vote, you pick that number or you pick fewer. It's just that simple. The next sprint, figure out the average of the previous two and adjust accordingly. Eventually your team will settle on a number, and you'll always pull in that many points.

Answer 6

This is exactly why you track Hours Velocity.

Hours Velocity is

 estimated hours of completed tasks in sprint   
-----------------------------------------------  
actual hours spent on completed tasks in sprint  

Oh sure some just add up the story points completed but that is a different metric. Neither are officially part of scrum anyway. Done this way you can focus on comparing estimations sprint to sprint. Otherwise you get distracted by time not spent on tasks, which is not constant.

Hours Velocity is a measure of how well your team estimates work. Programmers are notoriously optimistic. It's not uncommon for a new team to be off by as much as 1/3rd, that is, need 3 times what they estimated. By tracking and reporting this metric the team learns to adjust it's estimations and improve them over time. This leads to fewer features needing to be dropped mid sprint.

BTW, unless your scrum master is also a developer she shouldn't be voting on estimates or points.

As for taking on work other people has estimated... Screw it. As soon as you get assigned work, come up with your own estimate. If it differs, report it now. Don't whine if you grind away without saying anything and then run outta time. If it's going to be longer than a sprint figure out how to break it up into a smaller task now. Always have something presentable done at end of sprint.

I also recommend 1 week sprints. But that's mostly because it ensures you can't spend much time lost in the weeds.