Are bargaining and beat down attempts on Scrum estimations legitimate parts of the process?

Question

I noticed in scrum meetings, that developers often give realistic estimations on stories. However, even rather simple stories need a lot efforts for configuration, setting up third party components, testing and final build, and the system has accumulated quite some technical debt, so the estimates often appear too high for the product owner or management.

The PO frequently tries to beat down such estimates, like: "What, you want 13 story points [4 days] for this story, this can't be! I can't explain this to management, somebody should be able to code this with 3 SP [in 4 hours]!". As a result, the developers get their arms twisted to commit to a 5 or 8 story points [1.5 to 2 days] estimate (Scrum estimates are still taken as commitments, not just forecasts).

Of course, without any plan to strip down expectations (mainly on testing and quality), these sprints frequently fail. The estimate of the developers is a honest, realistic one, and beating down the estimate does not beat down the actual work to be done.

One can say: "You should not make an impossible commitment, just because somebody pushes you to do!" But in my opinion, the job of a developer is software design and coding, not bargaining or standing up against pressure! There may be jacks of all trades, typically those dealing directly with external customers, but this is not the majority of office developers!

To me, this practice just makes programmers look like jerks, causes constant sprint failures and prevents realistic estimations, as well as looking for actual improvements.

What do the Scrum guidelines say on this topic, or do they say anything on it?

EDIT: replaced times by story points. I was referring to the initial estimation phase with Planning Poker and story points, not the task detail planning. I just put the days/hours there, because it was a typical dialogue like this sometimes, also with time instead of points. Sorry for any confusion! The story point examples represent longer time periods than the time examples.

EDIT 2 There is currently no dedicated scrum master, and the PO takes that role, when it comes to estimation meetings. So it's probably the role conflict making this inappropriate bargaining worse, since he appears as an authority, instead of a neutral or developer scrum master. Perhaps, this can be fixed by taking him as a biased participant instead of a "master", as long as none is available.

Answer 1

The situation you describe is toxic. This sort of bargaining ignores reality and the expertise of the team, it willfully conceals information from the team and organization at large, and it inhibits the team from improving over time.

If you want to city http://www.scrumguides.org/scrum-guide.html as an authority I would highlight:

Only the Development Team can assess what it can accomplish over the upcoming Sprint.

and

Scrum relies on transparency. Decisions to optimize value and control risk are made based on the perceived state of the artifacts. To the extent that transparency is complete, these decisions have a sound basis. To the extent that the artifacts are incompletely transparent, these decisions can be flawed, value may diminish and risk may increase.

Your product owner might desire that a task be possible in only 4 hours. That might even be a reasonable goal. A healthy reaction might be to accept the team's estimate, measure it if is accurate, and ask "what would we need to change in order to be able to complete this sort of task more quickly?" Negotiating the scope of the work involved in the task might also be reasonable and highlight an important difference in how developers and the product owner understand that work at hand. Demanding that the team alter their estimates without new information only ensures inaccurate estimates.

There are many strategies the development team might use to try to change this pattern but when you give an example response of "I can't explain this to management" I get very nervous. It sounds like your product owner does not have the trust of management (the inaccurate estimates they are producing probably do not help) and may not be willing to be transparent about current process failings.

Answer 2

In my opinion, one of the greatest achievements of SCRUM is the development of story points, with the expressed explicit intent of avoiding the bargaining issues mentioned here. The whole point of story-points is to avoid a direct connection between a task and how many days it will take. Instead, those two concepts are connected by the idea of velocity.

If I were a developer, and I was being arm twisted into calling a 13 point story a 8 point story, I would happily oblige. Its their estimation capabilities they are impacting. I will simply scale all of my difficulties to match. Eventually the team's velocity will decrease in terms of story points per week to match management's willingness to "dole out" story points. The numbers delivered to management should match: "We've successfully reduced the number of story points of work needed to accomplish the milestone by 50%. However, we have seen a corresponding decrease in velocity by 50%, so the net effect is we are going to accomplish exactly what we were going to accomplish in exactly the amount of time it was going to take." The concept of velocity exists to combat the wishful thinking of upper management.

Now there's two extremes which I think are worth looking at. One is a complete failure of management and the other is actually a very valid concern for management to care about. The first case is a death sentence for SCRUM, when developers are told "you aren't being productive enough. You need to produce more story-points worth of work." If that happens, you are not actually using SCRUM, you're a Cookoo, who kicked SCRUM out of the nest and is trying to be fed by the mother bird who comes back to the next.

Now there is one case where I think management should be able to engage in arm twisting like this. It should be reasonable for the customer to say "I can't afford 50 story-points to complete this task if your velocity is 20 points/week. I need you to accomplish it in 30 story-points," if that customer is willing to renegotiate the content of those stories to decrease their scope by 40%. SCRUM is not a free meal-ticket for developers to do whatever they want, its a communication tool to help them and management openly converse about what needs to be done. It's also valid for a customer to put the squeeze on the developer and say "do the task in 30 points" if they are willing to accept the inherent risk that the work simply wont be completed in time. When the deadline is missed, if the developers can say "we gave you our best estimate which said the work could not be done in time, and you chose to continue on the current path without adjusting," then its on management to explain why the deadline was missed. I have seen this approach used to lean out a development process: the managers ask for more than can be done, push on the team to be as productive as possible, and then accept whatever actually got done. I think there's better ways to measure a team's productivity, but I have to admit that it's a process that works.

Answer 3

For one, the PO shouldn't be giving task information to management. The task estimations are strictly for the benefit of the team. The only thing anyone outside of the team needs to know is which stories they are working on, what their point values are, and what the average velocity of the team is. T

Second, unless the PO is a senior level developer with a deep knowledge of the software, their opinion about the task estimations shouldn't carry much weight. The developers are the ones with the knowledge of the system and the ones that are doing the work. Unless they are all fresh out of school with zero estimation skills, their estimations should be excepted.

That's not to say estimates can't sometimes be questioned. If so, this needs to happen in a positive manner. For example "have you considered that we've already done half the work for "x"", or "did you remember to include time to do Y?".

Bottom line: the developers should feel safe to make any estimates they want, as long as they make a good faith attempt to be accurate. If they say something takes four hours, you must accept that.

What do the Scrum guidelines say on this topic, or do they say anything on it?

They don't say anything at all. Task estimation isn't part of scrum. With scrum, estimation stops at story points. The task estimation is merely a tool to help teams do better at estimating story points by making sure they've thought through all of the work necessary to complete a story.

Answer 4

It is the task of the Scum Master to ensure the scrum process is being followed. This includes making sure that the team does not (consistently) overcommit on the amount of work that they can get done in a sprint.
On the one hand, that means reigning an overly enthusiastic team, but on the other hand also pushing back to the Product Owner if he is putting pressure on the team.

One good way to keep the commitment/forecast realistic is to look at the stories that got actually realized in the last few sprints. That is what the team has proven they can accomplish, so there is no point is pulling significantly more work into a sprint, as it will not get completed.

---

As other answers also indicate, bargaining on the estimates given by the team is not part of the Scrum process. The team is the most familiar with the software, so they should know best how much work something is.

What can be bargained about is the scope of a story. If the Product Owner thinks a story is estimated as way larger or smaller than they reasonably expected, then they can ask for a clarification from the team to see if the view of the work that needs to be done matches between the parties.
If the story is really that much work, the Product Owner can decide to split it up in several smaller stories and distributing the functionality over those smaller stories.

The team is responsible for delivering quality and that should never open for bargaining.

Answer 5

Yes and no.

Yes, the sprint team should negotiate with the product owner about what gets into the sprint.

No, the product owner gets no say in how long the things will take. You are the experts, not them.

Look, you shouldn't have to deal with that sort of garbage - your manager or team lead are there to set you up for success. That means dealing with the PM (or their boss) so that you will be successful. That said, it's not that hard.

"No."

What are they going to do? Write the code themselves?

Answer 6

Allowing this behavior is a failure of the Scrum Master. Her job, first and foremost, is to protect the team. The PO, for reasons described above, is being short-sighted. The Scrum Master should step in and, in a positive way, reframe the context of the discussion.

Such pressure will, of course, lead to a downward pressure on velocity, something that the OP already cited. If teams are consistently not finishing their sprints, the Scrum Master should again step in and ask why this might be the case. In truly toxic environments, team members might not feel free to be honest in the Retrospectives. But in that situation, the Scrum Master has a responsibility to call out bad behavior and protect the team.

If you find yourself in a situation in which your Scrum Master cannot or will not do these things, then you probably need a different Scrum Master. The PO is responding to typical pressures. The team, in caving, is also responding as humans often do. It's the Scrum Master's job to see this for what it is and put a stop to it. The OP's responsibility here is to bring this up in the Retrospective and, if necessary, to leads and managers. If that still doesn't resolve it, update your LinkedIn profile and find better people to work with.

Answer 7

A product development team (that is everyone from product owner, to developers, to testers) can follow the agile process, and get good results given reasonably competent people and realistic expectations.

Or they can follow a process that superficially resembles the agile process.

That PO probably thinks he will get better results by trying to get lower time estimates for tasks. Of course it doesn't work. If something takes three days, you give me lots of cash and I'll give an estimate that I can do it in an hour. It will still take three days. If you come and shout at me because it's not finished in an hour as promised it will take five days.

All he achieves is breaking the agile process, and giving up on all the positive results that he could get. The scrum master should have the experience, the power, and the courage to prevent this. If management makes someone scrum master who doesn't have the experience, or doesn't give the scrum master the power to prevent this, it's their fault that agile breaks. If the scrum master doesn't have the courage, then he shares the blame with the PM.

Answer 8

I'd say it depends a lot on motivation here. The overarching goal of estimation is to get an idea of how much the team can accomplish during any given sprint, which then allows future work to be forecasted based on that velocity. For example, if the team is completing 30 story points each sprint, and and there's about 60 story points ahead of Item X on the backlog, the Product Owner can then reasonably say Item X will be complete in something like 6 weeks (based on a standard two-week sprint).

To make this estimation as accurate as possible, it's not unheard of to have disagreement about how many story points a particular item is. Scrum actually encourages it. That disagreement can at times even be heated. However, the ultimate end goal should be accurately estimating the complexity of the PBI, not artificially deflating the effort so you can try to cram more PBIs into a given velocity.

This is actually how planning poker works, in principle. Everyone lays out their estimation. The Scrum Master, then takes the high and low estimate and asks why the team member thinks it should be that high/low. This gives the team a chance to hear alternate perspectives of the work involved and get a better idea of how complex the task actually is. After discussion, everyone throws their estimates again. This process is rinsed and repeated until there's a general consensus on the complexity.

In other words, it's not wrong to challenge your estimate, given that the challenger has a rationale for why they disagree, aside from they just want it lower. It's your responsibility, then, to convince your team that your estimation is correct, if you feel it still is.