1

We have been thinking about comparing product size/effort at least roughly and this is what some suggested:

  • There are multiple products, each with one scrum team
  • All the scrum teams estimate their stories relative to a common reference story. Therefore, e.g. in Project A, they look at their story and estimate it as requiring twice as much effort than the reference story.
  • In the end, all the projects are estimated with relation to this reference story and are somehow comparable in terms of expected effort - if one project has 200 SP and the other 400 SP, it could be expected that it is roughly twice as much WORK.
  • Individual teams have their own velocities, nobody compares that because productivity is of course different.

An analogy: when digging a hole, I can say that a hole 10 meters deep will be 10x more work than a reference hole (which is 1 meter deep). One team will use an excavator and dig their 10 meters deep hole in 30 minutes. The other team will use a spade and spend 2 hours digging their 1 meter deep hole. But the amount of work done is still the same and can be compared (1 vs 10), regardless of productivity. Sure, SW is far from that simple to estimate but it should not be completely off.

Is there a problem with that? To me it seems fine as the teams only need to compare their work with a common reference story and assign points relative to it (as they would do when estimating using with their own reference story). It is completely fine if team A takes a day to finish a story point while team B takes two days, what matter is that the estimation is consistent.

  • 2
    This question might be more suitable for Project Management. – Bart van Ingen Schenau Oct 31 '18 at 10:27
  • I suppose as long as the reference story is meaningful to everyone then there isn't a problem. After all, story points all relate to the simplest of stories - that being worth 1 point or whatever system you have going. – Robbie Dee Oct 31 '18 at 11:08
  • Do all of the members in each team have roughly the same proficiency and experience? A team of juniors might point a project as 500 whilst seniors might point the same project 100 due to the perceived difficulty, it doesn't mean the project is 5x as costly. – Frayt Oct 31 '18 at 11:27
  • @Frayt While there are differing views of cost, these are normally covered by planning poker or such like. I'd have thought for the reference story it would be taken as read that the estimate is pretty reliable. Perhaps the OP could clarify. – Robbie Dee Oct 31 '18 at 11:34
  • @Frayt But this should not happen with story point estimation - the sizing is relative. Both seniors and juniors can see that a task is roughly twice as difficult/big as the reference story. Both mark it 2SPs but seniors will deliver it much faster. In other words - both projects might be perceived as an amount of reference stories (100 SPs=roughly the size of 100 reference stories). – John V Oct 31 '18 at 11:42
3

I have tried this approach with several teams and it does not lead to cross-team efficiency. We always used a reference story, that everybody understood, as the basis and it had a non-1 value (2 in our case) in order to make sure that for things we knew were even smaller than that reference story we could give those a 1.

The concept falls apart as soon as the team is brought into the estimation room and needs to say what something is that is BIGGER than the reference story. Some teams, for some reason, see something as 4 times the reference story, others estimate at 2 times the reference story. They are both correct, because STORY POINTS ARE NOT HOURS.

As long as the team are being consistent in their sizings, the team estimates are valid and you can predict from the velocity that team achieves. But comparing across the teams does not work. Team A, regularly using larger increments from reference, will always seem to be accomplishing more story points in a sprint. Team B will be evaluated as 'low performing', when in actuality they just estimate on a different scale.

This was made even more clear to me when I took about 3 weeks of vacation, allowing for two estimation rounds to be done in my absence with a team lead that ran other teams that had 'higher' estimates. When I returned, our velocity had suddenly jumped dramatically and all the story points for the team were much higher, but we had not actually accomplished any more work.

(This also pointed out that I had an uneven influence on the size of the estimates done by the team)

In conclusion, using a reference story is very helpful. The organization can understand the estimation process and feel like everybody is standardizing. However, beyond that, I would not trust any alignment of estimate sizes across teams unless you also have the same people doing all the estimates and remove the team from the equation.

  • Yes, as I mentioned, this is not to be used for any cross-team comparison, rather for project size estimation. However - if one team see something is 2 times the reference story, then the other teams should arrive at the same conclusion too. Not how long will it take them, but how much effort they expect it to be compared to the reference story. Actually in this case, they are not both correct, because they use the reference story and thus the relative effort should be the same. That is why story points are said to be used when you cannot agree on time. – John V Nov 1 '18 at 13:07
  • I think this is the textbook example: one team says 1 hour, the other team says 1 day. Instead, they agree on 1 point, however it means different time for both of them. Exacty because story points are not hours. – John V Nov 1 '18 at 13:09
  • That is a good answer but you confused hours and story points. Time estimates vary among team members (or teams), story point estimates should not. In the end, that is exactly why they are used, to allow communication between members with different performance. Members are all correct in their different time estimates but they all need to agree (more or less) on a number of story points. – Ezoela Vacca Nov 1 '18 at 16:57
  • I was not intending to confuse hours and story points. In my illustration, it is points (relative, not hours) which changed depending on the individuals. The baseline can all be agreed to be a 2, but if you hand two different teams the same story for an estimate, one might call it a 5 and the other might call it an 8. Their idea of the relative scale of effort of what amount of effort a number means can be different. That is what I have seen in practice. An 8 for one team means a different thing than an 8 from another team. – Jay S Nov 1 '18 at 20:49
5

This is a bad idea. The entire point of estimating by story points is to have an abstract unit that isn't directly comparable to time or across teams. You don't learn any useful information by trying to have story points sized the same between teams. You need a reasonably accurate velocity and total story points to do any real comparison, if team A estimates a 200 point project it doesn't necessarily mean its smaller than team B estimating a 400 point story, you need velocity as well to get a meaningful comparison. It's possible that its 4 sprints of work for team A and 5 for Team B. The more you try to have things be similar the more comparisons will happen, even informally some people will start making those comparisons. There will always be subtle pressure to not be the lowest velocity team in any multiple team environment, by attempting to keep story points consistent across teams this will make that pressure more pronounced.

  • That is not really true (e.g.SAFE framework also include normalization) - you do not need velocity at all. If Project A is estimated as being as big as 150 reference stories and Project B is estimated as being as big as 250 reference stories, you can compare the volume of expected effort. I agree it has nothing do with productivity and efficiency, it is just abstract volume estimation. Junior teams might deliver 5 SPs per sprint and senior teams 15SPs, that is normal. – John V Oct 31 '18 at 13:06
  • I like what Mike Cohn coaches: "So, story points are about time—the effort involved in doing something. Because our bosses, clients and customers want to know when something will be done, we need to estimate with something based on effort. Risk, uncertainty and complexity are factors that may influence the effort involved." – John V Oct 31 '18 at 13:09
2

The first thing I wonder about this is: what do you hope to gain by doing this?

Are you looking to figure out the relative size of projects? Well, I don't see this helping much (aside from giving you a very rough guess as to whether something is bigger or smaller). Story points are, by their nature, only applicable to the people who estimated them.

Story time to help illustrate that. One project I worked on was a website that was mostly CRUD pages but had one super complicated interface. This project was supposed to be a replacement for an old mainframe app and they wanted their main page to mimic the old system in terms of being able to handle rapid keyboard input. And this page had tons of moving parts where changing one input might enable or disable tons of other fields, change validation requirements, hide or show parts of the page, you get the idea.

We had one junior dev who did mostly frontend stuff and me to handle the UI. The junior dev could handle HTML, CSS and some basic jQuery. Had she estimated the effort it would have taken her, it would have been 100+ story points.

I was a senior dev at this point and I suggested we use Angular for that page. And I estimated the work at a total of 30-ish points. Even though there was a learning curve for our junior dev, we ended up with a better UI with less effort than we would have otherwise.

The whole point of that story is that the effort depends a lot on the people doing the work. An experienced team can come up with better designs, work faster or have less problems. Some of this is swallowed up in team velocity, some isn't.

Back to what are you trying to gain. Are you trying to compare how teams work and how good they are? Because I'm pretty sure you are going to get that whether you want that or not.

When story points are relative to a team, the only thing you can legitimately compare a team to is itself. You can track increases in velocity, more accurate estimating, etc. for a given team. But you can't say "Team A gets 50 points done in a 2 week sprint and Team B only gets 30 done. Why is Team B slacking so much?". Any attempt at this can get shut down instantly by remembering that 1 Team A point =/= 1 Team B point (plus a whole bunch of other arguments). But as soon as you have a way to say 1 point is 1 point across teams, now direct comparisons are inevitable. Whether people consciously want to make those comparisons or not, they will happen. And I can promise some manager somewhere will do the same and use it to declare that a team is "under performing".

I may be missing something here, but all I can see is that this will end up with little benefit and a lot of potential downsides. In short, don't do it.

  • Thanks, but I in your example, it seems like you and the junior dev were not using the reference story. The story point should be the same, the individual effort differ. That is the point of story points, to agree on a relative effort. As somemody explained, two people can agree something is 1 mile away but both will walk there in a different pace. In your example, both of you should have compared to a reference story and agree that this task is approx. X times bigger. The individual time estimate would then be vastly different – John V Nov 1 '18 at 7:14
  • @JohnV The reference story was the same. The difference was the answer we arrived at to solve the problem. It's like both of us walking to a place a mile away, but I took a straight line path there whereas my coworker took a three mile detour. We both ended up at the same place, could have even walked at the same pace, but the effort expended was very different. – Becuzz Nov 1 '18 at 12:55
  • Exactly - but you could agree how much effort it might need. You could say it is just 10x this story. For you it means a few hours, for her a few days. But you both understand the effort involved. But of course I agree that more experienced devs of course might not more efficient ways that in turn are much faster to implement. – John V Nov 1 '18 at 13:11

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.