4

We are having a disconnect between development and business needs.

Business is asking me to produce an accurate list of deliverables for a fixed date and development being difficult to predict is pushing back saying that can only produce a list 80 or 90% accurate.

So my question is, how do we solve this problem? How to provide an accurate list of deliverables weeks before they are completed and fully tested?

5
  • 5
    "How to provide an accurate list" - I've been taught long time ago, and learned myself the hard way since, that the right answer to this "how to" is: don't do it. Be forthright about not promising completed features at concrete date. Search web for something like time-driven vs feature-driven releases if you're interested in details. If you want to make business happy, just make sure that subsequent releases are delivered 1) at regular intervals and 2) each next release is better than previous and 3) business has a say in prioritizing stuff you plan to be released eventually
    – gnat
    Jul 20 '13 at 17:49
  • There is one list of deliverables that is archiveable with 100% certainty: nothing. For any other answer they would have to change the question or you would have to start to lie to them.
    – Patrick
    Jul 20 '13 at 20:15
  • Estimates should not be used to schedule projects - if you are doing that, you are doing it wrong. Use project plans to schedule projects. Estimates are just one of many inputs into the project plans.
    – mattnz
    Jul 21 '13 at 3:56
  • You should read some of Nassim Taleb's books (particularly the "Black Swan"). You will realize that "guessing" is more accurate than "predicting" or "forecasting". A "prediction" or "forecast" carries with it a false impression of accuracy (even though they are often less accurate than a best guess) and will cause you more problems, than, if you just guessed and accepted that it was only a guess. Aug 8 '13 at 4:40
6

The disconnect is that neither side understands the situation; business wants what it wants when it wants it and thinks that IT can just magically deliver regardless of the features and timeframes.

While it's nice for the business side to have that much confidence in IT, it is a 'fairy godmother syndrome' in action. IT is not Santa Claus, there are no magic elves, it all takes planning and work and time and experience, and it's never perfect.

Time is incompressible. It cannot be created. It can certainly be wasted though.

Accuracy can only be measured after you're done.

Confidence is a statistical measure based on estimates and black magic math.

If you say "we can have these 10 features done in 3 months with a confidence level of 80%" this implies a better than 97.79% confidence level for each feature [they multiply, it's not an average]. What you probably mean is "8 out of 10 of us think we can do this in 3 months", which is not the same thing.

All that aside, this is a fundamental project management issue, not limited to IT. It has nothing to do with Agile. There is no crystal ball that will guarantee an accurate list of deliverables months in advance. Businesses must understand the variables, but most of all the risks involved. IT must deliver the estimates along with the risks.

There are methods to help measure and improve, but gnat's comment about steady improvement is the most practical immediate advice - if the business side will go for it.

If not, then you guess, promise, fail, apologize, measure, and improve, like everyone else. And just when your numbers start lining up, the team dynamic will change and throw them off again, or the business will shift direction into new domains, restarting the learning curve.

That's the nature of project management.

4
  • I can reduce a plan into features where I can be 98% confident in completing that one feature. If you can't be then I suggest breaking the feature up until each piece is reasonable enough. You only have feature vs resource vs time. Seems time and resource is fixed in this case. Jul 20 '13 at 19:48
  • while i agree with the need to break up and trade off features when time and resources are fixed, i am curious how you arrive at a 98% confidence level Jul 21 '13 at 17:05
  • I didn't arrive at it. I was only citing your confidence level from your post about the 97.79%. I rounded up because I was on my phone at the time and didn't want to deal with accuracy in the percentage. I completely agree with your answer and the other answers from an idealist point of view. However, I have worked at companies that have no understanding of proper estimations, thus you have to learn how to manage expectations and give rough confidence levels with the realization you will eventually refine them as time goes on. Jul 21 '13 at 21:36
  • @AndrewFinnell: dangit, i was hoping you had The Secret!!!! :D Jul 21 '13 at 23:46
1

Joel Spolsky wrote a good blog about making schedules - Evidence Based Scheduling

Software developers don’t really like to make schedules. Usually, they try to get away without one. “It’ll be done when it’s done!” they say, expecting that such a brave, funny zinger will reduce their boss to a fit of giggles, and in the ensuing joviality, the schedule will be forgotten...

The main ideas are -
Break down work into really small tasks which can be estimated in hours (he suggests maximum 16 hours).

Make sure all devs record their time against tasks and use the difference between recorded time and estimated time to correct future estimates.

Regularly recalculate your schedule based on the latest estimate/recorded time data, so you can see well in advance that your ship date is slipping and adjust your date/feature set accordingly.

It's the sort of process that would probably require a few releases before it became really accurate, so it might not help you right now. It may however be something to consider implementing and might help you push back on the demands for a definite release date for this release.

If nothing else it's an interesting read as is most of Joel's blog.

1

Take 80% of the features and give a 99% confidence level given the original timeframe.

For example:

If there is a 80% confidence for 10 features in 3 months, then give them a list of 8 features (80% of 10) for a 3 month time frame with a 99% confidence.

You should also show what the risks are, how you will mitigate them and what happens if the risk occurs.

Update

Many times the business side is asking for estimates and confidence levels in order to make proper marketing decisions.

I love Joel's blogs. I wish more companies read him and understood the principals behind proper development project management and time estimations. Unfortunately many companies are still stuck in a manufacturing mentality - "If you can build 1 part in 1 hour, why can't you build 10 parts in 10 hours?"

As proper engineers we read books, go to sites such as stack exchange and participate in the community. Proper time estimation is as much about math as it is people management. You have to manage your stakeholders (bosses, customers, project manager, etc) while not over burdening your engineering staff with impractical schedules and feature lists.

Too often management sees the end goal and wants to understand the time and effort to get to the finish line. This is where projects fall apart and estimations become fiction. Build a project plan in which your confidence level starts high and ends low.

If there are 5 features and 6 months your confidence scale could look like this:

Feature 1: 90%

Feature 2: 80%

Feature 3: 60%

Feature 4: 30%

Feature 5: 5%

You feel very confident you can complete Feature 1 in 6 months. However as the features start adding up you feel less and less confident you'll make it through all 5 features given 6 months. As the project progresses and features are completed this confidence scale is adjusted. As you can see the only way to be reasonably confident in completing the project on time is to only release Feature 1.

To give a significantly high confidence level in completing a set of features, the confidence scale needs to extend out until each of the required features are within a desirable range and only at the last feature does the confidence start slipping. This brings in it's own difficulties though. The longer a project takes the high the risk. This is why I like the idea of SCRUM.

Disclaimer All the numbers I have given are made up based on analysis of a fictitious company. Each engineering group is different. The numbers are here only to illustrate that as the features increase the confidence level significantly drops.

There is actual math that should be used when performing these scales. I have willfully and shamefully decided not to use math to ensure my numbers add up.

Cone of Confidence

8
  • 99% confidence for 8 items implies 99.87% confidence in each of the 8 items... if you're 80% confident in each of 8 items, the overall confidence level would be 16.77% Jul 20 '13 at 19:27
  • 4
    Actual research into estimation shows that very, very few people can make an "80% confident estimate" and actually make it right 80% of the time.
    – btilly
    Jul 20 '13 at 19:40
  • The idea is to slide back the features until confidence reaches an acceptable point. Jul 20 '13 at 19:44
  • I took it to mean his team was only 80% confident in the whole project plan not 80% for each feature. Jul 20 '13 at 19:46
  • @AndrewFinnell The more complex the thing you're estimating, the less able people are to make estimates of this kind.
    – btilly
    Jul 21 '13 at 3:28
0

The one line answer

There is probably nothing you can do now, but you can plan to improve over time.

An analogy

Ask your business partners how long it will take them as a team to mow the grass surrounding one of the hotels in your town. Most likely they won't be able to do it, because they've never mowed the lawn of a hotel before. However, if they mow that lawn, and then you ask them to estimate how much to mow the yard of another hotel, they will be able to give you a slightly better estimate.

Keep doing this over the course of the whole summer, and after a few months they can probably tell you with a fairly high level of accuracy how long it will take for any hotel, once they've seen and measured the hotel in person. Even though every hotel is different, they will be able to break the job down into pieces ("mow around the pool", "mow around the entrance", etc), estimate each piece reasonably accurately, and thus estimate the entire job.

Plan to do better

There is no way to look at a specification or idea and estimate how long it will take. However, if you are able to break a project down into a set of stories and accurately estimate the amount of time it will take to do each story, you can then use the historical velocity of your team(s) to predict when those stories will be done. With a mature team this can yield fairly accurate results. You still won't be 100% correct, but the probability is high that you will be pretty close.

The trick here is that this only works well for experienced, mature teams. There are two variables in play that are hard to stabilize. First, your team needs a stabile velocity. That is, they need to be able to regularly produce X units of work in a fixed period of time. Once they are able to do that, then it becomes much, much easier to estimate new stories because they have a frame of reference ("we spent Y time on feature A last month, and feature B is about as hard as feature A, so we expect to spend Y time on feature B, too").

When you have that stable velocity, you need to use what you've learned to estimate stories. This means that stories need to be small, no more than few days worth of work. Once you are able to accurately size your stories, the deadlines largely become an exercise in mathematics.

All that being said, even the best of teams can't predict what will happen over the course of several months. Make sure your releases (even if only internal milestones) are relatively short. Work on a couple months worth. Hit that goal, then use the data to figure out how much you can do in the next month or two. Hit that goall, then lather, rinse, repeat.

Summary

  • Giving accurate estimates is hard, because for many teams every project is like doing something new.
  • Giving accurate estimates is a skill. Like other skills, it takes practice.
  • Giving accurate estimates requires team maturity.
  • Giving accurate estimates requires that the feature to be built is well defined (ie: broken down into estimable chunks).
  • Giving accurate estimates requires commitment. When your team gives an estimate, they need to commit to doing what they said they would do. This will force them to take estimation much more seriously.

One final thought

This isn't a software engineering problem, and this isn't a business team problem. This is an organizational problem. The organization needs to learn to work together to determine how much can be done by what date. It's not you versus them, it's us.

0

This comment is far, far too short to properly explain why estimation is hard, and how to do it better.

Software Estimation: Demystifying the Black Art by Steve McConnell is the right length.

Often referred to as the “black art” because of its complexity and uncertainty, software estimation is not as difficult or puzzling as people think. In fact, generating accurate estimates is straightforward—once you understand the art of creating them. In his highly anticipated book, acclaimed author Steve McConnell unravels the mystery to successful software estimation—distilling academic information and real-world experience into a practical guide for working software professionals. Instead of arcane treatises and rigid modeling techniques, this guide highlights a proven set of procedures, understandable formulas, and heuristics that individuals and development teams can apply to their projects to help achieve estimation proficiency.

Discover how to:

  • Estimate schedule and cost—or estimate the functionality that can be delivered within a given time frame
  • Avoid common software estimation mistakes
  • Learn estimation techniques for you, your team, and your organization * Estimate specific project activities—including development, management, and defect correction
  • Apply estimation approaches to any type of project—small or large, agile or traditional
  • Navigate the shark-infested political waters that surround project estimates...

Just remember that in most organizations, the estimated delivery date tends to be the first date that nobody can prove it won't be done by. That is part of why most organizations tend to underestimate schedules by a factor of 2, and manage to hit that schedule only because they strip out an average of half the promised features.

2
  • 1
    I wanted to give a -1 because this answer mostly just says "read this book", which isn't very helpful. However, a big +1 to counteract that for "the estimated delivery date tends to be the first date that nobody can prove it won't be done by" Jul 20 '13 at 22:57
  • @BryanOakley Have you read the book? If you do, you may understand why I did not wish to summarize.
    – btilly
    Jul 21 '13 at 3:27

Not the answer you're looking for? Browse other questions tagged or ask your own question.