Dealing with awful estimates

Question

A recent project I worked on was proven to be severely underestimated by the architect. The estimate was out by at least 500%.

Unfortunately I was brought onto the project after the estimate had been signed off with the customer. As senior dev, I quickly realised that the functional and technical spec. contained some huge gaps and uncertanties.

As a result I felt compelled to call an emergency meeting with the business and technical directors to let them know the reality. As first and foremost a developer, I found this a very stressful and difficult situation. The "business" accused IT of being incompetent and being the messenger I received a few "bullets".

The customer threatened to cancel the account, however to date the project is still unfinished and I am no longer directly involved with it.

The architect was a nice guy socially, but based on this episode was either simply incompetent or there were large sales/business pressures influencing his estimate.

So, as programmers, what is your experience of this sort of situation and how would you advise dealing with it?

Answer 1

Long reply, but hey, I’ve got a summary on the end, so just skip to summary if you can’t be bothered reading the entire thing!

As a developer I had to deal with the situation literally every other project, but it's not until I moved into project management that I learned how to deal with it effectively. For me dealing effectively is about two things: managing expectations and understanding how estimation works.

Start with a premise that it is unethical to provide an estimate, commit to an estimate or give any other indication of estimate accuracy without being able to carry some due diligence first. Other people rely on your professional ability to predict an amount of work required, giving a false indication will hurt them and their business.

But you have to give something, in real life you dragged into an impromptu meeting or a late project and your superior will probably make it clear they expect you to come up with some figure straight away or comment on the estimate they provided. This is where expectations management comes into play.

Explain that it would be wrong of you to give any figure or any indication without understanding the problem and working the numbers out for yourself. Say that their figures might be quite correct, you just don’t know before you went through the estimation exercise yourself. And even though you might have a good picture of what is needed there and when, say that you still need some time to work the numbers out. There is only one estimate they might expect you to give: when you going to be able to provide an estimate. By all means do provide that figure.

As a developer never take responsibility for (or give indication that can be interpreted as acceptance of) other people estimates without being able to review them first. As a project manager it is a totally different matter, because then you actually have some control over the estimation process: the way an estimate is derived and reviewed and you have to rely on other people to get the actual work done and you need to make sure they are committed.

Never even comment on estimates without being able to do the due diligence. This is ethical. A lawyer or a doctor will make it absolutely clear they cannot give any advice unless a client (or patient) plays by their rules and goes through an assessment procedure first. You similarly have a right to satisfy your questions before giving professional opinion.

The second part is about how estimation works. I suggest researching various approaches to doing estimates and how estimation process works, including industries other than software development (manufacturing, financial markets, construction). This will give you idea what can be reasonably expected from you by your boss or client and, strangely, will help making more accurate predictions about the amount of work. It will improve your ability to defend your estimates and you will need to defend the figures every time they are different from the ones provided by an architect or a sales person.

Normally, the way it works, is that your estimate is first scanned for odd looking or relatively large items. Hence be prepared to defend anything with “non-standard” name. Also split larger tasks so that all tasks have same order of magnitude, i.e. if most tasks take 2 days and one single task is 10 days be prepared to get drilled.

Be clear about what is included in each task, its best to split dev and unit testing instead of just having dev and having someone assume that it includes documentation as well. Obviously this way you’ll need to produce a fairly fine grained estimate.

Next the drilling comes. Since it is quite difficult to review a long work breakdown your client or boss is likely to adopt a different strategy: concentrate on a random bit they might know something about and drill down until they manage to discredit the entire estimate or will be satisfied with your answers. The credibility of entire estimate might depend on a random sample! Hence once again, you need time to prepare it carefully, include only relevant bits, exclude any extras or move them to a “nice to haves” section and think through how you going to defend the figures.

Obviously you can be either consistent in your approach, i.e. estimating on the basis of features, number of screens etc or have a mix of approaches, but in any case be prepared to defend why you selected a certain way of estimation. Be also prepared to explain why your figures are different from whoever else’s attempt at predicting the amount of work required.

Learn the obvious signs of weak estimates:

• Filled with general run-of-the-mill tasks, copied from template (good estimates are specific to the task at hand).

• Coarse grained estimates (i.e. tasks longer than couple of days).

• Estimates done on early stage of a project or by someone who might not have actual knowledge of the requirements or work involved.

• Estimates compiled by people other than actual doers

• Vague estimates (not clear what is included and, equally important, excluded).

• Substantial difference in the order of task magnitudes.

Practise in evaluating other people estimates and drilling the figures without actual knowledge of implementation detail. This will help to back your claim for some extra time when pressed with the request to confirm someone else’s estimate when you have no hard evidence.

To summarise:

• Do not commit to an awful or any estimate for that matter, before you had an opportunity to do due diligence.

• Make it clear on the outset, don’t let anyone assume it is any other way and interpret your silence as a sign of agreement.

• Know how various estimation methods work, their practical application and merits, including these outside software development.

• Be prepared to defend your estimate.

• Learn how to evaluate other people estimates so you don’t have to commit yourself to vastly inaccurate figures.

Answer 2

It's impossible to predict the future. Requiring a prediction ("estimate") is simply asking for trouble. Everyone does it, and everyone gets it wrong.

Your judgement of "out by 500%" is probably just as wrong as the architect's estimate. After all, "...to date the project is still unfinished..." There are no facts available here.

No one knows the "correct" answer. And until it's done, no one will know.

And even after it's done, the "original" estimate (with or without change control) may not correlate with anything that was actually accomplished.

Estimating is a trap -- it's a rigged game. you can't win, you can't break even and you can't even get out of the game.

---

Edit

Dealing with bad estimates; A "legacy" estimate that appears wrong.

There it is. You don't agree with someone else's estimates. Either they omitted something you think is necessary; they had a different scope of work than you had, or they had a different productivity rate. Also, if the estimate is more than just effort, they have a different cost basis. All of which are disputable. So dispute the details leading up to the estimate. Don't dispute the overall number -- there's not real information in a summary. Dispute individual details that underpin the estimate. Find out what they were thinking.

It's just as likely that your assumptions are as wrong as their assumptions. Equally likely.

When Asked To Estimate.

1. You are going to be wrong.

   1. They lie about the scope of work.

   2. You don't know the team's productivity.

   3. Whatever new technology is involved has been misrepresented.

2. You can't just throw in padding to cover these things randomly. You actually don't know and don't have a basis for "estimating". It's just guessing. Get over it.

Rule 1: Since you're only guessing, guess in small increments.

The fundamental question in any "estimating" situation is not predicting the future. You can't do that with any accuracy over periods of time much longer than a week or two. Limit your predictions to a time horizon over which you have some direct and immediate detailed knowledge. For example, the next release.

The fundamental question is -- usually -- decision-making on the part of your buyers or customers. The question isn't "what will is cost?" -- that's incomplete. The question is "will the investment be worth it?" The real question is more along the lines of "what's the cost/benefit ratio" and "when should we stop spending because more investment won't create more return?" Those are the real questions.

Rule 2: Support the decision-maker with factual information.

Most folks are best served by an Agile approach. The first release -- a month from now -- will take 5 people × 4 weeks and it will yield feature X that fixes the $1m/day losses and feature Y that fixes the $200K/week missing opportunities. You have detailed knowledge of what you're doing, so this prediction makes sense. The release after that is a little hazy.

The release a year from now is so far in the future that any prediction in just a random number. Don't sweat the details of anything more than 6 months in the future, just use simple rules of thumb.

When they ask what the TCO is, you have to be honest. "The total cost occurs when you stop paying for development. Until you stop paying, you'll always incur costs."

The real question is "what problems are you trying to fix?" or "what new opportunity are you pursuing?" and "what is that worth?"

Rule 3: Make the software less costly than the problem it's supposed to solve.

If you don't know the problem very well, the estimate will be gamed to fit the perception. Try to avoid this.

---

On Probability. Except for debilitating disease or accident, no part of software development is governed by actual probabilities. The "risks" are simply bad management. Generally of the form "we didn't account for the complexity of business need A or technology B". More often of the form "as we learned more about the problem or the technology, we altered the schedule" which is punished as "scope creep".

There's no probability of learning stuff and changing the scope. That's a certainty.

On Planning. There's "planning" and there's "estimating". Planning what to build is one thing, best represented as a checklist or a dependency graph. "Estimating" the effort required is based on unknowable factors.

"Planning" is ordinary good management.

"Estimating" requires unknowable knowledge. To "estimate effort" accurately, you must have source-code level of insight into the product and you must know which person is going to type that source code and what mistakes that individual is going to make. Since you can't know that, any estimate must be wrong. And often wrong the point of misleading and therefore useless.

If the estimate was out by 500% and the project still went forward, what value does an estimate have?

None. All it did was make people unhappy. But the project went forward anyway.

Since no one can see the future, getting an estimate right doesn't mean anything. Make it useful, help people make decisions.

---

Keep the horizon short. Deliver value as quickly as possible. Create a plan that allows the customer to cancel the project at any time and still have value.

Don't let the plan become the only "sacred truth" in the project. The deliverable functionality is sacred. Everything else should change as the deliverables change.

Don't let the plan get beyond the value it's creating.

Answer 3

If there is not enough time, there is not enough time. There is no magic solution to finish the project anyway. In my opinion you have done what you could be stating it out. Turned out they had to find it out the hard way. That is how it usually goes. Hopefully the architect and the management have learned from their mistakes and don't do it again. If this is business as usual when the management is too ignorant to listen to your arguments and take appropriate action then it might be a good idea to look for other projects or another company.

"Developers are craftsmen, and the worst thing you can do to a craftsmen is to have him deliver a crappy product." Famous quote from somewhere but I can't recall from where.

Answer 4

Honesty should always be honored. I was on the receiving end of of an "architect's vision", and when the developer came to me with the dire news that the entire solution would not work, we went to the business units and had the awful conversation. The developer then came up with a new estimate which as comprised of 80% of the functionality, but he delivered on time and on budget.

The business units were won over by the fact that the developer spoke truthfully to them, acknowledged his companies short comings, and worked like a dog to deliver. We have had this guy work for us for the last 7 years because he was always honest.

The entire scenario is so rare - most times the business units think you're an idiot for not being able to deliver. You need to seek out those customers who do not operate this way, as you will earn more with them in the long run than you would trying to please the cretins who just want to treat you like a jerk.

In respects to architects who don't know their field, avoid them like the plague. I had a mentor who used to reinforce with me in a harsh way - "This. Is not. For. Practice!" He would tolerate mistakes only if you made them early, corrected them, and never made them again. Companies who allow non-technical, inexperienced people in positions with customers because they can "communicate" deserve to go out of business.

Answer 5

I had once faced this situation. A project ran out of schedule this was because the business changed the requirement and there was communication gap and senior management wanted the project at the project on time. To make it worse one guy who was working on this project was pulled out to fill another gap which had more priority.

My team was spending nights to finish the project. Late one night at around 3:00 AM in the morning (I was working 19 hours straight) I emailed my managers to do something about it.

The next day we had an meeting (Just the dev guys). The whole team came in on a weekend and tested the project even before it was complete. Few others joined the team in doing quick fixes. At last we were able to deliver the project with the effort of the whole team (Not just the project team). We followed the same process for other projects.

The whole team (even if they are not involved in the project) tested the application and helped in quick bug fixes.

Note: My team consists of about 25 people which again had sub teams working on different projects.

My only advice would be "Speak to the Manager. Tell them firmly that the project cannt be delivered on time. Also give them an alternative. Manager always expect the baby to be delivered on time no matter what :)"

Answer 6

Whilst businesses don't often like the truth that things take much longer than expected, they prefer being strung along even less. The sooner you let someone know how long it is really going to take, the quicker everybody can plan around the circumstances. Whilst this can initially be a tough time, in the long run it will be easier. Just get it right the second time and build in contingency.

Answer 7

Let me emphasize one key point when dealing with your management: managers appreciate solutions.

If you have to go to your management with a problem (e.g. the estimate is wildly unrealistic), work hard in advance to include alternatives for how to address the situation. For example, you could do a Pareto analysis (the 80/20 rule) to understand the value of the system, you could make a prioritized case for cutting features (at least from a first release) to concentrate on those with maximum business value, you could look for alternatives (open source projects, etc.) that are adequate replacements for custom parts of the system, etc.

There's no question that a five-pound bag won't hold twenty-five pounds of sand, but part of helping your management absorb your unpleasant message is offering evidence that you're an engaged ally.

Answer 8

You might be interested in this IEEE article that I have blogged about before. Here are the highlights.

• One of the biggest drivers to project failure is overly optimistic estimations.
• One big reason why estimates are too low is too much pressure from the top to deliver early.
• The other is scope creep during implement with no re-visitation of the estimates.

Answer 9

Firstly I would talk to the architect in question, informally, and go through a list of your problems with his estimate, and try and convince him where the problems are, and the difference in time they'd take to resolve.

Then I'd try and go to your line manager/project manager and discuss it with him/them.

At the end of the day, the architect gave ESTIMATES, so they are subject to change, and sometimes by large amounts, so as long as they're made aware of it, so they can make alternative plans, like rolling a product out in phases, reducing it's functionality or other means.

At the end of the day, once you've done the above, it's no longer your responsibility.

You should definitely not go directly to the client, or the architects boss, this only promotes bad feelings, and almost invariably you'll get some of the blame.

Answer 10

The best thing you can do is learn from your (not yours personally, in this case) bad estimates. One thing to learn is to never let someone other than the person implementing the ideas estimate how long it'll take. Programmers' speeds can vary by an order of magnitude, so estimating for someone else is next to impossible.

Answer 11

I feel your pain... I am not sure how to handle all this frustration :(

https://stackoverflow.com/questions/541873/developing-on-for-a-moving-target

Answer 12

In the past I had to deal with Project managers who slashed estimates to meet a figure that the sales department thought the customer would pay. The manager was of the opinion that it was better to beg for forgiveness than ask permission.

This is why developers have learned to pad their estimates, because they know they will be cut by their managers. So if you double the estimate and add 30% you run a good chance of getting a schedule that is reasonable.

Customers always want it cheaper, and if you come to them with a reasonable estimate, they will balk at it and demand you cut the cost or they walk. But, if you go too high, they'll simply walk without discussion ("We'll consider it and get back to you").

It's a game of managed expectations.

Answer 13

The problem wasn't that the original estimates were out - it's that management didn't believe you.

The best way to get management to make a decision is to:

1. Outline the problem with evidence to back it up; and
2. Provide multiple solutions for them to choose from (in order from least preferable to most preferable).

For (1) implementing Scrum and tracking actuals against the dodgy estimates works well to back up your claims.

For (2) one of my options would be to "Develop a Prioritised Feature List with the client (aka Product Backlog in Scrum terms)". This would be tricky in fixed-price or highly bureaucratic organisations, but it's possible.

Hope that helps!

Answer 14

I (as I'm sure just about everyone who codes) empathize. My last company was pretty terrible about this - the sales guys would go in and sell a project, and then you come in, see the estimates, and just laugh.

As Tomh mentioned - there is only so much time in a day. Even if you don't sleep.

Three things, I think.

Most of the time you just have to manage the client's expectations and be transparent. Be forthright with what you can do and show what you've got done - all of it. This is especially true if you're handed requirements that are way too coarsely chunked up (as Totophil mentioned.) If they can see the amount of work you've had to do, they can see how bad the estimate was. If they see productivity and progress that's more important than anything in my experience.

I think besides managing expectations and being transparent in your workload, the other big thing is scope management. There's a large area between being anal/offensive in being a scope nazi and covering your own tail end. When someone wants this extra feature or functionality - agree to it! And then give them a relatively accurate estimation on how much time that will add to the project (or next release.) The upside of this is not only not cramming yourself into another 80 hour week - you get some pad in that estimate too to possibly catch up with the other.

Good luck!

Answer 15

Never take on anything without seeing or understanding it. If the customer, or your own mgmt isn't willing to afford you that much, they are not setting you up to succeed.

This was (and often is) a failure to understand the details, data, and how they interact throughout the application being built. Assumptions are made instead of asking questions, finding answers, and nailing it all down.

One thing I often say to my clients is, if you're going to hang me, at least let me hang myself with my own rope, or shoot me with my own gun and bullets, not someone elses.

Having a revolving door of people trying to solve it will be far worse in the end for them.

The unrealistic, poor planning and lacking foresight may be signals of an organization wide issue in which case you better get used to it, or move on.

Answer 16

First, consider the possibilty that you're overestiming the scope of the project. Salespeople and architects tend to exaggerate their solutions. Don't take them at face value; they probably expect you to come up with less then they promised the customer.

What I would do here is take the amount of time I do have, and spend it as wisely as I can. Figure out the customer's priorities and deliver on those. Nine out of ten times, the customer will be happy his top priorities have been met, and overlook the 80% of the work that has not been done.

The last thing you want to do is go about calling emergency meetings and accusing people of being evil. You say:

"let them know the reality"

but reality is just an opinion! Relax, do your job, and spend your politcal capital on things that matter to you. Like, promotion, more money, more holidays.

Your boss trading a promotion for you working really hard on the customer makes sense. You going haywire on behalf of a customer does not.

Answer 17

One thing to remember is that estimates do not include scope creep or unavoidable delays (or problems based on the client not giving you waht they said they would such as a file of information in a particular format).

We have learned to push back both the estimate and/or the due date every single time one of these things happens. Add a new feature, get a new estimate and a new due date. Fail to provide needed information on the date agreed, get a new due date. Give the informaqtion but make it incomplete or incorrect or in anyway other than what was promised, send out a new estimate and due date, change the requirements of agreed-on features, get a new estimate and due date. Stop working on the project because the client wanted you to work on a higher priority, send out a new due date (and possibly new estimate becasue there will be catch up time if the project is on hold for long).

If the orginal estimate came from outside the development group and they were not comsulted on it, then when you get it, prepare an estimate of your own (at a detailed level of all the tasks you think it will have - at a far higher level of detail than the estimate you were given) and immediatly provide it up the chain and ask what you should cut out in order to meet the estimate you were given.

If the answer is nothing, insist the client be informed of the new estimate, now that we have looked at the matter in more depth. If management stil insists the client will only pay for X hours, ask them who will pay for the rest of the develpment hours because the job as described to you cannot be done for less than your estimate. It might turn out the company is willing to take the hit and pay for the hours themselves.

If they aren't willing to take it out of their profit and they won't tell the client they need more hours and the company won't back the development staff's detailed estimates over sales' "what we think the customer will go for in order to win the project" estimate, you are on a death march project and your best bet is to get out as soon as possible. You can point out that the client will be happier knowing the project will take longer as soon as possible than they will when you spend the 50 hours they agreed to pay for and aren't even close to being done with the 500 you really needed. Remind them that overly optimistic estimates are one of the biggest predictors of project failure. It won't work with these types of companies, but maybe eventually it will sink in if you repeat it enough times.

Answer 18

I would also take in account estimate refinement. I mean "as I see it now this project will take X man-hours". After 20-30% I will re-estimate and so on.

After all how does a file downloader do its estimation? It constantly refines it.

Answer 19

I think not enough estimators do not put enough emphasis on the facts of "Estimation is you asking me to do math and guesses to predict the future in a useful way" and "The commitment we make is completely separate from the math that we do to make the estimate; We can agree to do stupid amounts of work, agree to things we see we likely can't finish, but none of these really change the math of the solution" and "We can do development methodologies that aren't giant batch of features A will be done by date B that make failure much less likely"

To many estimates are couched in the language of negotiation. They need to be couched in the language of math and talking about the uncertainties of the math.

The estimator can do nothing to make reality bend to the will of the businesspeople out negotiating him. His only job is to make them stop trying.