I suck at estimates. When someone asks me how long something will take, I don't even dare to make a guess since I will be completely off the mark. Usually I'm way too optimistic, and should probably multiply my guess with some large X factor...
How can I learn to make better estimates? It's not taught at my uni, and even though we have deadlines for all laborations I never think about how long something will actually take. Which I should. For everyone's sake (especially mine).
I'm still not great at it, but I have found that tracking how long you estimate for tasks and how long you actually take can be a big help. That way you can get a solid idea of how far off you usually are. Issue management software with time tracking (Jira in my case) or a spread sheet can be a big help with this.
I think more than anything it's an experience thing.
Murphy's Law of Time Management: To figure out how long something will take, figure out how long it should take and double it.
Then, move up to the next higher unit of time. Thus, we allocate two weeks for a one-day project.
Software Estimation by Steve McConnell (MS Press) is a good read.
The main thing with software estimate is summarized by the following
Without historical information, your estimates are useless.
This is one reason I think why iterative projects have much more success that large phased waterfall projects. They aren't trying to build out a plan for a year at a time with little information other than some black box voodoo of what they think it should be. Every iteration, they are reestimating/replanning and have the last several iterations to base their estimates on.
A few other points to keep in mind:
You can learn by doing them collectively.
I use Planning Poker. It a consensus-based technique for estimating.
Your estimation must be tracked and compared to what you have effectively done. You will get the Velocity.
Each time you estimate something, multiplicate by your recent velocity to get accurate estimation.
I'm surprised that nobody mentioned the PERT-style estimation technique that is described in Robert Martin's The Clean Coder. In that method, you estimate how long it will take for 3 scenarios: optimistic (O), nominal (N), and pessimistic (P). Then the expected duration = (O+4N+P)/6 and you get a standard deviation of (P-O)/6.
This seems to work pretty well, and I've used it a few times when management really cares about how long something will probably take.
As others have commented, I've also made estimates by examining historical data ("How long did it take to do this similar thing?").
But my favorite method is to not do time estimates at all, and only do point estimates and get a velocity over iterations. If a team is fairly consistent at sizing and completing work (user stories), then you save a ton of time by not even asking how long each thing will take.
Hour estimates are fiendishly hard to get right, and they require a lot of work to break things down into small enough chunks to effectively measure. And even then they're rarely correct because there are too many variables and we forget to account for things like sickness, vacation, or even distractions.
If I have to do hour estimates, I try to only do them for smallish tasks within an iteration. I measure everything in half-day estimates (4, 8, 12 hours) unless I know it could be less. But I rarely estimate anything at less than 1 hour.
First and most important, you have to define a process and stick by it. Include revising the plan at the end of each phase of the process. You can also revise the process, but in an orderly way.
Second, do some kind of design. Design is the first step to planning, you don't build a house without drawings.
Third, track time (effort). You should at least differentiate:
Analysis
Design
Code
Unit testing (include fixing defects)
Integration testing (include fixing defects)
Acceptance testing, with the user (include fixing defects)
It would be great if you measured the defects fixing effort for each testing type, but it adds complexity, so you can do it later on.
Fourth, identify key base items for estimating. For example:
Fifth, correlate base items and effort. For example:
Sixth, keep track of performance and deviation from estimates for each project. So you can fine tune your correlation factors.
Seventh, repeat and improve. You will gain a lot of insight just at the end of the first project, by the third you will feel at ease planning and estimating.
Have a look at http://en.wikipedia.org/wiki/Personal_Software_Process, it really works.
Whenever you encounter an estimation problem, try to split them in to smaller pieces. Then see if you have already done stuff similar to the pieces. If you have, you should already have a fair idea of how long each piece takes. If you don't, you should start to actively keep track of time taken for various kinds of tasks. This will help you in future estimations.
Total time needed will be more than the sum of the individual pieces, since you need some time for integration and testing.
If you haven't done something similar, you can probably rely on other peoples experience and get an estimate from them. Don't take this at face value though. Nothing teaches you like experience.
Its kind of like shooting a target. Previous shots at estimation should tell you how off the mark you are, so that you can correct it.
I find it easiest to do the process of division to the minimal tasks as mentioned above work out each one and then double that estimate. Then I add them together and add fifty percent. That gives me an approximate project time in ideal conditions. If the work is practically going to be happening in parallel with others it's going to need longer. If you are going to have to wait for other people, expect them to take twice as long as you think it will. Waiting for content or feedback or other information often takes far longer than seems possible.
Where I work we work out a best case/expected case/worst case estimate for each step of the process, which is useful as a guide and also for evaluating how your estimates have worked out.
The technique is not ever so important except that you need to be able to combat the programmer's temptation to underestimate tasks, but what is important is being conservative about when you can deliver something. If it takes you seven weeks to build something and you promised eight weeks, you can come in a little early and look good for it or do some extra testing and be assured of reliability. If you promised six weeks you can look bad even if it's absolutely not your fault. If in doubt, guess conservatively.
You could try to build a track record of what was the estimate and what was the actual for various tasks to build up enough of a record to then know what multiplier to have for specific things that get repeated in your list. Granted this is a trial and error exercise but it has seemed to work fine for me. There is also something to be said for many trials before the pattern emerges probably. This is likely similar to a lot of other answers that would say that one could boil down to, "Just do it!" as that is really how most of us developed the skill. Is it a major pain to see how wrong one can be when making estimates? Yes, but if the estimates get better then everyone can be happy eventually.
If you can decompose the project into smaller tasks and do estimates for those you will be more accurate over-all. Any task bigger than a couple days should be broken down further. If you can't break it down further than you probably have a requirements gap. If you have to do a back-of-the-napkin estimate for a one-line requirement well...nothing can really help you much. Sadly a lot of shops work this way much of the time.
Rather than write a book on it, I'll just offer a little advice on how to use the "break down" method of estimation:
Break your assignment down into smaller component tasks. Estimate each task as best as possible.
Add a task for planning and design (which includes what you're doing now.) Estimate it.
If you don't already have one, add a task for "bringing the tasks together." This task may not seem useful at first. However, when you use this "break down" method of estimation, there are always time consuming things to do that "fall between tasks" and that "pull the tasks together." This one can be tricky to estimate. Try your best.
Add a task for testing and documentation. Your assignment may not require a lot of testing and documentation, but you should at least spend a little time thinking about it.
Add up the task estimates to get an overall estimate.
Go ahead and multiply that total estimate by two††. This will give you padding time to:
And last, but not least, don't be afraid to sketch out estimates for yourself that are probably totally wrong. Sometimes just sketching everything out, no matter how potentially wildly inaccurate, can help you start on the path to getting a better sense for what's involved.
††As you get more and more experience, this "fudge factor" can be tuned to suit your personal style, and your work environment.
The formula that works when working for myself :
do a break down of todo's to a 1 - 4 hour granularity. I find that i'm usually accurate with these
the 'unknowns factor': Multiply by a factor of 2 raised to number of unknowns. I.e. if you are to develop a couchdb applicaiton, but do now know anything about javascript and http .. add 2 ^ 2 as mult factor.
context-switch-factor : multiple by 1.5 if you will work in perfect environment( at home in study corner etc ) , or 2.5 if you will work in imprefect environment ( office / crowded place etc )
I find this to be within +/- 20% of actual time taken !!
Learn your own bias. If your last estimate has been too low by factor two, next time, double your initial estimate. (Of course, Hofstadter's law makes it hard to do that right...)
It's also always a good idea to remember how much work was needed after the initial release of the previous work, and add that to the next estimate. E.g. your last task took 2 months to complete, but after going live, support, hotfixes and additional changes have cost you another month, so, next time estimate 3 months for a similar task.
For openers, read "Software Engineering Economics", by Barry Boehm, and "Controlling Software Projects", by Tom DeMarco. After you have read and digested those two, read "Software Cost Estimation With COCOMO 2", by Barry Boehm.
For what I have to say next, it will help you a LOT to have taken a probability and statistics class, even a basic cookbook one.
No estimate is perfect. There is some probability of coming in early, and some probability of coming in late. Boehm's original detailed COCOMO model gave predictions that turned out to be within 30% of the actual result, better than 60% of the time. That was a lot better than what was common when he wrote and published the book.
When you take your best guess (and that's all an estimate is), you are including those probabilities. If you pull the estimate in, you are increasing the probability that you will come in late. If you increase the estimate, you are increasing the probability that you will come in early or finish on time. How much you pull it in or let it out controls how the probability changes, and must necessarily depend on the penalties for being early or late. (Insert horror stories here - and there've been a LOT of them over the years!)
DeMarco addresses this to some extent. He also points out that there is an "impossibility region": some schedules are just too tight to be made, no matter what kind of heroics are attempted.
