How to keep management out of our development process

Question

I'm a software engineer in an software development team. The last 3 years we worked for an internal customer on a new product. Now this product is finished we're going to work on major new features for existing products. For a particular feature, product management have guessed it takes 150 hours to develop. Together with our project manager we have created a very detailed plan and we come to an effort of 300 hours. Yesterday we discussed this and they think we have grossly overestimated things.

In our planning we estimated hours for writing unit tests, their idea is to dump them to save time. The decision has not been made yet and I will defend this planning and the unit tests if needed. But what I really don't like here is that management is interfering with our development process. How do I keep them out of our development process? And what arguments could I use to keep the unit testing in place (besides quality and long term time saving)?

As a side note our company has 3 engineering teams and the team I'm in delivers their software on time (give or take a 10% margin). While the other teams always deliver late, mostly due to underestimation in planning. They only plan the coding and not the management, testing and handling around it.

Answer 1

First, let me say I completely sympathize with your position. It can be frustrating when you have a lack of understanding or a communication breakdown between different parts of the business. Having said that, I don't think you should try to keep them out. Instead you should show them the numbers about why this is a good idea. What facts do you have that justify unit testing is worth the effort you put into it? If you don't have any, then you should start to gather those figures, or show some research to back up your claims.

I have had to deal with similar scenarios myself and I answered this question on a similar topic. I also blogged about how I dealt with it here:

http://practicalagility.com/show-them-the-numbers-its-results-that-matter/

In case you don't feel like link chasing, I'll repeat my summary from the related question:

To summarize, I compared our estimated hours against actual hours for the project and then compared our defect rate against other teams' defect rate. In our case these numbers compared favourably and there were no more concerns.

My conclusion based on this experience is:

...the best way to convince anyone that your approach to doing something is practical and pragmatic, is to do it and measure it against other approaches. People don’t care about dogma, or why you think something should be the best way. Only by showing people the numbers and measuring the effectiveness of your approach can you truly show that your practices are effective.

If your management team don't agree to investing what they see as an additional 150 hours on unit testing, perhaps you can convince them to invest in one small area of the product (or even agree to suck the hours up yourselves to provide some data). Do unit testing in that one area of the product then gather data about the defect rates in that area of the product and the cost to find and fix those defects compared to the defect rates in other areas of the product. Hopefully you'll gather some data to back up your case and this will be a non-issue for your next project.

Answer 2

The number one rule to follow, regardless of the method you use, is that

1. Developers should have the right to estimate their own work.
2. Stakeholders should have the right to prioritize among that work.

Estimation and prioritization are two forces that work very well together once both parties accepts their own responsibilities. So instead of wasting time arguing, agree upon this and respect that both parties will do their work to the best of their abilities.

Answer 3

You might point out that unit tests save time, so if you drop them the estimate goes to 500 hours.

Answer 4

Tell them about technical debt and the value of unit testing

Look at this post from some nice idea's on technical debt. Following through from that post you can get to the following pdf

I like this post on the value of unit testing (there are probably more to find)

The hope is not to get them out of your development process but get them involved and committed in the right way.

IMHO you need to write your original planning down, add chapters 1 and 2 (not in an appendix) in which you explain technical debt and the value of unit testing. Give them alternatives:

• less hours (not the entire 150, that sounds ridiculous) where every change (during development phase and during maintenance) on average will take:
  • small 4 hours
  • medium 16 hours
  • large 40 hours
• your estimated hours where every change (development phase and during maintenance) on average will take:
  • small 2 hours
  • medium 8 hours
  • large 20 hours

(The hours are just indicative. You are way better equipped to give proper estimates.)

Don't forget to include your track record for on-time on-budget deliveries.

Write it down and discuss this with them. They might have some valuable points in features the don't actually need right now or some technical debt they are willing to take in order to deliver on time. Just make sure these are conscious choices.

Hope this helps and good luck.

Answer 5

First of all, don't split out "write unit tests" as a separate task to be estimated, scheduled, and potentially, cut. Your estimates should be at the feature level "Implement the XYZ -- 18 hours". Those 18 hours should include whatever it takes in your process to get that feature to "DONE", including "write unit tests".

That's one good way go get the non-technical development "out of your development process". Don't include your development process in the task list or project schedule that you give them!

Secondly, it sounds like your team is already delivering good products to them and on time, but that other teams are not. Maybe this management group is used to having to micromanage those teams. Play to your strengths -- offer to show them weekly or bi-weekly updates with working features, and they will get off your back about "development process".

Answer 6

I once was in a situation where I was working with a code base in a very good state; a challenging new feature was needed in a very short time frame, and I managed to deliver the feature in a very short time. At that point the code base was in a much worse state. So the feature was delivered, but my work was not done: I had to put everything back into an equally good state.

I explained to the manager two levels up like this: It's like doing a paint job in your home. If all tools are there where they belong and in a good state, all the brushes cleaned and so on, you can do a paint job very quickly. But then you have to spend the time to put all your tools back in order. If you don't do that, then your next paint job will take a lot longer. Actually, you won't remember where your tools are, your paint brushes cannot be salvaged anymore, and it costs you much more extra time and money as if you had done the cleanup job immediately.

And the same in my programming job: By cleaning up, I get the codebase into a state where I can deliver something very quickly again the next time it is needed. If not, the next time it will take a lot longer.

Answer 7

You can't keep them out of your process completely, after all they pay your wages and they will be using your product (if not directly, presumably someone in your company is the end user).

Managers accusing devs of over-estimating time is a very common scenario in my experience, and if not dealt with it can lead to a pretty dumb arms race where you next estimates are doubled because you know the bosses will halve them, they know this so they quarter them, so you quadruple them etc etc. You need to avoid this vicious circle if possible.

Assuming that there is no "drop dead" reason for the deadline then I would suggest 2 things.

1. Deliver a detailed plan of what you think you can do in 150 hours, sticking to your current approach of high quality work. Enumerate exactly what can be delivered in this time frame. The answer from KeesDijk has some very good links on planning at a fine grained level.
2. Carry on in the same style of detailed planning to cover all features and show how it will take 300 hours (or whatever the figure comes out at).

Then get to work and report back regularly on progress, and if at possible have some deliverables at regular intervals. This should give the management confidence in your estimating skills and ability to deliver.

Answer 8

Ask them for the basis of their estimates. It is only fair to discuss the discrepancies. Dumping unit tests is a false economy, what you don't spend writing unit tests you will spend in a debugger later (and longer). Essentially, you have documented the fact you plan on testing the work you complete. They are telling you not to test at all. Whether you test using unit tests or ad hoc testing as you develop the project you still need to account for that time. Removing the time you allotted for unit testing also removes the time allotted for ad hoc testing.

Bottom line: stick to your guns with your estimate. Your track record shows you know what you are talking about, and can give a reasonable answer as to the basis of your estimate (assumptions, expectations, past performance, etc.). It seems as if your upper management doesn't have the visibility they need to create a reasonable estimate. Are they assuming 8 hour days with no interruptions for meetings? Are they budgeting for system testing in their estimates? How did they come up with the number that is half of yours, considering your track record?

Answer 9

I would estimate it will take 300 hours and if they budget 150 give them the option it will be either buggy rush job or late to be delivered. When project is complete and it is as you predict then you can just tell them that is what you asked for.

Answer 10

What would Wally do?

There are multiple ways of interpreting what management ask of you, one is that they don't want you to deliver on time.

Seems absurd? Yes, but how else can they know that you are not overestimating? Don't meet your deadlines (slack if necessary), should you slip and accidentally deliver something on time be sure to look really tired as to not give the impression that it was a walk in the park.

Answer 11

You're in a pickle. If you stick to your guns and want to stick with unit tests, and want to claim the 300 hours, you'll make enemies.

If you reduce to 150 hours and chuck unit tests, you can deliver a buggier product faster, but it will cause grief down the road, with a higher maintenance cost.

Either way, you lose.

Or so it seems.

You see, you are not running a science lab at a university. You are providing a business service to a business unit in a company that provides services to customers in an ecosystem of companies. Your company may need your product to start delivering faster and better services to its customers, and thus raise needed revenues.

You see, what you need is a ROI analysis, and you don't have all the data to make that analysis. You only have some of the cost part (you don't know everyone's payroll numbers) and you certainly don't have the revenue parts, especially not the revenue projections.

Your management, believe it or not, is adept at making the ROI projections (that's what they teach in business school) and may have run multiple ROI projections and come up with the "if we act now we'll make so much more money even with paying triple for the maintenance on the software the bozos in IT complain about."

So, if you want to run the joint, start your own company. You'll see, it's not that easy.

In other words: do what you're told. If management knows what it's doing, you'll come out ahead. If not, you're out of a job, unit tests or not.

What's ROI you ask? Return on Investment. It's a bad name though. It needs to be Return On Timely Investment (ROTI), because timing is everything in investment.