Dealing with management that does not see value in improvements that are not immediately visible to the user

Question

I can understand schedule pressure. You want to please your users, as they are the lifeblood of the company. However, it is also true that certain changes will make everything easier down the road. Unfortunately, management in my organization has an instinctive resistance to such changes and this resistance is so strong that it gets in the way of long-term improvements.

For example, Apple recently introduced Automatic Reference Counting for iOS programs. This is a major improvement over the manual retain/release calls one previously had to use. The code is easier to write and easier to maintain. The changeover itself is likely to produce some crashes. But once those are worked out, the number of random weird crashes is likely to go down.

I recently mentioned to my boss that I wanted to switch to automatic reference counting. His response was that he wanted to concentrate on visible improvements. It is likely that this response was in turn driven by pressure he is getting from above him - and probably right from the CEO.

There are a lot of similar examples. The common thread is that something needs to be fixed but the short-term costs of the fix outweigh the short-term benefits, where "short term" is defined as "within the next few weeks."

How should I handle the situation?

EDIT: Thanks for the responses. Keep 'em coming. Because it is relevant to my situation, I should make it clear that my manager and the CEO are both programmers -- though the CEO may by now have forgotten what this is like. Apparently their programmer sides have been overwhelmed by other pressures.

Answer 1

You are really talking about technical debt. Maybe a metaphor would help your managers. I often compare the effect of technical debt in software to cooking in a dirty kitchen. If the sink and counters and stove are piled with dirty dishes and there is trash on the floor, it takes longer to make a meal. However, the fastest way to prepare the very next meal is to work around the mess. Cleaning the kitchen, and keeping it clean, will delay the next meal, but will improve the delivery of all subsequent meals. And just as the hungry person in the dining room can't see the messy kitchen, and won't understand why you want to clean up before starting to cook, your management can't see the mess in the code. You need to either show them the mess, or show the quality problems and delays that are caused by the mess.

Perhaps you could also talk about urgent tasks and important tasks. When important tasks aren't done, then urgent tasks take longer and cost more.

Answer 2

You've stumbled across something that plagues programmers everywhere at some point in their careers: this code needs to be refactored, there are architectural issues over there, this module is becoming unmaintainable, etc. Because of the present culture of your organization, however, you're being pushed to focus on work that only yields directly visible benefits.

Its the classic Iceberg Secret all over again. The secret has to do with the fact that just as an iceberg is 90% underwater, so to is the majority of any development project: 90% of the work is going to be completely invisible to the end user. That code will have an impact on the end user but management has trouble wrapping their minds around why you spent six hours refactoring the maintain/release and automatic referencing calls when They Can't See Any Difference and everything is Working Just Fine.

Here are some facts you can take with you on this issue.

• Management, unless they are programmers themselves, are not going to understand the Iceberg Secret.
• This is a problem of ignorance, not malice. The CEO wants a good product - he just doesn't understand everything that goes into a good product.
• The CEO (and your direct boss) are not stupid - study and prepare some facts and some concrete data for why you should spend the time on this, and other Iceberg problems.

Don't forget - you're a company man (or woman). Not a code man. You're developing this product for a company who has a vested interest in its success or failure - your projects and project proposals should reflect this. Show your passion for the company and the product, show your knowledge, and prove to your boss and CEO that they should trust you when you come to them and say that Something Needs Work. Show them how it will contribute to the bottom line - whether by adding value to product (more people buying copies) or by saving time down the road (fewer angry customers when your product fails).

Answer 3

You don't.

I see this question and all questions like it as a bit of a dead end. You can't "convince" people of anything. If they aren't already aware of things like this or investigating it, chances are they don't give a flip. And no amount of data will convince them otherwise. Change must come from within. You can lead a horse to water but you can't make him drink.

I say bake your desired changes into your next technical estimates. Be like, hey, we "have to" upgrade to this new framework Apple introduced. Don't let not using ARC be on your roadmap. There are no options; migrating to ARC is the only way.

Answer 4

I've answered a similar question here before so this might be considered a duplicate. Basically, you're not going to get signoff to do a "refactoring effort". The way you make the code cleaner is to follow the boy scout rule: always leave code cleaner when you leave it than when you arrived.

Just like paying down real debt can seem like an insurmountable task (or cleaning up a messy house). The trick is to make it better piece by piece until you start seeing "islands of cleanliness". Once you have significant momentum, other developers on the team will start noticing and eventually contributing to the task.

I'd suggest reading the Clean Coder by "Uncle" Bob Martin. Writing good code is part of your job. You don't ask permission to do your job, you just do it.

Answer 5

As with other questions of this nature, you need to provide numbers that management will understand. Numbers that show how much time will be saved by implementing these improvements, how many less "random weird crashes" will occur, etc. Convince them that crashes are visible to the end user and that anything done to prevent them is good for business.

You could also attempt to implement these improvements on your own time (i.e. outside of work hours) and then show the benefits to management afterward. I would only do this when it is clear that management does not understand what you are trying to convey and/or that they don't want to allocate time for you to even attempt it.

Good luck!

Answer 6

Present a Business Case

There are many reasons why engineer recommendations often get ignored. The best way to deal with almost all of the reasons is present the business case of why it should be done. The classic cost/benefit analysis. This not only makes a compelling argument, but it also gives your bosses something to take to their higher-ups.

• What is the upfront cost?
• What is the ongoing cost?
• What are the projected money/time savings and where do they come from?
• How long will it take before we see the ROI?

When doing a business case you should always back up your argument with data.

• How much time is development currently spending dealing with problems this will remove or mitigate?
• How many user complaints do you get related to the problems this will remove or mitigate?
• What other benefits will it have?

Line up the numbers and make it a rough, but simple equation. It will cost X to do and it will benefit the company Y.

Note: Don't be surprised if it is prohibitively expensive to implement an academically good idea.

Answer 7

This kind of a change falls into the refactoring category. The Agile approach would be that you should be incorporating AMPLE refactoring time into each story you estimate and this is exactly why. Aside from engineers, nobody is going to understand why you want to do this and that's okay, it's not THEIR job to determine how to code correctly, it's yours.

So next time you have a chunk of work to do, make sure that these changes are part of it. If you are providing estimates, be sure to add 30% to your estimate for refactoring, if you are not providing estimates then just do the refactoring as part of your work.

It may make you slower--well no, that's not the way to look at it, the way to look at it is that your current velocity is an illusion, essentially a lie that you are passing up the chain, you actually should be a little slower because of this work that you know needs to be done.

You could probably build houses more quickly if you didn't use concrete as a foundation--and they would look as good to the customer but--well--Even if the customer doesn't see the need for the foundation, the builder needs to. (This is actually an interesting parallel because it turns out that builders don't always do what they know they should do so we need to pass laws to force them to--there are no such laws governing software development even though we face the same decisions and often make the wrong ones...)

Answer 8

You mention that you are fortunate enough that your manager and CEO are both programmers. So they probably do understand what technical debt is.

You should handle the situation by trying to resolve the situation based on facts, which means there is a real possibility that you won't end up making the technical improvements you want (facts can be annoying that way).

Your job is to make sure they understand the costs and benefits of paying off any particular technical debt you incur. Their job is deciding if the best use of resources is in paying it off or in doing something else.

Just as it can be hard for people not involved with the code to see the benefits of improving the "hidden" stuff, it can be hard for programmers to see the benefits of visible code changes when the benefits accrue to areas of the business somewhat "hidden" from the developers.

It's nice if your management can explain to you why the visible features are more worth your time than paying the technical debt, but really, it's not your job to make the determination anyway. So explaining it to you doesn't do much for the business except keep you happy. And in a way, insisting they do so is not trusting them to do their job. If you don't like it when they micro-manage you then you should understand.

So, as long as you are keeping them aware of the status of all technical debt and the costs and benefits of ignoring it versus paying it off, you have done your job. Unless you really don't trust management to do theirs, in which case you have a much bigger problem that would be a whole other question to address.

Answer 9

Maybe this isn't an answer, but a response based on mistakes I've made. Also a disagreement with some of the philosophies I read here.

• Don't fall afoul of the belief that the programmer knows best.
• Be honest. Re-factor as you go along but don't lie and pad estimates for your own purposes.
• You don't own the code. Don't undertake work that's not approved by the lead.
• You could be right about something; you could be wrong... but you should do what you're paid to do.

But certainly follow the excellent advice given on improving things.

Answer 10

You obviously work for a pointy-haired boss (PHB). He doesn't program anymore, if ever, and if he did he probably wasn't really any good (although he does like to drop in phrases like "const correctness" and "segmentation fault" so that the guys know he's earned his stripes) - that's how he got singled out for management.

The CEO (who practically has a Mohawk) likes the PHB because the PHB delivers features. Every sprint he proudly demonstrates a new tick-box (slightly misaligned, with an ambiguous label), a sparkly icon (not yet working in any environment but 8-bit color over Citrix) and a form (that has random crashes due to race conditions in the bespoke xml variant based C implemented local database that no one on the dev team dare touch because it was written by one of the old guard dev legends 10 years ago and does EVERYTHING with macros with 5 letter names, whether they needed 5 letters or not).

The programmers who actually do the "work bit" (you know that bit that happens, inconveniently after all the real work like drawing circles on whiteboards, shouting, and eating miniature Battenburgs is done) know that in a sane system the work that just took 10 guys 10 days to laboriously hack out of the unmaintained jungle, would probably amount to one or two man days, including the testing. But to get the system from where it is to sane might take 6 months of genuinely hard work, with little in the way of obvious external reward.

The PHB knows that in 6 months, by hook or by crook, he can get thirty or forty new features into the application that the salespeople (who must be magicians given what they're actually selling) can be used to tempt new purchases and upgrades.

However to give the PHB his dues: without those tick-boxes and forms sales may well stagnate or decline, a competitor may gain market share, and that might be worse than the alternative.

The conclusion that I've come to is that the only way out of the quagmire is to work in a new start-up, with a few people who share your vision of how software should be done and then doggedly stick to that philosophy (I'm still working on getting there!)

Answer 11

There is another hidden cost not mentioned in other answers. Namely that good engineers tend to leave in the type of environment described. Eventually anyone with the ambition or ability to refactor has left the company, and then things will be in a very bad way, probably unfixable. Unfortunately you can't tell your manager this without coming across as arrogant ("I'm one of your best programmers"), and a pushy jerk ("I'll leave if you don't do what I want"). However, do remember to mention it in your exit interview, to ensure you're put on no re-hire status.

Answer 12

You are both right and both wrong, that's why those issues stick around for long time and create hard feelings.

The reasons why are clearly stated above: management thinks in money; or even more specifically: revenue. To them something which has a cost but no visibility directly to the customer doesn't add revenue so it's a bad plan.

Your vision is also right: it will be good for customers but on longer term. Your actions might generate an even bigger revenue in the future compared to the short term plans.

You are not the manager, you are not the one directly responsible for the revenue (I assume of course). So you are mixing up (that's how it feels for management) with their issues and you are not focussing on your job: further expanding the software.

All hard, clear words but that's how most issues arise, out of communication errors. You both want the same thing but your short term decisions are made differently. All because the developer has a different outlook compared to the manager.

How do you solve this issue? You communicate and understand each other quite well on a number of issues. That will be the focus on customer engagement and satisfaction most likely.

To solve this issue in a stable and future proof method agree on some thing: measure the cost of bugfixing in old code. Measure the time it takes additionally to work with the old software and how it would be with the new codebase.

To prove this you could do for example this quite simple thing: branch the software in your versioning. First do what management wants, so create that feature. You do this but the agreement is you get time to show the different way. Then do the same thing in the new branche but first fix the issues you want to get rid off. Then also develop the solution.

Now you have the same solution but totally developed differently. Create a case from it, put the solutions next to each other including some management info like stability, amount of code needed and the code itself.

Now grab together a coffee and start taking a look, not debating who is right but what would be the influence of both directions for the company. That will create a meeting which is really useful and not a better-worse discussion because that won't generate the atmosphere you both will need. That won't make your product better.

Answer 13

If you have a code review , create a technical ticket stating that the code should be improved by using ARC and assign to the manager.

Convince him. Explain him some small examples of similar technical enhancements that you did and its benefits to the projects.

Answer 14

You have to sell those changes in the only way that management is willing to buy:

Find a user-facing feature so compelling that management just has to have it, but would be impossible to do without some low-level improvements in the code.

Answer 15

It's your boss's job to make sure the company focuses development on delivering what clients perceive as added value. There are two factors that can alter this perception:

1. How long does it take to deliver on a client request?
2. Do you deliver when you say you will?

Most would rather you say it will take 6 weeks and deliver in 5 than say you will deliver in 3 but take 4.

Having a current code base that is easier to manage and enhance allows you to deliver faster and increase predictability. If you're spendning too much time on bug fixes, you have fewer resources available to add new features. Evaluate the amount of resources spent on fixing the code and how accurate you are on feature promisses. This one way to determine if your current code (under your boss's philosophy) is costing too much.

Answer 16

Let me tell you about a $2 billion opportunity that nearly slipped through our hands because of management inertia.

Some of us have a point of listening to the voice of the customer, meaning understanding what he wants. It's not always what he asks for, but if you are in a position to have conversations with your customer, you eventually come to a mutual understanding of what he wants. (Then he can explicitly start asking for it.)

That being said, its important to understand who should be having that conversation with the customer. You generally have your business development folks along with some principal designers doing this. If you have any competition, you can expect that the customer is having this conversation with them also.

At the same time, it's important the the developers keep current in their fields, because there will be competition that will beat you if you aren't.

In our situation, we had an existing customer, and a somewhat effective product based on old technology, and the customer needed rapid upgrades. What they really needed was a complete replacement product, but the mindset at our company was that this would immediately force us into a competitive position, whereas modifying the existing product allowed our customer to continue with us without legally being required to make it competitive. Our management thought they owned this market. The problem was they couldn't keep up with the upgrade requests, because the existing product structure was not flexible enough to upgrade.

The customer became impatient and started having conversations with competing sources. We lost our "ownership" of that market and a couple billion dollars in revenue. However, once the market forced us into a competitive position, management had no choice but to either go out of business or compete. (Most of the ones who were roadblocks eventually were fired.) We competed, and were able to recapture the business by the thinest thread.

I said at the start that this opportunity nearly slipped through our hands. We got the business back, for the revenue I mentioned. If we had been more prepared to adapt to the customer's concerns in the beginning, there wouldn't have been any real competition.

The take-away I am offering is this: Always try to stay leading edge in your personal capabilities. Always develop a product that is flexible and can be adapted quickly to what your customer needs. If you work too long for a company that doesn't think like this, you will wither, and your personal motivation will diminish. I have seen it happen.

When you have an idea to improve your product for whatever reason, ask yourself these questions: -- Is this what I think the customer wants? Can I validate my thoughts about the product development against the voice of the customer? Is my company in a position to address the customer needs? If so, how? -- You should then be a in position to make a persuasive case regarding your product development proposals.

Answer 17

The common language of business across all fields and industries is money. The problem that you are describing is not a programming problem: it is a business problem. If you want to get an appropriate response to a business proposition you need to frame it in the language of business. This means that you must be able to present an alternate project plan including all of the costs and benefits.

You need to learn about things like opportunity costs, ROI (return on investment), and NPV (net present value). You don’t need an MBA to do this, but you do need access to data that may not be available to you, such as manpower costs, overhead costs, and potential for revenues. If you can make a clear argument that one path will provide more value than another using hard numbers, you will get full attention from management.

Answer 18

When I was a newer developer on a very small team, I did a lot of this kind of improvement in my free time. I enjoyed it; I would log on and try out interesting new techniques while sitting in the living room with my wife at night. As I got to be a more senior developer and then manager of a bigger team, my free time disappeared. We were working extra hours just to get the features and critical fixes done. It became really hard to justify spending anyone's time on that regular housekeeping work, even though we all knew how important it was.

Your bosses don't need an explanation of how important it is, to keep maintenance costs down, reduce the cost of future growth, keep a talented development team engaged, etc. If they do, you have big problems. What they might need, however, is a plan. Because right now I'm guessing that they have all sorts of plans, schedules, roadmaps, promises, and pressure on the "add functionality" side, which are competing against mere wishful thinking and open-ended requests from the developer team.

One thing you could try is making a documented plan. See if you can tie it to releases, or to exit criteria for new functionality. A request to "spend some time redoing the reference counting" might be tough for your boss to approve, but scheduling a 5-day block of time 4 weeks from now might be easier. Basically, however you see that new features are planned and scheduled, try to mimic that, or inject a "under-the-hood" portion into it.

Start small by asking for 5% time allocated to that type of stuff, and then gradually build up your own technology roadmap priorities, and keep plugging away at justifying the business case, ROI, risk levels, etc. on each of them. Pretty soon you'll even get a taste of your bosses' challenges, as you'll quickly find many more great ideas than you have time to do.

Good luck!

Answer 19

Here's why your request for automatic reference counting is being rejected:

1. It's not an improvement. Once you have anything larger than hello world, any change is going to wrong direction. No amount of refactoring is going to change the fact that if you need to do changes, it's always going to wrong direction. The trick is to know when your changes are significant enough that it is still worth the risk of causing new issues. Your management have clearly said that if end users cannot see it, it's not worth the risk.
2. Reference counting is completely crazy feature. You saw some existing big famous company implement some feature, and immediately thought that you need the same feature. Well, it's very likely your codebase is completely different from what apple is using. Probably reference counting is not going to work, or it's just waste of time. You should find a different way which does not require spreading reference counting primitives everywhere in your code. Probably your refcounting plan requires thousands of modifications in the codebase, most of those changes are not necessary. Just waste of time and money.
3. You're solving the wrong problem. Management usually knows what kind of problems there are in the system. Some irrelevant memory leak problem that the refcounting solution is solving is not one of them. Focus on real problems and not some imaginary problems with how computers handle memory.
4. It takes too long time to implement it Apple is slightly bigger company than your insignificant team with few programmers and some managers. They can do big changes by just paying the price. If it takes few millions to implement something, it's peanuts. If small companies tries to do the same thing, they're going to run out of money fast. Software development is already very expensive; adding unnecessary costs is not going to help one bit. One irrelevant feature like memory management is going to open the floodgates: after they approve it, half your programmers want their own "improvement" to be implemented. It's neverending story and the company could be doing something useful instead.

Here's some steps you can use to handle the problem:

1. Drop the feature
2. Find out what is the real problem
3. Start doing something useful
4. Carefully plan how you use your time
5. Find out how your improvements are bringing in money
6. Choose only the features that bring in most amount of money
7. Realize that the programming aspect of software development is only small part of the whole system -- improvements to it is never going to be worth the time