Is micro-optimisation important when coding?

Question

I recently asked a question on Stack Overflow to find out why isset() was faster than strlen() in PHP. This raised questions around the importance of readable code and whether performance improvements of micro-seconds in code were worth even considering.

My father is a retired programmer, and I showed him the responses. He was absolutely certain that if a coder does not consider performance in their code even at the micro level, they are not good programmers.

I'm not so sure - perhaps the increase in computing power means we no longer have to consider these kind of micro-performance improvements? Perhaps this kind of considering is up to the people who write the actual language code? (of PHP in the above case).

The environmental factors could be important - the Internet consumes 10% of the world's energy. I wonder how wasteful a few micro-seconds of code is when replicated trillions of times on millions of websites?

I'd like to know answers preferably based on facts about programming.

Is micro-optimisation important when coding?

My personal summary of 25 answers, thanks to all.

Sometimes we need to really worry about micro-optimisations, but only in very rare circumstances. Reliability and readability are far more important in the majority of cases. However, considering micro-optimisation from time to time doesn't hurt. A basic understanding can help us not to make obvious bad choices when coding such as

if (expensiveFunction() || counter < X)

Should be

if (counter < X || expensiveFunction())

(Example from @zidarsk8) This could be an inexpensive function and therefore changing the code would be micro-optimisation. But, with a basic understanding, you would not have to, because you would write it correctly in the first place.

Answer 1

I both agree and disagree with your father. Performance should be thought about early, but micro-optimization should only be thought about early if you actually know that a high percent of time will be spent in small CPU-bound sections of code.

The problem with micro-optimization is that it is usually done without having any concept of how programs actually spend more time than necessary.

This knowledge comes from experience doing performance tuning, as in this example, in which a seemingly straightforward program, with no obvious inefficiencies, is taken through a series of diagnosis and speedup steps, until it is 43 times faster than at the beginning.

What it shows is that you cannot really guess or intuit where the problems will be. If you perform diagnosis, which in my case is random-pausing, lines of code responsible for a significant fraction of time are preferentially exposed. If you look at those, you may find substitute code, and thereby reduce overall time by roughly that fraction.

Other things you didn't fix still take as much time as they did before, but since the overall time has been reduced, those things now take a larger fraction, so if you do it all again, that fraction can also be eliminated. If you keep doing this over multiple iterations, that's how you can get massive speedups, without ever necessarily having done any micro-optimization.

After that kind of experience, when you approach new programming problems, you come to recognize the design approaches that initially lead to such inefficiencies. In my experience, it comes from over-design of data structure, non-normalized data structure, massive reliance on notifications, that sort of thing.

Answer 2

Micro-optimization is only important if the numbers say it is.

The requirements that you are developing against should have some specification of performance data, if performance is at all an issue to the client or user. As you are developing software, you should test the performance of your system against these requirements. If you aren't meeting performance requirements, then you need to profile your code base and optimize as needed to reach the performance requirements. Once you are within the minimum required performance, there's no need to squeeze any more performance out of the system, especially if you are going to compromise the readability and maintainability of the code any more (in my experience, highly optimized code is less readable and maintainable, but not always). If you can get additional performance gains without degrading the other quality attributes of the system, you can consider it.

There are instances, however, where performance is of the utmost importance. I'm mainly thinking in real-time systems and embedded systems. In real-time systems, changing the order of operations can have a huge effect on meeting the hard-deadlines that are required for proper operation, perhaps even influencing the results of computations. In embedded systems, you typically have limited memory, CPU power, and power (as in battery power), so you need to reduce the size of your binaries and reduce the number of computations to maximize system life.

Answer 3

Whenever anyone asks about optimisation, I'm reminded of the quote from Michael A. Jackson

The First Rule of Program Optimisation: Don't do it.

The Second Rule of Program Optimisation (for experts only!): Don't do it yet.

^{Quote from Wikipedia corrected for British English. *8')}

Implicit in the second rule is to profile your code and only spend time optimising things which will make a difference.

When programming in assembly, your father's assertion was correct. But that is much closer to the metal than most people program these days. Today even trying to optimise yourself (without profiling) can result in making your code run slower than if you'd done something the more common way, since the common way is more likely to be optimised by modern JIT compilers.

Even in C you have to be pretty good at optimisation to know more about how to optimise a section of code than the compiler does.

If you aren't familiar with much of what Ulrich Drepper talks about in his excellent paper What Every Programmer Should Know About Memory, then you are probably on a loser even trying to optimise youself. Contrary to the papers title however (which is actually an homage to David Goldberg’s equally fantastic What Every Computer Scientist Should Know About Floating-Point Arithmetic) not having this level of understanding does not necessarily stop you being a good programmer, just a different sort of programmer.

isset() vs. strlen() in PHP

I don't know PHP, so it really isn't obvious what isset() is meant to do. I can infer it from context, but this means that it will be similarly obscure to novice PHP programmers and might even cause more experienced programmers to double-take. This is the sort of idiomatic use of a language which can make maintenance a nightmare.

Not only that, but there is no guarantee that isset() will always be more efficient, just because it is more efficient now. An optimisation now might not be an optimisation next year, or in 10 years. CPUs, systems, compilers improve and change over time. If the performance of PHP's strlen() is a problem, PHP might get modified in the future, then all of the isset() optimisations may need to be removed to optimise the code once more.

Every optimisation has the potential to become a future anti-optimisation, so should be considered a possible code smell, to be kept to a minimum.

An example from personal experience

As an example of such an anti-optimisation, I once had to sanitise a huge codebase filled with code of the form if x==0 z=0 else z=x*y because someone had made the assumption that a floating point multiply was always going to be more expensive than a branch. On the original target architecture this optimisation made the code run an order of magnitude faster, but times change.

When we tried using this code on a more modern CPU which was highly pipelined the performance was horrifically bad - every one of these statements cause a pipeline flush. Simplifying all of these lines of code to just z=x*y made the program run an order of magnitude faster on the new architecture, restoring the performance lost by the anti-optimisation.

The problems we typically have to optimise for today are very different to the ones we had to optimise for 20 or even 10 years ago.

Then we worried about doing the most work per clock cycle, now we are more likely to be worrying about pipeline flushes, branch mispredictions and cache misses at the CPU level, but locks and interprocess communication are becoming much more significant as we move to multi-process and multi-processor architectures. Drepper's paper can really help with understanding many of these issues.

Answer 4

1. Write code, that is clean, concise, simple and self-explanatory.
2. Make performance tests to identify bottlenecks.
3. Optimize critical sections.

As a rule of thumb: 95% of your code is run 5% of the time. There's no point in optimizing before you profile/benchmark your code and see which are the 5% that are run 95% of the time.

Every idiot can do micro-optimization and any decent compiler/runtime will do this for you.
Optimizing good code is trivial. Writing optimized code and trying to make it good after that is tedious at best and unsustainable at worst.

If you seriously care for performance, then do not use PHP for performance critical code. Find the bottlenecks and rewrite them with C extensions. Even micro-optimizing your PHP code beyond the point of obfuscation won't get you nearly as much speed.

Personally, I liked doing micro-optimization a lot when I started programming. Because it is obvious. Because it rewards you quickly. Because you don't have to take important decisions. It's the perfect means of procrastination. It allows you to run away from the really important parts of software development: The design of scalable, flexible, extensible, robust systems.

Answer 5

I would agree with your father: "If a coder does not consider performance in their code even at the micro level, they are not good programmers." The key is to "consider performance". That is not equivalent to "do micro-optimizations at every step".

I agree with most of the other comments - what used to make C programs faster may not do so today - but there are other serious things to consider: Should I use C or C++? Classes with simple overloaded operators can kill performance if you use them a lot. Does that matter? It depends on your application, but if you don't even CONSIDER it, you're not a very good programmer. Some folks might consider it for about 2 milliseconds and dismiss it, but I think far too many literally don't even consider it.

The reality is that with optimization, code can get harder to read. Code will take longer to write. Code will be somewhere between a little faster and an order of magnitude faster (that's rare). Your customers probably won't know the difference.

Another reality is that people like to reuse code, and where it ends up may be quite different than when you wrote it. Suddenly people may care.

As an example, say you wrote something like Angry Birds on a PC, or for a game console. You disregarded optimization in your physics system and your graphics system - because 2.5 GHz dual cores are basically the minimum these days, and your game works good enough. Then smart phones come along and you want to port it. Oh darn, a 600 MHz ARM core?

Think of a web site - something thrown together in a weekend like Hot or Not. You use whatever database is handy, write everything with rapid development in mind, launch by emailing your friends a URL. Three months later your server is dying from overload and there's nothing you can do to scale up. It happens all the time. The largest web sites have power bills measured in hundreds of kilowatts or even megawatts. Think about that, there are megawatts to be saved through code optimization.

Like your father said, you must at least consider it. Most folks don't even know how much performance is on the table and gloss over it with a quick quip about "premature optimization" and "don't do it". These are not good programmers.

This is not a popular opinion on these sites. Bye bye reputation points....

Answer 6

Let's first take your case: PHP

• I don't think PHP is a kind of language (both because of its nature and its main domain of application) that you need to worry about these "micro-optimizations". PHP code is mostly optimized with opcode caching.
• The programs written in PHP are not CPU bound, they're mostly I/O bound, so these optimizations won't be worth your time anyway.
• Anything that you MUST optimize should probably be snuck into a C extension and then dynamically loaded in the PHP runtime
• So as you can see, we get no incentive by micro-optimizing our code in PHP -- on the other hand, if you spend that time in making PHP code more readable and maintainable -- that will pay you more dividends.

In general,

I wouldn't spend "TOOOOO" much time optimizing my code in a dynamic language like Python or Ruby -- because, they are NOT designed for CPU intensive number-crunching tasks. They solve different class of problems where modelling a real world situation in an elaborate manner (which is easy to read and maintain) -- which is called expressivity -- is more important than speed. If speed were the prime concern, they wouldn't be dynamic in the first place.

For compiled programms (C++, Java), optimization is more crucial. But there too, first you have to look at the nature/domain/purpose of the program you're writing. You should also carefully weigh the time to micro-optimize against the gains from that optimization. If you need even more optimization, then you may as well consider going a step downward -- and code those pieces of your code in assembler.

So, to answer your original question -- "Is micro-optimisation important when coding?" -- the answer is -- it DEPENDS --

• What kind of thing you're doing: application domain, complexity?
• Are microsecond speed improvements REALLY important for your program?
• What sort of optimization will be most gainful? It may not always be code optimization, but something external.
• How much "goodness" (in terms of speed) you're reaping out of the time you invest micro-optimizing?
• Can better speeds be achieved in other ways -- by changing the hardware, RAM & processor, executing code parallelly, or on a distributed system?

Not only is micro-optimization (code optimization for that matter) time consuming, it "distorts" the natural readability of your code, thus making it maintenance-heavy. Always consider it a last resort -- always try to optimize the entire application by adopting better hardware and better architecture than optimizing individual code files.

Answer 7

There seem to be many answers that say micro optimization is

all about tradeoffs - performance, time, cost, effort, readability/maintainability, and so on.

But well I know that there are some optimizations where there is no readability impact, and I just like to do those for fun. I saw on some of my school assignments (which were all speed based), that it's always nice to consider micro optimization when writing conditional statements like:

if (counter < X && shouldDoThis()) // shouldDoThis() is an expensive function

is always nicer than

if (shouldDoThis() && counter < X ) 

There are quite a few ways to "speed up" your code like that and the difference is usually negligible (not always though), but I feel better if I write it like that.

I don't know if anyone even considers this as an actually optimization, but I think a programmer should know and consider these things when writing their code.

Answer 8

Early in my career, blanket statements such as, "don't micro-optimize" led to a lot of confusion. Everything is circumstantial; thus, the reason people say, "best practice" rather than "do this".

"Best practice" is the top choice considering all circumstances. For example, LINQ and Entity Framework should be used in lieu of inline SQL. At my company we are on SQL Server 2000. SQL Server 2000 does not support Entity Framework. Best practices require:

• Selling my boss on the idea of buying a new version of SQL Server which means several thousand bucks.
• Selling developers on the idea of learning new technology.

I know from experience this is not going to happen. So, I could quit, stress to no end or not follow the best practice.

There are political, technical and monetary considerations behind decisions that affect the overall outcome. Remember this fact when making a decision and choose your battles wisely.

"For everything there is a season, a time for every activity under the heaven."

Answer 9

This is always a tradeoff.

First, the computer industry is about money at the end. What you need to do as a developer is generate value for customer so you get money (that's an oversimplification but the main point is here).

Developer time costs money. Machine power costs money too. Usually, this second cost is way lower than the first one. So, this is capital to have a readable code and maintainable code, so developer can spend most of their time delivering value.

Micro-optimization can, in some case, be important. But they usually involve less readable code, or less extendable code (this is not the case of your linked example, but in general, it is). This will cost at some point developer's time. This time being more expensive than machine power, this is a waste.

Second, micro-optimization in a large project can make it harder and harder to maintain/evolve. The problem with that is that when evolving, some other optimisation may be now impossible to do. With an evolving application, you'll typically end up with a solution that is slower than what you would have had without doing those optimizations.

Third, the optimization is often irrevelant as algorithm complexity will generally overcome any micro-optimization you could have done if the data set grows. Sadly, as micro-optimization make your code harder to maintain/evolve, thoses optimisation may be harder to do.

Sometime, the value is in this optimization (think about latency critical programs, like in video games or an aircraft's autopilot). But this has to be demonstrated. Usually your program spend most of the time in a limited portion of code. Whatever micro-optimization you do, you'll never get your programs any valuably faster without identifying the bottleneck and work on this part.

Asking your question as you did showed that you didn't benchmark the problem on an actual program. In this case, you could have do the trick and noticed if it was faster or not. So you were asking that before having any problem. This is where the problem is. You were handling the problem of optimization the wrong way.

As maintenance and evolution are usually more valuable than micro-optimization, be sure to have the right interface before doing any. Then if parts of your program are abstract enough for one another, you can micro-optimize one without messing up the whole thing. This requires that your interface is running for long enough to be trusted.

Answer 10

Performance is a Feature

Jeff Atwood's article is a great one on building a high performance website and the importance of doing so...

That said, don't focus on micro-optimization until you need to. There are more cost beneficial optimization you can perform. Focus on the architecture, not the code. Most web sites I have seen that have performed slowly had high level problems (unnecessary web service layer, poor database design, overly complicated architectures) that not only adversely affected performance but were deeply ingrained and difficult to fix.

When you're building a website, your client side code and database logic is far more likely to cause performance problems than your server side code. Like anything though, if you have performance problems you'll know, even better profile your code and you can find them early on.

Answer 11

Developer time costs more than computer time. That's usually what you want to optimize. But:

• There's a difference between micro-optimization and algorithmic complexity. Spend enough time to be confident that you are using the right algorithm.
• Make sure you're asking the right question, select (select count(*) from foo) >= 1 is not the same thing as select exists(select 1 from foo).
• some language idioms are popular simply because they are faster, It's ok to use those, since most fluent users of the language will be familiar with them. (your example is a good example).

Answer 12

What do you want to optimize?

• Software performance?
• Reliability?
• Programmer productivity?
• Customer satisfaction?
• Power efficiency?
• Maintainability?
• Time to market?
• Cost?

"Optimize" doesn't always mean making the code run as fast as possible. There are times when finding the absolute fastest way to do something is important, but that's really not all that common in most code. If users can't notice the difference between 50 and 100 microseconds, there's effectively no difference between the two in code that will only run occasionally. Here's an example:

If you need to continually update a display of the length of the user's input and the amount of time that it takes either of two routines to determine that length is much smaller than the time between two consecutive keystrokes, it really doesn't matter which routine you use. On the other hand, if you need to determine the length of a billion strings, you'll probably want to pay close attention to performance differences between different routines. In the first case, you might prefer to write code that's easy to understand and verify; in the second case, you might be willing to trade readability for speed.

In any case, if you're going to optimize your code, you should profile your code before and after any changes you make. Programs these days are complicated enough that it's often hard to tell where the bottlenecks are; profiling helps you optimize the right code and then show that the optimizations you made really worked.

You didn't say when your father retired or what kind of programming he did, but his reaction isn't surprising. Historically, memory, secondary storage, and computing time were all expensive, and sometimes very expensive. These days, all of those things have become very cheap compared to programmer time. At the same time, processors and compilers have become able to optimize code in ways that programmers could never match. The days when programmers use little tricks to bum a few machine instructions here and there are mostly gone.

Answer 13

It is not important to micro optimize while writing the code. Optimization should be done with the help of a profiler, optimizing the code where it matters.

HOWEVER, a programmer should try to avoid doing obviously stupid things while writing the code.

For example, don't do repeated expensive operations inside a loop. Store the value in a variable outside the loop and use that. Don't do things like string comparisons or regex over and over again in an often called function, when you can go up a level, do the comparison and make it into an integer or a function reference or a derived class.

These things are easy for an experienced programmer to remember and almost always improve the code quality.

Answer 14

When deciding what to optimize, always remember Amdahl's law. See the link for precise math; the pithy statement to remember is:

If one part of your program accounts for 10% of its runtime, and you optimize that part to run twice as fast, the program as a whole will only speed up by 5%.

This is why people always say it's not worth optimizing parts of your program that don't occupy more than a few percent of total runtime. But that's just a special case of a more general principle. Amdahl's law tells you that if you need to make the entire program run twice as fast, you need to speed up every piece by an average of 50%. It tells you that if you need to process twenty gigabytes of data, there are only two ways to make that go faster than the time it takes to read twenty gigabytes off the disk: get a faster disk, or make the data smaller.

So what does Amdahl's law say about micro-optimizations? It says that they're maybe worth it if they apply across the board. If you can shave one percent off the runtime of every function in your program, congratulations! You've sped up the program by one percent. Was that worth doing? Well, as a compiler guy I would be delighted to find an optimization that did that, but if you're doing it by hand, I'd say look for something bigger.

Answer 15

It depends upon what stage of development you are at, when initially starting to write something, micro-optimizations should not be a consideration as you are going to get more performance gains by using good algorithms than you are by using micro optimizations. Also, consider what you are developing as time sensitive applications are going to see more benefits from micro optimization considerations than generic business applications.

If you are testing and extending software then micro-optimizations will likely hurt you they tend to make code harder to read and even introduce their own unique set of bugs that need to be fixed along with anything else that needs to be fixed.

If you are actually getting complaints from users about slow code then they might be worth considering but only if everything else has been resolved, namely:

• Is the code well written?
• Is the application able to access its data without any issues?
• Can a better algorithm be used?

If those questions have all be answered and you are still having performance issues, then it might be time to start using micro-optimizations in the code but odds are that other changes (i.e. better code, better algorithm, etc) is going to net you more of a performance gain than a micro-optimization will.

Answer 16

Execution speed is one of many factors that contribute to the quality of a program. Often times, speed has an inverse correlation with readability/maintainability. In almost all cases, code must be human readable so that the code can be maintained. The only time that readability can be compromised is when speed is an essential requirement. The requirement to make code faster than full readability/maintainability allows is hardly ever applicable, but there are certain cases where it will. The main thing to remember is that micro-optimized code is often hacky code, so unless there is a defined requirement somewhere, it is almost always the wrong way to solve the problem. For example, the user will almost never notice the difference between .5 second and 1 second execution times in CRUD operations, so you do not need to go into an assembly-interop-hackfest to get to that .5 seconds. Yes, I could fly a helicopter to work and it would be 10 times as fast, but I do not because of the price and the fact that helicopter is much more difficult to fly. When you micro-optimize code needlessly, this is exactly what you are doing: adding needless complexity and cost to reach a superfluous goal.

Answer 17

Micro-optimisation is important when you hit a constraint. The thing you care about might be memory, might be throughput, might be latency, or might be power consumption. Note that these are system-level characteristics; you don't need to (and can't) optimise every function in every way.

Embedded systems are more likely to need micro-optimisation because constraints get hit more easily. However, even there micro-optimisation only gets you so far; you can't micro-optimise your way out of a bad design. The point about good design in a system is that you can reason about the system as a whole. Components that needs micro-optimisation should be neatly exposed and optimised in a way that doesn't compromise the design of the system.

Note that small "embedded" systems today can be pretty close to the Vaxen or PDP-11s of yesteryear, so these issues used to be more common. On a modern general purpose system doing modern general commercial computing, micro-optimisation is rare. This is probably part of why your father takes the position he does.

However, it doesn't matter whether you are dealing with nanoseconds, milliseconds, seconds or hours; the issues are the same. They have to be evaluated in the context of the system and what you are trying to achieve.

This is an example from a recent question I answered on Stack Overflow for a case where micro-optimisation was necessary: Open source video encoders for an embedded system.

Answer 18

The greatest problem with micro-optimization is that it leads you to write a code harder to maintain.

Another issue is depending on the computer configuration sometimes your micro-optimization might have a worst performance than without the 'optimization'.

Making many micro-optimization will consume you a lot of time fighting against something that doesn't really matters.

A better approach is by making a cleaner code, easier to maintain, and if you get performance issues you run a profile to figure out what really is making your code slow. And knowing exactly what is really bad you can fix it.

I'm not saying that not doing micro-optimizations is excuse for writing stupid code.

Answer 19

If you start to worry about milliseconds, you should consider to give up PHP and use C or Assembly instead. Not that I would want to do this, it just makes no sense to discuss such numbers and use a scripting language. Does your code iterate with this command that often anyway?

The environmental factors are out of question here, those servers run 24/7 anyway and if they actually process something would only matter if it's really a task running for a very long time.

Most likely the light in your office and the energy our computers used while we all were typing questions and answers, used up far more energy than any kind of micro optimization you can reasonably apply to your applications will ever save.

Answer 20

You should choose the best, simple algorithm for the task. The reason it needs to be simple is to keep the code readable. The reason it needs to be the best, is to avoid starting out with bad runtime characteristics. Do not blindly choose BubbleSort when you know that you will have large datasets. It is fine, however, for the occasional sort of 10 elements.

THEN, if the profiling numbers show that your choice of best, simple algorithm wasn't good enough, you can start optimization (which is usually as the cost of readability).

Answer 21

I've said it before, and I'll say it here: "Premature optimization is the root of all evil". This should be one of the rules at the center of any programmer's mind.

Code can, to a point, always be faster than it currently is. Unless you're hand-packing assembly with a particular chip in mind, there's always something to be gained through optimization. However, unless you WANT to be hand-packing assembly for everything you do, there has to be a quantitative goal, which once you meet, you say "that's enough" and stop optimizing, even if there is still a glaring performance-sucker staring you in the face.

Beautiful, elegant, extremely performant code is useless if it doesn't work (and by "work" I mean produce the expected output given all expected inputs). Therefore, producing code that works should ALWAYS be the first priority. After it works, you evaluate performance, and if it's lacking you look for ways to make it better, up to the point where it's good enough.

There are some things you do have to decide up front that will impact performance; very basic decisions like what language/runtime you will use to implement this solution. Many of these will impact performance by many orders of magnitude more than calling one method vs another. Honestly, PHP, as a scripted language, is already a performance hit, but as very few scripted sites are built from the bottom up in C/C++, it's comparable to other technologies you'd likely choose from (Java Servlets, ASP.NET, etc).

After that, I/O message size is your next-biggest performance killer. Optimizing what you read from and write to the hard disk, serial ports, network pipes, etc will usually improve the program's run time by many orders of magnitude even if the algorithms behind the I/O operations were efficient. After that, reduce the Big-O complexity of the algorithm itself, and then if you absolutely must, you can "micro-optimize" by choosing less expensive method calls and making other esoteric decisions at low levels.

Answer 22

You mention that your dad is a retired programmer. Programmers who worked in the mainframe world had to be very concerned about performance. I can remember studying a US Navy activity where their mainframe was hardware-constrained to 64 KB of memory per user. In that programming world you have to eek out every little tiny bit you could.

Things are vastly different now and most programmers don't need to worry so much about micro-optimizations. However, embedded systems programmers still do and database people still very much need to use optimzed code.

Answer 23

Code should be written to be absolutely clear about what it does. Then, if and only if it's too slow, go back and speed it up. Code can always be changed to be faster later, if it can be understood- but good luck changing it to be clear if it's fast.

Answer 24

It is important if:

1) Someone's life depends on your code. A function taking 25ms to execute in someone's heart rate monitor is probably a bad idea.

I personally take a two pronged approach -- there are micro-optimizations you can do that won't affect readability -- obviously you want to use those. But if it affects readability, hold off -- you won't get much benefit and it actually might take you longer to debug in the long haul.

Answer 25

Is micro-optimisation important when coding?

No, given that there are platforms like JVM and .NET where code is written for a virtual machine and thus trying to optimize execution may not work out so well as what is optimal on a developer's desktop isn't necessarily the same on a server. Look at how far removed from the hardware some of these pieces of high-level software are for another point here. Something to consider is given the diversity of hardware, how realistic is it to optimize code for specific chips like a CPU or GPU when a new model will probably come out in less than a year?

Another question here to consider is performance as measured by what metric: Speed of execution, memory used in execution, speed of development of new features, size of the code base on a server in compiled or de-compiled forms, scalability, maintainability, etc.? If taken broadly enough the question becomes trivial, but I'm not sure how broadly you meant to take performance as really that can be almost anything as long as it can be measured in some way.

---

Some micro-optimizations may work and some may not work which is where one can wonder how worthwhile is it to perform such work compared to other work that may be seen as much higher priority such as new features or fixing bugs. The other question would be whether or not an upgrade of hardware or software may break some of those optimizations, too.

Answer 26

I think that there is a big difference between good programing and micro-optimization.

If there are two ways to do the same task, one being faster than the other and both having the same readability, you should use the faster. Always. And this is good programing. There's no reason to not use a better algorithm to solve a problem. And even documenting it is easy: give the algorithm name, everybody will be able to google it and find more information about how it works.

And good algorithms are already optimized. They will be fast. They will be small. They will use the minimum required memory.

Even if using them your program still don't have that performance, them you can consider micro-optimizing it. And you'll have to really know the language to be able to micro-optimize.

And there's always room for spending some more money on hardware. Hardware is cheap, programmers are expensive. Don't spend too much time / money by optimizing when you can just buy hardware.

Answer 27

IMHO code-readability is more important than micro-optimization because in most part of cases micro-optimization does not worth it.

Article about non-sense micro-optimizations:

As most of us, I am tired to read blog posts about non-sense micro-optimizations like replacing print by echo, ++$i by $i++, or double quotes by single quotes. Why? Because 99.999999% of the time, it is irrelevant. Why? Because 99.99% of the time, you'd better install a PHP accelerator like APC, or add these missing indexes on your database columns, or try to avoid those 1000 database requests you have on the homepage.

print uses one more opcode because it actually returns something. We can conclude that echo is faster than print. But one opcode costs nothing, really nothing.

I have tried on a fresh WordPress installation. The script halts before it ends with a "Bus Error" on my laptop, but the number of opcodes was already at more than 2.3 millions. Enough said.

So in most cases micro-optimization saves 1 operation among of millions, but makes readability worse.

Answer 28

Other answers are right on the money. But I'll add another point where one have to differentiate what's premature optimization/micro-optimization and writing performant code that reflects an understanding of the behaviour of the language/framework constructs (sorry, couldn't find a one word for the last). The last one is a good coding practice and one should generally do!

I will explain. Bad optimization (read premature/micro optimization) is not when you optimize sections of code without profiling to know if they are really the bottlenecks. It's when you optimize based on your assumptions, hearsays and undocumented behaviours. If it's documented and does something in a more efficient/logical manner however small it is, I call it good optimization. As others have stated, both of these have cons and almost no pros at all as far as earning you good business is concerned, but still I do the latter, not the former, if it doesn't totally defeat readability. Yes readability/maintainability is of utmost importance and it's about where you draw the line.

I'll reiterate the points made by others here as the futility of both good and bad optimizations:

1. Your dependencies to a specific problem can change and any time spent on optimizing before completing the logical section of your application is a waste of time. I mean optimizing at a relatively early stage. Today you have an List<T> and by the time your app ships you had to change it to LinkedList<T> and now all the benchmarking was a waste of time and effort.

2. Mostly the real bottleneck of your application (read as measurable difference) could be 5% of your code (mostly the sql ones) and optimizing the other 95% doesn't give your customers any benefit.

3. Usually "technically" better performing code means more verbosity which in turn means more error prone code which in turn means harder maintainability and more time spent which in turn means you earn less money.

4. The carbon footprint you save for the whole world thru 1% performance gain is easily dwarfed by greenhouse gases your team will have to emit on debugging and maintaining that code.

The negatives of bad optimization in particular are:

1. It doesn't often given you the performance you expect. See this question on SO, where optimizations have gone wrong. In fact it can have adverse effects. That's the problem with undocumented behaviour.

2. Mostly modern compilers will do it for you anyway.

I'll give some examples of bad and good optimization:

The bad -

1. use of smaller integer types instead of Int32.

2. ++i used instead of i++

3. for instead of foreach (the worst I have seen, totally defeats logic)

4. avoiding closed out variables

   string p;
   foreach (var item in collection)
       p = ...;
   

5. using char instead of string during string concatenation, like:

   string me = 'i' + "me myself"; // something along that line - causes boxing
   

The good (from .NET world. Should be self-explanatory) -

1. Double lookup

   if (Dictionary<K, V>.TryGetValue(K, out V))
       do something with V
   

   instead of

   if (Dictionary<K, V>.ContainsKey(K))
       do something with Dictionary<K, V>[K]
   

2. Load 'em all

   DirectoryInfo.EnumerateFiles();
   

   instead of

   DirectoryInfo.GetFiles();
   

3. Two stage casting:

   s = o as string;
   if (s != null)
       proceed
   

   instead of

   if (o is string)
       s = (string)o;
   

4. If the order doesn't matter

   if (counter < X || expensiveFunction())
   

   instead of

   if (expensiveFunction() || counter < X)
   

5. Boxing

   void M<T>(T o) //avoids boxing
   {
   
   }
   

   instead of

   void M(object o)
   {
   
   }
   

If you ask me if these give a noticeable performance benefits, I would say no. But I would suggest one should use them because it stems from an understanding of the behaviour of these constructs. Why make two calls when you can do just 1? From a philosophical standpoint its good coding practice. And 1 & 3 are slightly less readable too in strict terms, but do they trump readability? No, not much, so I use. Now that's the key - maintaining a decent performance to readability ratio. And when it's that, its about where you draw the line.

Answer 29

"Worth it" needs context, like how much simpler it is to write and read and maintain vs. how much faster it makes something to the user markedly more responsive, interactive, requiring less time for them to wait.

Saving a few pennies to purchase a can of soda won't do me much good if I have to travel a distance to save those pennies, especially given that I rarely ever drink soda these days. Saving a few pennies per can on a purchase of a million can of sodas could be a huge deal.

Meanwhile saving a few pennies when two people are right next to me and one offers the exact same thing for a few pennies cheaper and the other doesn't, and I choose the more expensive one because I like their hat better seems like a foolish case of pessimization.

What I often find people calling "micro-optimizations" seem to be curiously devoid of measurements, and context, and user-end discussion, when there should absolutely be all three to consider such optimizations if they aren't trivial to apply. To me a proper micro-optimization these days relate to things like memory layouts and access patterns, and while they might seem "micro" in focus, they are not micro in effect.

I managed to, not so long ago, reduce an operation down from 24 secs to 25 milliseconds (about 960 times faster), with identical outputs (secured by automated tests), with no change to algorithmic complexity, for volumetric heat diffusion skinning, through "micro-optimizations" (the biggest of which came from a change in memory layout which got it down to about 2 seconds, then the rest were things like SIMD and further analysis of cache misses in VTune and some further rearrangement of memory layout).

Wolfire explains the technique here, and he struggled with the time required: http://blog.wolfire.com/2009/11/volumetric-heat-diffusion-skinning/

My implementation managed to be able to do that in milliseconds while he was struggling to get it down to less than a minute:

After I "micro-optimized" it down from 24 secs to 25ms, that was a game-changer in workflow. Now artists can change their rigs in realtime at over 30 FPS without waiting 24 secs every time they made any little change to their rig. And that actually changed the whole design of my software since I no longer needed progress bar and things of this sort, it just all became interactive. So that might be a "micro-optimization" in the sense that all of the improvements came without any improvement to algorithmic complexity, but it was a rather "mega-optimization" in effect that made what was formerly a painful, non-interactive process into a realtime, interactive one which completely changed the way users worked.

Measurement, User-End Requirements, Context

I really liked Robert's comment here and maybe I failed to make the point I wanted:

Well, c'mon. Nobody is going to argue that this kind of change isn't "worth it." You were able to demonstrate a tangible benefit; many so-called micro-optimizations cannot.

This is, in spite of working in a very performance-critical field with often real-time requirements, the only time I consider any micro-optimization which requires going out of my way.

And I'd emphasize not only measurements but the user-end side of it. I'm an oddball in that I came to my current field (and formerly gamedev) as a user/fan first, developer second. So I was never that excited by the usual things that excite programmers like solving technical puzzles; I found them a burden, but would bear through them through the user-end dream I shared with other users. But that helped me to make sure if I was optimizing anything, it'd have a real impact on the users with real benefits. It's my safeguard against micro-optimizing aimlessly.

That is actually as important as the profiler in my opinion, because I had colleagues who did things like micro-optimize subdivision of a cube into a billion facets only to choke on real-world production models like characters and vehicles. Their result was impressive in some "tech demo" sense, but almost useless to actual users, because they were profiling and measuring and benchmarking cases that didn't align with real-world use cases. So it's so important to understand what's important to users first, either by learning to think and use the software like one or collaborating with them (ideally both, but at least collaborate to them). The most important thing to me if we want to get micro and start shaving cycles and cache misses and so forth is to start becoming really good at prioritizing where we go out of way to do that (which naturally corresponds with knowing all the places we shouldn't do that, for which both profiler and user-end understanding are very important).

Answer 30

I'll put it this way - micro-optimisation is a process of optimising something that is not a bottleneck at all. For example, if your program calls two functions A and B, and A takes 100 milliseconds to complete and B takes 2 microseconds, and you keep optimising function B. That is not only not important, that is absolutely wrong. But optimising function B is called optimisation and not micro-optimisation. Importance of optimisation depends. Say you have nothing else to do and your program is bug-free, then yes, it is important. But generally you have priorities. Let's say you need to add/write function C. If you think that writing function C will make you more money than making your program faster without that functionality, then go for optimisation. Otherwise pursue functionality. Also, experienced programmers focused on performance don't spend much time optimising, they just write fast programs. Of at least they know what tools to use and what to do not to spend years doing meaningless (read micro) optimisations.