I've heard of some techniques to optimize code and make it faster:

  • On one side are clearly relevant optimization: use of better algorithms, benchmarking, etc.

  • On the other side are techniques with a more doubtful relevance. I don’t know if they are a myth or a reality or just deprecated usage.

My question is about the latter ones. Some examples:

  • Big function (several thousand lines) to avoid function call overhead
  • No SRP to avoid Object overhead
  • Reuse variable as much as possible instead of having scoped variables
  • ++i/i++
  • And many some other practices.

Often those techniques go against a readable, understandable and maintainable code base. So I'd like to know if they have a founded reason to exist.

A few things to note:

  • I'm only assuming there may be rules in place and not that those are only bad practices that took root in code and in developers minds.
  • I'm currently working in C++98 and accordingly old compiler and hopefully in c++14 more up to date compiler in the next future. I'd like to know if the answer depends of the environment

My question: Are those practices myths or reality? In the context of an application with heavy performance criteria (Like High Frequency Trading Software) could those practices supersede "good development" practices?

  • This question is very interesting. I dared to edit it in a way to remove some very subjective wordings, since they could have encouraged opinion based contributions, which are out of scope here. Please review and edit in cas I misunderstood something. – Christophe Jan 15 at 17:32
  • "And many some other practices." That makes the question too open ended to answer. Asking about a specific practice is fine; saying "there are some dubious practices that are too many to list; are there legitimate reasons for these practices that I haven't specified to exist?" is not answerable. – Nicol Bolas Jan 15 at 18:29
  • @Christophe Thanks for the edit, it's clearer. – JayZ Jan 16 at 8:10

Performance optimization doesn't lend itself to these kinds of generalized rules, and I'm not sure that the rules you proposed were ever good ways to optimize.

Here's a better plan:

  1. Set specific performance requirements. (i.e. define "good enough.")

  2. Start with well-written code that takes advantage of routine performance optimizations such as sensible choices for collection implementations.

  3. Find the hot spots in your code, the places where the processor is spending the most time, using a profiler. This code will represent less than 10 percent of your overall code.

  4. Optimize only the hot spots for better performance.

  5. Repeat steps 3 and 4 until performance requirements are achieved.

That's it.

If performance is of paramount importance, you can violate "best practices" to achieve it. But most of the time, you won't have to.

|improve this answer|||||
  • 2
    I think step one is to "define good enough performance". Optimization problems can force you to do things in a less than clean way, so if you pretty much are already fast enough you don't have to keep squeezing blood from a stone. – Berin Loritsch Jan 16 at 19:08
  • Sometimes, (or, where I work, always) "performance requirements" are vague - something like "it should be as slow as possible, but not so slow that your customer rejects your product". In absence of real requirements, step 5 in the answer above will be subjective. – anatolyg Jan 28 at 14:41

"Big function (several thousand lines) to avoid function call overhead"

In my experience exactly the opposite. Compilers tend to have problems creating good code for huge functions. I've had examples where execution time was really important to me, and I got considerable measurable speedup by extracting small loops into separate functions - code generated for the same time critical loop was better when it was in a separate function. (What actually happened was that a simple loop had all variables in registers, while the same loop inside a large function used stack memory for the same variables).

But for "High Frequency Trading Software" the key is fast communication with other devices, and that requires totally different techniques.

PS. I just find it a bit disturbing seeing all these suggestions that are not very likely to speed up your code, but make it a lot less readable and maintainable. What about a simple trick that any C++ developer should know: Say you have a standard vector containing instances of class T. Now compare these two loops:

for (T t in vector) {}
for (T& t in vector) {}

The first loop causes a copy of every single item in the vector to be constructed and destructed; two potentially expensive operations. The second loop does no such thing at all. That single character "&" can make a loop 100 times faster under the right circumstances.

|improve this answer|||||

Three quotes to sumarize good practices regarding optimizations:

  • Premature optimization is the root of all evil
    Donald Knuth
  • Don't diddle code, find better algorithms
  • Debugging is twice as hard as writing the code. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.

The rest is myth.

Some less important comments about your examples:

  • Not breaking down a thousand line function to avoid a couple of function calls puts the performance focus on 0.1% of the instructions: 99.1% of probability that performance issues are elsewhere IMHO.
  • Anyway, isn't it somewhat obsolete in a time of global optimizers and inlining? Does anyone remember the keyword register that was the summum of hand-crafted optimizations 30 years ago?
  • No SRP to avoid object overhead could be valid but only in case of a misunderstanding of SRP. SRP is according to Uncle Bob -- the inventor of the concept -- about (human) reasons to change, not about multiplying classes for decreasing the number of functions (which to the extreme would make OOP a variant of procedural programming). The benefits of real SRP far outweighs its drawbacks IMHO.
  • ++i; , i++; and i=i+1 taken isolated generate exactly the same code with a decently optimizing compiler. So ++i is no longer an optimization! But it's still valid from the point of view of expressivity. But this is a question of style: you like it or not. If not, maybe C++ is not the most suitable language for you ;-)
  • Many other human micro-optimizations are nowadays outperformed by the optimizer, which can take advantage of deep knowledge of the target CPU and analyse in-depth several alternatives. But finding better algorithm is still for us. I hope that we will outperform AI in this area for a long time (although some day I wonder).
|improve this answer|||||
  • 1
    ++i and i++ should generally produce the exact same code with a decent optimising compiler. There are cases where that would need more than decent compilers though, up to mind-reading, if i is just complex enough. – Deduplicator Jan 15 at 19:15
  • Even the Donald Knuth quote has become somewhat of a myth. – Robert Harvey Jan 15 at 20:13
  • Did you mean to imply that we should look for a ++C language if we don't like the C++ style? – Bergi Jan 16 at 1:23
  • 1
    @Bergi you made my day :’-) Well, may be there’s a c+1 language out there? more seriously, what I wanted to say is that pre and post-increment belong to the C/C++ culture and idioms so much that it appears even in the name. – Christophe Jan 16 at 6:38
  • 1
    @JimmyJames My statement shall not be misunderstood as a generalization: I only meant to criticise people whi claim that ++i should be favored over i=i+1 for performance reasons. – Christophe Jan 16 at 16:27

Not specific to C++ but one of the most common misconceptions I see is the idea that things will be faster if you pull all your data together before moving to the next step of your algorithm. This manifests itself in a number of ways. For example:

  • Building gigantic data messages
  • Loading large data structures into memory
  • Running process A for every item in a set, then running process B for every item, etc.

These approaches are disastrous from an performance perspective and they can be challenging to eliminate once in place. These designs often escape the abstraction boundaries and therefore it often requires a major re-write to move to a more efficient solution. For example, if you are generating a large document for another application to consume, you can't simply break it up into smaller pieces without modifying the downstream application. Or if your application is built to expect an array or other type with a known size, its not always trivial to replace that with an iterator.

For this reason, I consider this to be something you need to consider early on and an exception to the generally good advice to wait until you find you have a performance issue.

|improve this answer|||||
  • 1
    In some cases, things can actually be faster when you process your data in columns. In particular, this is the case when you're running in a SIMD architecture, such as GPGPU as column oriented processing means the GPU does not need to decode the same instructions for each row or with interpreted/VM-based language that doesn't use JIT recompilation, where the languages' regular looping construct can be much slower than calling functions working on a list that uses compiled code loops. Code in column oriented processing can also sometimes be much more readable when there's little branching. – Lie Ryan Jan 20 at 1:29
  • @LieRyan I'm not sure how that relates to my answer. Can you elaborate? – JimmyJames Jan 20 at 14:45

I see lots of good answers, but not one currently answering your literal question

So I'd like to know if they have a founded reason to exist.

As I remember correctly, each of these techniques had their justification around 20 to 30 years ago on slow hardware, especially for some not-so-sophisticated compilers or interpreters, in cases where even micro-optimization could bring a required performance gain.

Today, using modern compilers for languages like C++, Java or C#, I think there is not one of them where the examples listed in the question will bring a speed benefit in optimized code, not even in hot-spots. In some older, but still used language-implementations like VBA, some of those techniques may still be valid - sometimes.

Even today there exist a lot of small embedded devices which don't have huge CPU power and which are usually programmed in C. I am not actually sure about the optimization quality of the C compilers for those platforms, but I can imagine that there may be cases where some of those optimization techniques you mentioned still matter.

But as others have pointed out, there is no justification to apply micro-optimizations in advance, "just in case". The most sensible approach is

  • to implement the code as clean as possible
  • optimize afterwards, and only the relevant parts
  • keep only the optimizations which did really bring proven (=measured!) benefit
|improve this answer|||||
  • To your point, there's a lot of advice I remember from around 20 years ago about Java that if you followed it, can actually reduce performance and make it harder to optimize. For example, pooling objects is counter-productive in current GC models. – JimmyJames Jan 16 at 20:11

Make it work, then make it beautiful, then if you really, really have to, make it fast. 90 percent of the time, if you make it beautiful, it will already be fast. So really, just make it beautiful!

by Joe Armstrong

|improve this answer|||||

In reality, lots of times your code is just fast enough, so you do nothing.

If it’s not fast enough, lots of times you don’t need to optimise, you just have to figure out what you did that was completely stupid and killed performance, and stop doing it. That has been my experience in recent years, that when something was too slow, it was because of someone doing something stupid. (I suppose “optimising” involves doing something clever, so stopping to do stupid things doesn’t count as optimising. ) Picking the wrong data structure, like looking up keys in an array by sequential search instead of using a dictionary falls into this category.

The next step is looking for work that you do repeatedly for no good reason. Like sorting an array repeatedly. Calculating the same dataset repeatedly. Reading the same data from a database multiple times, and so on. Avoiding that can give you massive savings.

And if that is not enough and it looks like you need some real optimising: Find someone who knows what to do.

Micro optimisations like the ones suggested come at the very, very end of the list.

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.