What are good practices to perform software based optimization?

Question

On a project that I am working on we encountered some performance problems.

We decided to drop hibernate in favor of plain Jdbc to gain database performance.

By converting our code to Jdbc we achieved the performance that we aimed for but the code quality seems to have deteriorated. And it took us three months to implement the system, which has me wondering if throwing in more hardware would've been less expensive.

What would you do to optimize your application in a manner that limits side affects on architecture and complexity? (Mainly interested in answers from perspective of web applications but other perspective are welcome)

Answer 1

Optimization is always good - all you have to do is to optimize the correct metric.

Usually, it's worth assuming the cost of optimizations, and the gain - how much does it cost (in programmer salary) to optimize? How much does it increse the revenues?

A few (contrived) examples:

• A batch file moving some doc files from the secretary's computer to a local server once a week. It may take 1 hour, nobody cares. Important is to write the script.

• A video codec. It will process zillions of pixels through it lifetime, and it will run on embedded devices, with very limited resources. If it's slow, even if it does its job quite well, it will be replaced. So, a team of gurus will write a good part of it in ASM, and will carefully manually reorder instructions to get a few clock cycles out of it.

• Some backend Facebook code. If it runs 10% faster, it will shave off 10% of the datacenters energy bill - which runs into zillions $.

• A messaging system (Skype, Gmail, etc) it is used by millions. If it's lazy, everybody will complain. You loose market share. It's worth investing man-years to optimize.

• A Java accountability app, developed for your company - probably used by a few hundead people. It will be cheaper to buy two more computers than to spend a week optimizing.

In the context of web applications, there are two main reasons to optimize:

• When your app/site slows down below user expectations
• When your server becomes overloaded, and it's cheaper to optimize than to buy new machines.

Answer 2

Of course it's not always a bad idea. If it were, you could simply use generate-and-test for every computation (or even randomize-and-pray) and depend on faster hardware to do the job cheaper than your programmers could. And in fact, right now the pendulum is swinging towards software optimization, because raw processor speeds are plateau-ing and efficient multi-threaded coding is becoming way more important than it ever was.

That said, it is still often the case that buying a faster machine is ultimately cheaper than uglifying your code by contorting it in the name of performance. It's necessary to be aware of the pressures that will influence the trade-of in one way or another.

• The longer your code will have to maintained, the more you suffer from contorted, un-understandable code. (Note that this is not the same as longevity. A sort routine in an OS might be written once, proven correct for all inputs, and then literally never be changed again. In such extreme cases, spending lots of time optimizing and ending up with horrible-looking code makes sense.)

• Obviously, you need to know the cost of programmers, the cost of hardware, and at least an idea of how much technical debt you are accumulating, i.e. how much maintenance times will go up with more complicated code. This can vary enormously between individual programmers, so actually knowing your people is surprisingly important.

• Never assume. Always measure. Know whether or not the optimization you are pondering will actually improve things, and by how much. If that means you have to write a great deal of code that may need to be taken out again, so be it. Ultimately you still come out ahead because you have more knowledge of your system, and with some care, such experiments can inform your next experiments and actually save you time when a similar question comes up.

As for avoiding technical debt, the important thing is to limit the impact of whatever you change; a data access layer is large, but with proper encapsulation it can have effects only on the data access layer, so at least you know the limit of how much debt you can accumulate all told. Finding a few routines that eat 90% of your time and changing only those is even better (and quite often such routines do exist). A powerful profiler is your friend here.

Answer 3

It depends on what is actually causing the performance problems. Performance problems appear in all types of shapes and sizes, and did you actually fix the real bottlenecks?

IMHO a few hours with a query profiler tool and taskmgr/top can solve 90% of the database performance problems. Some additional indexes, restructuring queries or add some caching can make a huge difference.

Even if you add more hardware you might find your application design can't use it. If it is bottlenecked-by-design then more hardware can't help.

Answer 4

First, why would software based optimization be a bad idea at all?

When you're deploying web services which has their performance dictated more by the underlying technologies than the code itself, then it might be questionable to focus on optimizing your own product code-wise. However, note that as with any kind of optimization, you must profile the perfrormance of the target. If the bottleneck is in used technologies, then either find alternatives or try to find ways to rely less on them with performance-critical tasks.

In general I'd argue that "throwing hardware at it is cheaper" is a very stupid way of thinking. With such attitude, not only are you being lazy and irresponsible, you also lock out people with less hardware resources. Of course this might be a bit different in perspective of web applications which run on dedcated hardware, where such attitude might not be as deceiving as it s with client-deployed software.

Answer 5

So is software based optimization always a bad idea?

No. Neither it's always bad, not always good. However, the optimization you describe — going lower level would not be the preferred way to go about it.

Generally spending numerous hours of development time to shave off few percent of execution time in case of web apps it a waste of money. On the other hand, spending time on making it scalable and eliminating code with suboptimal time complexity might be worth while.

You have to keep in mind, that unless your application is designed to be scalable, you won't be able to easily add more hardware. If parts of the code have ridiculous time complexity, adding hardware won't help deal with growth either.

Another thing is that in many cases, rather than optimizing the code, you should optimize the architecture.