I'm a junior software developer and I was wondering when would be the best time to optimize a software for better performance (speed).
Assuming the software is not extremely large and complex to manage, is it better to spend more time at the beginning optimizing it or should I just develop the software that executes all functionality correctly and then proceed to optimize it for better performance?
The number one thing should always and forever be readability. If it's slow but readable, I can fix it. If it's broken but readable, I can fix it. If it's unreadable, I have to ask someone else what this was even supposed to do.
It is remarkable how performant your code can be when you were only focused on being readable. So much so I generally ignore performance until given a reason to care. That shouldn't be taken to mean I don't care about speed. I do. I've just found that there are very few problems whose solutions actually are faster when made hard to read.
Only two things take me out of this mode:
In any case, avoid analysis paralysis by making yourself think you shouldn't try a solution because it might not be the fastest. Your code will actually benefit if you try multiple solutions because making the changes will force you to use a design that makes it easy to change. A flexible code base can be made faster later where it really needs it. Choose flexible over speed and you can choose the speed you need.
If a certain level of performance is necessary (a non-functional requirement), then that should be a design goal from the start. E.g. this can influence which technologies might be appropriate, or how you structure the data flow in the program.
But in general, it is not possible to optimize before the code is written: first make it work, then make it right, and, finally, make it fast.
One big problem with optimizing before implementing most functionality is that you've locked yourself into sub-optimal design decisions at the wrong places. There's often (but not necessarily) a tradeoff between maintainability and performance. Most parts of your program are totally irrelevant for performance! Typical programs only have a few hot spots that are really worth optimizing. So sacrificing maintainability for performance in all those places that don't need performance is a really bad trade.
Optimizing for maintainability is the better approach. If you spend your cleverness on maintainability and clear designs, you will find it easier in the long run to identify critical sections, and safely optimize them without compromising the overall design.
when would be the best time to optimize a software for better performance(speed).
Begin by removing from your mind the concept that performance is the same thing as speed. Performance is what the user believes performance is.
If you make an application respond twice as fast to a mouse click and you go from ten microseconds to five microseconds, the user does not care. If you make an application respond twice as fast to a mouse click and you go from four thousand years to two thousand years, again, the user does not care.
If you make your application twice as fast and you use up all the memory on the machine and crash, the user does not care that it is now twice as fast.
Performance is the science of making effective tradeoffs about resource consumption to achieve a particular user experience. The user's time is an important resource, but it's never just about "faster". Achieving performance goals almost always requires tradeoffs, and they're often trading off time for space or vice versa.
Assuming the software is not extremely large and complex to manage
That's a terrible assumption.
If the software is not large and complex to manage then it probably does not solve an interesting problem that a user cares about, and it is probably super easy to optimize.
is it better to spend more time at the beginning optimizing it or should I just develop the software that executes all functionality correctly and then proceed to optimize it for better performance?
You're sitting there at a blank page and you write void main() {} Do you start optimizing? There's nothing to optimize! The right order is:
If you try to do it in any other order you end up with wrong code that is a mess, and now you've got a program that produces wrong answers really quickly and resists changes.
But there is a step missing there. The real right order is:
As a general rule, it's best to optimize for performance later, but I've seen many projects go bad when developers realize they've ended up with software that is to slow when any significant load or data is added to it.
So, a middle ground approach would be best in my opinion; don't put too much emphasis on it, but don't disregard performance altogether.
I'll give an example that I've seen many times; given a ORM library, we have a User entity that can have one or more Orders. Let's loop all Orders for a User, and find out how much the User has spent in our store - a naive approach:
User user = getUser();
int totalAmount;
for (Order o : user.getOrders()) {
totalAmount += o.getTotalAmount();
}
I've seen developers write similar things, without any thought of the implications; first we get the user, which hopefully will just be one SQL query on the User table (but might involve much, much more), then we loop through the orders, which might include getting all relevant data for all the order lines on the order, product information, etc. - all this just to get a single integer for each order!
The amount of SQL queries here might surprise you. Of course, it's dependent on how your entities are structured.
Here, the correct approach would most likely be to add a separate function to get the sum from the database via a separate query written in the query language provided by the ORM, and I would advocate doing this the first time around, and not postponing this for later; because if you do, you'll probably end up with a lot more issues to take care of, and not be sure where to start.
Total system performance is a product of the complex interactions of the totality of the system components. It's a nonlinear system. Therefore performance will be gated not just by the components' individual performance but by bottlenecks between them.
Obviously you can't test for bottlenecks if all the components of your system aren't built yet, so you can't really test very well early on. On the other hand, after the system is built, you may not find it so easy to make the changes you need to make to get the performance you want. So this is a bone fide Catch-22.
To make matters more difficult, your performance profile can change drastically when you switch to a production-like environment, which is often not available early on.
So what do you do? Well, a few things.
Be pragmatic. Early on, you can choose to use platform features that are "best practice" for performance; for example, utilize connection pooling, asynchronous transactions, and avoiding statefulness, which can be the death of a multi-threaded application where different workers are contending for access to shared data. Normally you wouldn't test these patterns for performance, you'd just know from experience what works well.
Be iterative. Take baseline performance measures when the system is relatively new, and re-test occasionally to make sure newly introduced code hasn't degraded performance too much.
Don't overoptimize early. You never know what is going to be important and what isn't going to matter; a superfast string parsing algorithm may not help if your program is constantly waiting on I/O, for example.
In web applications especially, you can focus not so much on performance but on scaleability. If the application can scale out, performance almost doesn't matter, since you can keep adding nodes to your farm until it is fast enough.
Special attention goes to the database. Due to transactional integrity constraints, the database tends to be a bottleneck that dominates every part of the system. If you need a high-performance system, make sure you have talented folks working on the database side, reviewing query plans, and developing table and index structures that will make common operations as efficient as possible.
Most of these activities are not for the beginning or end of the project but must be attended to continuously.
I'm a junior software developer and I was wondering when would be the best time to optimize a software for better performance (speed).
Understand that there are 2 very different extremes.
The first extreme are things that effect a large part of the design, like how to split the work into how many processes and/or threads and how pieces communicate (TCP/IP sockets? Direct function calls?), whether to implement an advanced JIT or a "one opcode at a time" interpreter, or whether to plan data structures to be amenable to SIMD, or ... These things tend to have a strong influence on the implementation and become excessively difficult/expensive to retro-fit after.
The other extreme is micro-optimisations - tiny little tweaks all over the place. These things tend to have almost no influence on the implementation (and are often best done by a compiler anyway), and it's trivial to make these optimisations whenever you feel like it.
In between these extremes is a huge grey area.
What it really comes down to is experience/educated guesses being used to answer a "do the benefits justify the costs" question. For optimisations at/near one extreme if you guess wrong often it means throwing all your work out and restarting from scratch or project failure (too much time spent on an unnecessarily over-complicated design). At/near the other extreme it's far more sensible to leave it until you're able to prove it matters using measurement (e.g. profiling).
Unfortunately we live in a world where far too many people think optimisation only includes the (mostly irrelevant) things at the "trivial" extreme.
It is easiest to write code that is neither porformant nor maintainable. It is harder to write porformant code. It is harder yet to write maintainable code. And it is the most difficult to write code that is both maintainable and performant.
Now, obviously, it depends on the type of system you are making, some systems will be heavily performance critical and need that planned in from the start. For the extremely talented people like Eric Lippert, whom answered above, these systems may be common; but for most of us, they are the minority of the systems we build.
However, given the state of modern hardware, in the majority of systems, it is not needed to pay special attention to optimization from the beginning, but rather, avoiding performance destruction is usually sufficient. What I mean by this is, avoid doing plainly stupid thing such as bringing back all the records of a table to get a count instead of just querying select count(*) from table. Just avoid making mistakes and make an effort to understand the tools you are using.
Next, first focus on make your code maintainable. By this, I mean:
Maintainable code is much easier to optimize when statistics shows it is needed.
Next, make sure your code has LOTS of automated tests, this has several benefits. Less bugs means more time to optimize, when needed. Also, when you do optimize, you can iterate and find the best solution much faster since you find bugs in your implementations much faster.
Automated deployment scripts and scripted infrastructure are also very useful for performance tuning, as again, they allow you to iterate faster; not to mention its other benefits.
I might be biased working in very performance-critical areas like image processing and raytracing, but I'd still say to optimize "as late as possible". No matter how performance-critical your requirements are, there's always so much more information and clarity in hindsight, after you measure, than in advance, which means even the most effective optimizations are typically applied later after gaining such knowledge.
Peculiar Cases
But sometimes "as late as possible" is still pretty damn early in some peculiar cases. If we're talking offline renderers, for example, the data structures and techniques you use to achieve performance actually seep into the user-end design. This might sound disgusting but the field is so cutting-edge and so performance-critical that users accept user-end controls specific to the optimization techniques applicable to a particular raytracer (ex: irradiance caching or photon mapping), since some of them are used to waiting hours for an image to render, and others are used to dishing out enormous sums of money to rent or own a render farm with machines dedicated to rendering. There's a massive reduction in time and money for those users if a competitive offline renderer can offer a non-trivial reduction in time spent rendering. This is a sort of area where a 5% reduction in time actually excites users.
In such peculiar cases you can't just pick one rendering technique willy-nilly and hope to optimize it later, since the entire design, including the user-end design, revolves around the data structures and algorithms you use. You can't necessarily even just go with what worked well for other people since here, you, as the individual, and your particular strengths and weaknesses, factor in heavily to delivering a competitive solution. The mindset and sensibilities of the main developer behind Arnold is different from those working on VRay who used a very different approach; they can't necessarily swap places/techniques and do the best job (even though they're both industrial leaders). You have to kind of experiment and prototype and benchmark and find what you're particularly good at doing given the endless array of cutting-edge techniques out there if you hope to ship something competitive that will actually sell. So in this peculiar case, performance concerns move way up to the front as perhaps the most important concern prior to even beginning development.
Still that's not necessarily a violation of optimizing "as late as possible", it's just "as late as possible" is rather early in these extreme and peculiar cases. Figuring out when and also what doesn't need such early performance concerns, if ever at all, is probably the main challenge to the developer. What not to optimize might be one of the most valuable things to learn and keep learning in a developer's career, since you can find no shortage of naive developers who want to optimize everything (and unfortunately even some veterans who managed to somehow keep their job in spite of their counter-productivity).
As Late As Possible
Perhaps the most difficult part is to try to understand what it means. I'm still learning and I've been programming for almost three decades. But especially now in my third decade, I'm starting to realize it's not that difficult. It's not rocket science, if you focus more on design than implementation. The more your designs leave breathing room for appropriate optimizations later without changes to the design, the later you can optimize. And the more and more productivity I've gained seeking out such designs which afford me that breathing room.
Design Which Offer Breathing Room to Optimize Later
These types of designs actually aren't that hard to achieve in most cases if we can apply some "common sense". As a personal story I'm into visual arts as a hobby (I find it somewhat helps to program software for artists being somewhat one myself to understand their needs and speak their language), and I spent some time in the early 2000s using Oekaki applets online as a quick way to doodle and share my work and connect with other artists.
In particular my favorite site and applet there was riddled with performance flaws (any non-trivial brush size would slow to a crawl), but had a very nice community. To work around the performance issues I used teeny little 1 or 2-pixel brushes and just scribbled my work like so:
Meanwhile I kept giving the author of the software suggestions to improve performance, and he noticed my suggestions were of a particularly technical nature talking about memory optimizations and algorithms and so forth. So he actually asked if I was a programmer and I said yes and he invited me to work on the source code.
So I looked at the source code, ran it, profiled it, and to my horror he had designed the software around the concept of an "abstract pixel interface", like IPixel, which ended up being root cause behind the top hotspots for everything with dynamic allocations and dispatch for every single pixel of every single image. Yet there was no practical way to optimize that without reconsidering the entire software's design because the design had trapped him into a corner where there's not much beyond the most trivial of micro-optimizations when our abstractions are working at the granular level of a single abstract pixel and everything depends on this abstract pixel. And so we gave up on the idea of optimizing the software much to handle bigger brushes and real-time filters and such and I went back to doodling with 1 or 2-pixel brushes.
I think that's a violation of "common sense" but obviously it wasn't such common sense to the developer. But it's like don't abstract things at such a granular level where even the most basic use cases are going to be instantiating by the millions, as with pixels, or particles, or tiny units in a ginormous army simulation. Favor the IImage (you can handle all the image/pixel formats you need at that bulkier aggregate level) or IParticleSystem to IPixel or IParticle, and then you can put in the most basic and quick-to-write and simple-to-understand implementations behind such interfaces and have all the breathing room you'll ever need to optimize later without reconsidering the entire software's design.
And that's the goal as I see it these days. Excluding the peculiar cases like offline renderers above, design with enough breathing room to optimize as late as possible, with as much hindsight information as possible (including measurements), and apply any necessary optimizations as late as possible.
Of course I'm not necessarily suggesting to start off using quadratic complexity algorithms on inputs that easily reach a non-trivial size in common user-end cases. Who does that anyway? But I don't even think that is such a big deal if the implementation is easy to swap out later. That's still not a grave mistake if you don't have to reconsider any designs.
It depends on what that performance means to your application. And on whether it is even possible to optimize performance before your application is functionally complete.
Most often you should not worry about it until you have nothing better to do, but it could be that a certain level of performance is critical to the success of your application. If that were the case and you suspect it may not be easy, you should start looking at performance for the sake of "failing fast".
An important principle for any project is to focus on the hard parts first. That way, if it turns out you can't do it, you will know early and there will be time to try something totally different or the project may be cancelled before too much was spent on it.
I am going to suggest performance is more than speed. It includes scale (hundreds to thousand of concurrent users). For sure you don't want the application to tank when it gets a production load. Performance includes how much resource (e.g. memory) does the application consume.
Performance is also ease of use. Some users would rather have 1 keystroke do a task in 10 seconds than 2 key strokes do the task in 1 second. For stuff like that ask your design lead. I don't like to take stuff like this to users early. In a vacuum they may say X but once they are working with a functional pre-release they may say Y.
The best individual speed is to hold a resource such as a database connection. But for scale you should acquire the connection as late as possible and release it as soon as possible. One trip to the database to get 3 things is faster than 3 separate trips to the database.
Are you making a trips for information that does not change during the session. If so get it at the session start and hold it is memory.
When picking collection type consider the functional, speed, and size.
Are you sure you need to hold the items in a collection? A common problem there is read all lines from a file into a list and then process the list one line at a time. It is much more efficient to read the file one line at a time and skip the list.
Are you looping three times when you could loop once and do three things.
Is there a spot where you may need to process on another thread with a call back. If so package the code with that possible need in mind if it does not interfere with the immediate design needs.
A lot of performance is also clean code.
There is premature optimization and there is just doing common sense stuff up front that does not really take more time.
In database is where I see premature optimization. Will de-normalize for speed before there is a speed problem. The argument I get is if we change the table later then we have to change everything. Often you can create a view the presents the data that way and it may need to be swapped of for a de-normalized table later.
First get the MVP to the customer. Then do statistical profiling to see what's slow.
