At the beginning. Identify required performance characteristics. If you can't identify the target, you either need to step back to better understand your requirements or defer until you know your component requirements with the risk you might be rewriting. Then, test. Don't optimize, test. If you code fails the performance test, optimize. With a test framework in place, usage of existing performance monitoring tools should make the task reasonably easy.

Keep the performance tests in place for the lifetime of the project as a regression test. Maintenance code is notorious for triggering performance problems because the 'fixes' often have a very narrow focus.