# How to maintain well structured code and latency optimized code together?

I'm working on a c++ data analysis project. My workflow goes like this

1. Analyze the data and build models
2. Optimize the code for latency, to deploy for production
3. goto 1

Step 1 has lots of machine learning parameters, using which I test very minor variations of algorithms. In step 2, I clean up the unused parts of the code (non optimal parameters), optimize code for latency (changing maps to arrays for example), and deploy the code. These modifications are done directly on the step 1's code. No separate branch in maintained.

When new data is obtained and step 1 is required to be repeated, I would have lost the ability to test minor variations of an algorithm. One way to solve this is to maintain two branches. One will be an experimental branch, which has all the parameters for the minor variations of the algorithm. Another branch will be latency optimized code. But, the problem here is any small change in experimental branch will need to be repeated in the latency optimized branch, because there two branches cannot be merged. There are huge differences (even new files appearing) between experimental branch and latency optimized branch, which hinder direct merging.

Is there any other way to solve this?

EDIT1: Another example of step 2

For the sake of illustration, lets say step 1 leads me to a predictor y = f(x) = floor(x^3 + 3x^2 + 5x), where floor(z) = (value of integer closest but <= z). x in [0, 100]. The basic way to make prediction (in deployment) is to evaluate f(x). But, observe that the whole function is discrete in output and increasing. So, another way to predict is to store ranges of x, which map to y = 1, 2, ..., and do binary search on it. This will lead to maintaining a vector with entries, {(lbx_1, y_1), (lbx_2, y_2), ...}, where y_i = i, lbx_i gives the least value of x such that f(x) >= y_i. In this case, for any input, a simple binary search will fetch the prediction much faster. (In practice, f(x) is much more complicated than the above mentioned function).

The latency optimized branch will have this map for prediction. The experimental branch will evaluate the function. But I also need the general evaluator for experimentation, in case my predictor turns out to say y = f(x) = (x - 1)^2.

• I find that copy/paste and sensible follow-on testing works pretty well. Mar 24, 2021 at 14:42
• I've edited the description for more details on step 2 Mar 24, 2021 at 15:50
• Writing an optimizer for this would in itself be another huge project, especially catching the example in EDIT1. Mar 24, 2021 at 16:00

I would not try to abuse a version control system like git for this.

For complex optimization as described here, which can only be done manually, it is probably best to keep both versions of a function in the same code base in parallel. Both, the non-optimized version and the optimized one should be placed somewhere near to each other. In C++, switching between both versions could be done, for example, by using the preprocessor, or by a run time mechanics. (Template metaprogramming zealots will surely prefer to use partial template specialization.)

For example

`````` int f_slow(int x) {return floor(x^3 + 3x^2 + 5x);}

int f_fast(int x) {/*  50 lines of optimized code*/}

#ifndef OPTIMIZED
#  define f f_slow
#else
#  define f f_fast
#endif
``````

This way, `f_slow` is kept as a way more readable version of `f_fast` which serves as documentation. You should also use this for automated tests, asserting `f_slow(x) == f_fast(x)` for a range of test values, so making sure your optimization does not introduce some unexpected bugs.

Of course, this does not take the burden from you of maintaining two versions of the same function. But I think this is more or less unavoidable, since writing an optimizer seems to be pretty infeasible for this task.

The most effective course of action in this situation is probably to apply some self-discpline and work within clearly defined phases:

• an experimental phase, where only `f_slow` exist, and where you are modiyfing it quickly until it suits your needs

• a phase where `f_slow` is kept unchanged, and `f_fast`will be created, including the tests

When it comes to the point where you are going to do you experiments with a new version of f(x), give it another name (like `f_slow_v2`), keep `f_slow` as it is, so you can use `f_fast` as a blueprint for `f_fast_v2`. Later, when you are sure neither `f_slow` nor `f_fast` are needed any more, you can remove them both from the code simultanously.