Note: I'm not sure if the way I phrase the question initially will fit 100% on this board, so please help me to focus on the main point.

Scenario: You develop an algorithm or a neural network that performs a certain task, in the case of the algorithm in many steps that use input from the previous step(s) to compute some output. Then you want to evaluate this algorithm, maybe benchmark it under different circumstances, with differing inputs. In the case of the NN you experiment with precisely what information you provide as inputs, in the case of the algorithm maybe you have to debug it, so you want to be able to run the whole algorithm, a certain step i in isolation, and steps i to j.

Now when I did this in the past, this resulted in a lot of very messy and copy-paste heavy imperative code. Is there a way to use OO, maybe through inheritance or composition, to reduce this complexity and avoid having 20 versions of the same method that just run your algorithm?

  • Would "record and play back testing" approach solve your problem? To do so, implement methods which save and load intermediate data, at the beginning and end of function calls. – rwong Apr 8 '18 at 13:29
  • In very large scale software development (for example, the difference in source code between different "implementations" may be 1k - 10k lines of code), these different "implementations" will be in different branches (on a version control system), each branch will be compiled and performance-benchmarked by the continuous integration system. Note that this applies to software with millions of lines of code, not to small or medium-sized projects. – rwong Apr 8 '18 at 13:31
  • 2
    "Exposing the individual steps of a multi-step algorithm, so that individual steps can be manipulated through API (programming interface)" may be necessary for testing purpose. This goes against the "information hiding" principle. If it is decided that testing is more important, then "information hiding" becomes relatively less important; and vice versa. – rwong Apr 8 '18 at 13:33
  • "Now when I did this in the past, this resulted in a lot of very messy and copy-paste heavy imperative code" - why? Nothing you wrote before gives a hint why this is the case, so we could only guess around here. I am voting to close this question as "unclear". Best recommendation I can give you is to post a real example, maybe on codereview.stackexchange and ask for a review. – Doc Brown Apr 8 '18 at 16:26
  • 1
    ... another recommendation is to be aware that benchmarking code is not very different from any other kind of code, so use the same principles and tactics you use for any other kind of code to keep it maintainable. Like KISS, SOLID, proper versioning and source control, proper naming etc. – Doc Brown Apr 8 '18 at 16:32

Perhaps I should set the correct expectations at the beginning: the best solution might not be object-oriented (OO). You should use what is the most appropriate, not what is the purest-OO idea.

Are you working mainly in Python, or one of the domain-specific languages (DSLs) used for neural network training?

You can try refactor the code from e.g. total 150 lines, breaking down into several functions. These functions are reusable across your variants of programs. Once you have refactored (extracted reusable functions), discuss with your team members so that they all know that you have provided these functions to them, so that they will use it. Reusable functions shared within a team requires a moderate effort in code maintenance.

Some techniques that are needed to solve the problem (such as "polymorphism", "interface/implementation", "higher-order functions" or simply exec() / eval()), depend on whether they are allowed in that language. Most languages will try to provide at least one way to solve this particular problem, however DSLs might not provide any. (For example, GPU programming languages don't have any.) In that case, you may have to do part of the programming (the outer-loop) in Python, calling into the DSL for smaller pieces of tasks.

In other cases, you may have to use Python to generate source code for the DSL, by formatting and concatenating strings and writing them into a text file which can be used as a source file for the DSL.

To give more useful and specific answers, please try to explain your situation better. Try to explain with pseudo-code. It will also be language-specific. I would say that Python is designed to solve this particular problem for scientists better than other languages.

An example of loading and saving at intermediate checkpoints look like this: (pseudo-code)

// code for step 7
if (min_step <= 7) and (max_step >= 7)
    if (input_for_step_7 == null)
        if (step_7_input_filename == null)
            throw error ("either need execute step 6 or need input file for step 7")
            input_for_step_7 = load_input_for_step_7(step_7_input_filename)
    assert(input_for_step_7 != null)
    output_for_step_7 = execute_step_7(input_for_step_7)
end // min_step, max_step

In neural network training, "record and play back" are still useful, but have their limitations.

This is because modifications of one step of the algorithm will fundamentally change the characteristics of its output. Therefore, to test the effects of modifying one step of the algorithm, the entire chain of algorithm steps have to be re-executed. Thus, the idea of "testing in isolation" is only useful in a limited sense.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.