When module calling gets ugly

Question

Has this ever happened to you? You've got a suite of well designed, single-responsibility modules, covered by unit tests. In any higher-level function you code, you are (95% of the code) simply taking output from one module and passing it as input to the next. Then, you notice this higher-level function has turned into a 100+ line script with multiple responsibilities.

Here is the problem. It is difficult (impossible) to test that script. At least, it seems so. Do you agree? In my current project, all of the bugs came from this script.

Further detail: each script represents a unique solution, or algorithm, formed by using different modules in different ways.

Question: how can you remedy this situation?

Knee-jerk answer: break the script up into single-responsibility modules.
Comment on knee-jerk answer: it already is!

Best answer I can come up with so far: create higher-level connector objects which "wire" modules together in particular ways (take output from one module, feed it as input to another module). Thus if our script was:

Foo.Input fooIn = new Foo.Input(1, 2);
Foo.Output fooOutput =  fooModule.operate(fooIn);
Double runtimevalue = getsomething(fooOutput.whatever);
Bar.Input barIn = new Bar.Input( runtimevalue, fooOutput.someOtherValue);
Bar.Output barOut = barModule.operate(barIn);

It would become with a connector:

FooBarConnectionAlgo fooBarConnector = new fooBarConnector(fooModule, barModule); 
Foo.Input fooIn = new Foo.Input(1, 2);
Bar.Output barOut = fooBarConnector.operate(fooIn);

So the advantage is, besides hiding some code and making things clearer, we can test FooBarConnectionAlgo.

I'm sure this situation comes up a lot. What do you do?

Answer 1

You've got it right. This is called "Structured Programming". It works well when you have a problem to be solved with a series of sequential steps. You just break each problem into a few sub-problems, then break down the sub-problems, then break down the sub-sub-problems, then.... You seem to be at the point where you have a top-level subroutine--er, method, and a bunch of bottom level ones. The top one's getting big, so you need an middle layer, as it were. If your problem (I'd like to call it a program, but it's really a part of a program) gets big enough, you could end up with many levels of methods.

Each method's actions should be determined solely by the parameters passed to it. I say this to be complete; with your focus on testing, you understand this as well as any old-time Structured Program evangelist. Ideally, each method would be called from more than one place, showing that it is generally useful, but it's better that methods use sub-methods that only they call, and that they only call once, than that the main method get large and unweildy.

I'm not sure how Sructured Programming and OOP relate. They seem to blend nicely, and it's always obvious when to use what. OO seems more for tracking and maintaining a program's state, for handling periodic input from users and such. SP is more for crunching numbers. I have found that when I've written big number-crunchers, classes start to creep in at the lower levels of my SP hierarchy.

Answer 2

I think what you're referring to is an integration test. Typically you would have a set of test data and known results, just like for a unit test, but in this case you're testing the whole integrated work, instead of only one module in isolation.

An integration test can't get into every nook and cranny of every bit of code, but it'll give you pretty good confidence that you've wired everything up correctly.

Answer 3

In isolation, I think you are on the right track. I am not sure I would bother writing "connectors" that join modules for each specific case. However, it does make sense to create objects that perform the logic and can have its dependencies injected into it. Why not just do that for the entire "script" itself? Then the script itself is testable. You can then inject mocks or "doubles" to ensure things happen the way you expect. After all, I expect that it is the script as whole that you care about, rather then each individual interaction between the modules.