When decomposing a large function, how can I avoid the complexity from the extra subfunctions?

Question

Say I have a large function like the following:

function do_lots_of_stuff(){

    { //subpart 1
      ...
    }

    ...

    { //subpart N
      ...
    }
}

a common pattern is to decompose it into subfunctions

function do_lots_of_stuff(){
    subpart_1(...)
    subpart_2(...)
    ...
    subpart_N(...)
}

I usually find that decomposition has two main advantages:

1. The decomposed function becomes much smaller. This can help people read it without getting lost in the details.
2. Parameters have to be explicitly passed to the underlying subfunctions, instead of being implicitly available by just being in scope. This can help readability and modularity in some situations.

However, I also find that decomposition has some disadvantages:

1. There are no guarantees that the subfunctions "belong" to do_lots_of_stuff so there is nothing stopping someone from accidentally calling them from a wrong place.
2. A module's complexity grows quadratically with the number of functions we add to it. (There are more possible ways for things to call each other)

Therefore:

Are there useful convention or coding styles that help me balance the pros and cons of function decomposition or should I just use an editor with code folding and call it a day?

---

EDIT: This problem also applies to functional code (although in a less pressing manner). For example, in a functional setting we would have the subparts be returning values that are combined in the end and the decomposition problem of having lots of subfunctions being able to use each other is still present.

We can't always assume that the problem domain will be able to be modeled on just some small simple types with just a few highly orthogonal functions. There will always be complicated algorithms or long lists of business rules that we still want to correctly be able to deal with.

function do_lots_of_stuff(){
   p1 = subpart_1()
   p2 = subpart_2()
   pN = subpart_N()
   return assembleStuff(p1, p2, ..., pN)
}

Answer 1

Keep each function as simple as possible.

Think of it in simple terms, the way a function is meant to be:

• gets 0 to N inputs,
• returns 0 or 1 result (possible a composite or collection),

• and isn't tied to state.

  

When you stick to functional programming idioms, you get rid of most of these questions you're asking yourself. Sure, your class will get bigger in terms of number of functions. But if the methods are not tied to each by internal state changes, they get easier to understand, manage and compose to achieve an end result.

Also, try to give them appropriate accesses based on the above design decisions. Helpers will commonly easily be declared as statics (and if they don't seem to need to be private or can be reused, they could extracted to an helper class), which gives a strong hint to the other developers: this thing is meant to be independent and side-effect free.

Repeat the following mantras to aim for purity:

• My function shall be:
  • short,
  • side-effects free,
  • realizing one and one function only.
• My function shall be strict on output. ^[1]
• My function shall be testable, and tested.
• My function shall be readable and read like a natural language expression.
• My function shall be documented. ^[2]
• My function shall be null-hostile.

^{[1] Whether it shall be strict or lenient on input depends on whether it's consumer code or library code.
[2] Self-documentation counts, comments for tricky parts count as well.}

---

Of course, if you are in a generally non-FP-oriented code base, you won't manage to avoid shared mutable state for ever, but it's a very good, sensitive and no-BS guideline to follow. Even if you do get it wrong by over-modularizing and complexifying your class, it'll still be easier to pick up from there and refactor again than from a giant dump of code with high complexity and tight coupling.

Regarding the rules of the compositionality of your functions, these are your business rules. They are dictated by what you want to achieve, there's no automagical way of determining it for you.

Answer 2

Just because your functions can call each other a variety of ways doesn't mean they do. Your program's complexity isn't suddenly quadratic because you've named the blocks of code you ran before...

All of the other advice is good, but missing a key issue: you have a function doing a lot of stuff. Breaking that function into a series of function calls doesn't make it do less stuff, it just pushes the stuff around. Having a function that does too much is a sign that you have poorly abstracted your problem. Fix your abstraction so that the functionality is a better mapping to what you need to do and the function will naturally be less complex.

Answer 3

More dynamic languages like javascript and python have the ability to declare functions inside of functions

function a(param1, param2) {
  function helperFunc1() {
    return param1 % param2 * param2
  }
  function helperFunc2(param3) {
    return (param1 - param2) * param3
  }
  return helperFunc1() + helperFunc2(2) - helperFunc2(1.5)
}

These internal functions should have access to all vars inside the main function while hiding them from the outside, like a class would.

Answer 4

I would suggest moving to a more functional programming style. The fact that you are talking about "calling in the wrong order" and showing functions that don't return data, this means that you are working with very impure functions.

The better approach is to work with pure functions:

1) Reject global state variables. They are bad. Enough said.

2) Every function should take arguments and return something.

3) Given the same arguments functions should output the same data.

When you follow these goals you will find that your code becomes much cleaner, functions can call other functions without fear of side-effects because functions are not allowed to mess with global state.

Now, in a web/gui app, you will have to modify global state at some point, in that case you can break the above rules in small fringe cases.

At this point, decomposing functions is extremely simple. You simply start taking logical parts in the code and breaking them into functions. When you find duplicated code, merge the functions into a single function. If you find yourself writing the same code over and over, then find a way to write that as a single function.

For instance, in a pseudo language, a for loop could be re-written as a map. So instead of this:

arr = [1, 2, 3, 4, 5]

for (x = 0; x < arr.length; x ++)
{
   arr[x] += 1
}

You could write a map function once:

function map(func, arr)
{
   out = []
   for (x = 0; x < arr.length; x ++)
   {
     out.add(func(arr[x]))
   }
   return out;
}

And in the future, instead of a for loop, you simply write:

function inc (x) { return x + 1; }

map(inc, [1 2 3 4])

So instead of a single-use-case imperative for-loop. Now you have 2 functions that you can reuse in thousands of places in your program.

Resources:

http://skilldrick.co.uk/2011/07/easy-functional-programming-in-javascript-with-underscore-js-part-1/

http://www.ibm.com/developerworks/library/wa-javascript/index.html

EDIT:

Update to address the OP's edits:

In your example, I suggest using maps in a situation like this. So in your example:

function foo(arg) 
{
    data = {"p1": func1(arg),
            "p2": func2(arg),
            "pN": funcN(arg)}
    return assemble(data);
}

So in general, try to start with a empty hash map, and build out the structure by applying small functions to it until you build out your entire data set.

Or in a much cleaner way:

function reduce(fn, data, init)
{
   for (x = 0; x < data.length; x ++ 1)
       init = fn(init, data[x])
   return init
}

Then simply:

function foo(arg)
{
    fns = [p1, p2, pN]
    data = reduce((l, f) -> f(l), fns, {})
    reduce assembleData(data)
}

Using reduce you can also implement business rules with ease. Focus around data, and functional programming and you'll start to see how simple these things can really be.

Every day I spend close to 10 hours programming in both C# and Clojure, so I've seen both extremes and the functional approach simplifies things dramatically.

Perhaps a full real-world example would be in order?

Answer 5

I would suggest that the OP look at refactoring the problem and solution before starting to code. If something is so complex that you start to worry about these issues, then you really need to jump outside of the box and look at the problem from a different angle.

Just for fun, Maybe you can create a DSL to solve your problem ^^