Using a closure to avoid code duplication in Python

Question

Sometimes I find myself wanting to run the same code from a few different spots in the same function. Say I have some function func1, and I want to do the same thing from a few different spots in func1. Normally the way to do this would be to write another function, call it "func2", and call func2 from several different places in func1. But what about when it's convenient to have func2 access variables that are local to func1? I find myself writing a closure. Here's a contrived example:

import random
import string

def func1 (param1, param2):
    def func2(foo, bar):
        print "{0} {1} {2:0.2f} {3} {4} {0}".format('*'*a, b, c, foo, bar)

    a = random.randrange(10)
    b = ''.join(random.choice(string.letters) for i in xrange(10))
    c = random.gauss(0, 1)
    if param1:
        func2(a*c, param1)
    else:
        if param2 > 0:
            func2(param2, param2)

Is this the Pythonic way to handle this problem? A closure feels like pretty heavy machinery to be rolling out here, especially given that I have to construct a new function every time func1 is called, even though that function is going to be basically the same every time. But it does avoid the duplicated code and in practice the overhead of repeatedly creating func2 doesn't matter to me.

Answer 1

It is a an acceptable form. As @Giorgio said, I would put the closure after the captured variable definition to ease the flow of reading.

The alternative form would be to define another function, taking a, b, c as parameters. That is 5 parameters which is a lot. The closure allows you avoid repeating yourself in a very simple way. This is a big win for your version.

You can use the timeit module to compare the performances of simple snippets. You should check yourself that a closure is not a heavy machinery. The only problem I see is that it creates more nested elements. So if you find yourself writing a big closure, you should try to extract the complex part outside. But in this case I don't think it is an issue.

import timeit
import random
import string

def func1 (param1, param2):
    def func2(foo, bar):
        return "{0} {1} {2:0.2f} {3} {4} {0}".format('*'*a, b, c, foo, bar)

    a = random.randrange(10)
    b = ''.join(random.choice(string.letters) for i in xrange(10))
    c = random.gauss(0, 1)
    if param1:
        func2(a*c, param1)
    else:
        if param2 > 0:
            func2(param2, param2)

def func4(foo, bar, a, b, c):
    return "{0} {1} {2:0.2f} {3} {4} {0}".format('*'*a, b, c, foo, bar)

def func3 (param1, param2):

    a = random.randrange(10)
    b = ''.join(random.choice(string.letters) for i in xrange(10))
    c = random.gauss(0, 1)
    if param1:
        func4(a*c, param1, a, b, c)
    else:
        if param2 > 0:
            func4(param2, param2, a, b, c)

print timeit.timeit('func1("tets", "")',
 number=100000,
 setup="from __main__ import func1")

print timeit.timeit('func3("tets", "")',
 number=100000,
 setup="from __main__ import func3")

Answer 2

I am not a fan of OP's style here. The inner function is not testable and not reusable. In addition, if either the inner or outer function grows significantly long / complex, it becomes a mess to read / debug / review.

For short inner & outer functions, this pattern is perhaps acceptable, but if the inner function is long or if there are multiple such functions, I'd find a class definition more readable.

Variants I find preferable:

• Have an outer function which takes some parameters (the ones you expect to reuse) and returns a closure / inner function which only takes parameters that need to vary between cases.
• Define & instantiate a class which stores the repeated variables on self
• Define a common-parameters class and build functions which take instances of it. All the repeated parameters then get compressed down to one.

Each of these other options is more testable, more reusable, and more extensible.

Answer 3

TL;DR: Create function (ideally at import time, not execution time) unless you need to save state

A closure is so named because you "close over" a lexical variable in the enclosing scope. This allows you to save state. That is why you use closures.

If a, b, and c represent variables declared in func1 you have a closure. If they do not represent captured variables from the enclosing scope the you just have a function declared inside a function.

If you want to capture values, but aren't going to modify them I would use functools.partial instead of a closure or a dynamically defined function. ie.

from functools import partial

def func1(a, b):    
  print a, b

def func2(i, j ,k):
  fn = partial(func1, i)
  fn(j)
  fn(k)

I would still make func1 a regular helper function declared outside of func2. Then possibly use partial to save on passing arguments on subsequent calls.