Approach for updating status of a function

Question

When I call a function, I want to receive updates when the function reaches some milestones:

def do_something():
    start_with_something()
    # update
    for x in iterate_something():
         # update each iteration
    do_something_expensive()
    # update again
    finish_with_something_else()
    # final update
    return the_nice_result

This would allow me, for example, update an UI with a progress bar. Or show some initial results to the user after the first update.

A way to do this would be using callbacks:

def do_something(initial_update, iteration_update, update_again, final_update):
    some_initial_data =start_with_something()
    initial_update(some_initial_data)
    for x in iterate_something():
        iteration_update(x)
    ...

Another way, in python, would be to (abuse?) generators:

def do_something():
    some_initial_data= start_with_something()
    yield some_initial_data
    for x in iterate_something():
        yield x
    do_something_expensive()
    yield ...

There are pros and cons to both situations, but I do not really fancy any of them. For example, the first one clutters the function signature and updates information passed-in values, and the second one abuses an iteration mechanism.

In the end I want to keep my do_something function separated from any UI stuff.

I would like to know if there is an approach for this situation or a way to architecture this, specially in Python, which can avoid both of the cons.

Answer 1

All of the approaches you mentioned are valid, and no approach feels entirely natural in Python.

Passing event callbacks as arguments is usually the most appropriate approach, simply because it doesn't complicate using the function, and the event handler can be implemented with low ceremony. Something like this:

def slow_function(*arguments, on_iteration=None, on_final=None):
  state = init(arguments)

  for x in whatever():
    state += x
    if on_iteration is not None:
      on_iteration(x)

  if on_final is not None:
    on_final()
  return state

def on_iteration(x):
  print("iteration:", x)

slow_function(1, 2, 3,
              on_iteration=on_iteration,
              on_final=lambda: print("done"))

If you have many such events in a function, it will likely be more convenient to use an object:

# by default, handlers do nothing (null object pattern)
class SlowFunctionProgressHandler:
  def on_iteration(self, x): pass
  def on_final(self): pass

def slow_function(*arguments, event=SlowFunctionProgressHandler()):
  ...

# can implement/override a single event type
class MyProgessHandler(SlowFunctionProgressHandler):
  def on_iteration(self, x): print("iteration:", x)

slow_function(1, 2, 3, event=MyProgressHandler())

This also works well with type annotations!

Using generators to yield progress messages might sound elegant, and can be appropriate, but in practice runs into various problems:

• this strategy won't work well if the function is already a generator for some other reason
• using return in a generator is possible, but quite tricky in practice
• calling the function to extract the progress messages is quite tedious

To illustrate the last point, you'd have to do something like this:

def slow_function(*arguments):
  state = init(arguments)

  for x in whatever():
    state += x
    yield ('iteration', x)

  yield ('final',)
  return state

progress = slow_function(1, 2, 3)
result = None
while True:
  try:
    message, *args = next(progress)
  except StopIteration as e:
    result = e.value
    break

  if message == 'iteration':
    ...
  elif message == 'final':
    ...

Consuming a generator that can return is extremely tedious. If the generator doesn't return you can simplify slightly by using a for-loop, but you still need to manually handle different different events. Only when there's a single type of progress event do you get similar or lower complexity than callback-based solutions.

Answer 2

The function you are showing us calls at least 4 other functions. So together with the calling one this makes 5.

This looks like a perfect candidate for building an encapsulating class around it.

Just pass the required callback (or callbacks) through the constructor of that class to store it in a member variable. This way, do_something can use the callbacks without a changed signature. Of course, the way you call it changes a little bit, since now you have to create an object of the new class first, but you can do the object creation (with the callback passing) decoupled from the calls to do_something.

Answer 3

This is IO. Sure, you're trying to make it sound like something else, but this is IO.

There is nothing about it that is 'not pythonic'. You could do this with print statements. It is, however, less than purely functional. You're asking for a side effect. However, it is completely object oriented.

def do_something(out=Null()):
    start_with_something()
    out.update()

    for x in iterate_something():
         out.iteration()

    do_something_expensive()
    out.again()

    finish_with_something_else()
    out.final()

    return the_nice_result

You can disable out by using a null object pattern version of out that does nothing when called. This makes for a good default.

Key here is that these out methods all return void. They work as events.