Are there different ways in Python to decorate class methods that dynamically assign themselves to a dict within the class?

Question

Here's what I'd like to do in the form of working code, since it's difficult for me to explain otherwise:

from typing import Callable, Generic, TypeVar
from typing_extensions import Self

# The type checker (Pylance / mypy) whines about an incorrect
#  type for the first argument to the method (what should be
#  `self`). This TypeVar helps it not do that.
t_CallbackSelf = TypeVar("t_CallbackSelf", bound="HasCallbacks")

class Callback(Generic[t_CallbackSelf]):
    """ Decorates another class's methods, and once that 
        class is instantiated, that class collects all instances
        of this class that have been assigned to that class. """

    # An identifier by which to reference the callback function
    #  without using its assigned name on the class.
    ident: str
    # The function identified by `ident`.
    callback: Callable[[t_CallbackSelf], None]

    def __init__(self, ident: str) -> None:
        self.ident = ident
    def __call__(self, func: Callable[[t_CallbackSelf], None]):
        # This is the function that actually decorates a
        #  classmethod.

        self.callback = func
        # Return self so that we replace the value defined on
        #  the class with this instance.
        return self

class HasCallbacks:
    """ Collects all its instances of Callback in a dict upon
        initialization. """

    callbacks: dict[str, Callback[Self]]

    def __init__(self):
        self.callbacks = {}

        for key in dir(self):
            val = getattr(self, key, None)
            
            if isinstance(val, Callback):
                self.callbacks[val.ident] = val
    
    def do(self, ident: str):
        """ Calls one of the callbacks, referenced by its
            identifier `ident`. If none is registered,
            ignore. """
        if ident in self.callbacks:
            self.callbacks[ident].callback(self)
    
    # Decorates the `ping` method to be a `Callback` that has the
    #  identifier "Ping".
    @Callback("Ping")
    def ping(self):
        print("Pong")

callmeback = HasCallbacks()
callmeback.do("Ping") # -> Pong

class AlsoHasCallbacks(HasCallbacks):
    def __init__(self):
        super().__init__()
        self.val = 0

    @Callback("Inc")
    def inc(self):
        self.val += 1
        print(f"Val is now at {self.val}")

callmebacktoo = AlsoHasCallbacks()
callmebacktoo.do("Ping") # -> Pong
callmebacktoo.do("Inc") # -> Val is now at 1
callmebacktoo.do("Inc") # -> Val is now at 2

What bugs me about this implementation:

• Callback and HasCallbacks are tied directly to each other, which to me means they should simply be combined into the same class.
• It feels wrong to be replacing the methods themselves with completely different objects (Callbacks) on the class itself.
• All of the Callbacks are collected every time an instance is made, when it should really only need to happen once and be collected into a class variable.

I feel like there are better ways to do this. I've done a lot of studying on decorators, __init_subclass__ and metaclasses, and I've gone through a couple of iterations on ways to do what I'm doing in the code above (and the above attempt is still the most clean version).

I'd like to know if this is the only way to do what I'd like to do, or I'd like to know the name of the thing I'm trying to do (if there's a single name for it) so that I can look that up and figure out how to do it.

Answer 1

A Python class is created as follows:

• the body of the class is evaluated in a new scope
  • however, the contents of functions defined within this scope don't have access to this class scope
• the local variables are collected as a dictionary
• a new type() is created and assigned to the class name.

So very roughly,

class Foo(Base):
  some(code)

is turned into:

locals = {}
exec("some(code)", locals=locals)
Foo = type("Foo", [Base], locals)

This means that a class body can include arbitrary code. For example:

class Foo:
  for i in range(3):
    print(f"arbitrary code, i={i}")

print(f"Foo.i={Foo.i}")

So the issue can be reduced to: how can we register a bunch of functions in a dictionary?

For example, we might try:

class HasCallbacks:
  callbacks = {}

  def register(name):
    def decorator(fn):
      callbacks[name] = fn
    return decorator

  @register("Ping")
  def ping(self):
    print("Pong")

  del register  # so that this doesn't become a class member

But this doesn't quite work because register() can't see the callbacks variable.

Instead, we could do:

def register(callbacks, name):
  def decorator(fn):
    callbacks[name] = fn
  return decorator

class HasCallbacks:
  callbacks = {}

  @register(callbacks, "Ping")
  def ping(self):
    print("Pong")

Limitations: you can only register the callbacks in a dictionary that is in the current scope. This will not work as expected with inheritance. Also, the class will necessarily contain a ping member because the result of applying a decorator is always assigned to the variable previously containing the decorated function.

To address these limitations, your idea of wrapping relevant items in a custom type is a very good approach. But instead of collecting these marked items in a method, you might do so at class construction time with a custom metaclass.