Is ad-hoc polymorphism a good practice in functional programming?

Question

I am developing a utils data engineering package in python, and for the sake of reusability and readability, I chose the functional programming (FP) approach.

Assume a key task of converting data from types a, b, c to type X. I write three functions from_a_to_x, from_b_to_x, and from_c_to_x.

Is it a good practice in FP to have a to_x function such as:

from typing import Union
from mypackage import from_a_to_x, from_b_to_x, from_c_to_x
from package_a import a
from package_b import b
from package_C import c

def to_x(data: Union[a, b, c]) -> x:
    """Converts data types a,b,c to x"""
    if isinstance(data, a):
        return from_a_to_x(data)
    elif isinstance(data, b):
        return from_b_to_x(data)
    elif isinstance(data, c):
        return from_c_to_x(data)
    else:
        raise TypeError("Unknown data type")

Or, this to_x function is unnecessary and the users should themselves handle the polymorphic behavior in their application?

Answer 1

It seems to be a common cargo cult today to ask if something is "a good practice". Usually, this is the wrong question, since in programming there is almost nothing "good" or "bad" per se - it always depends. So which "cons" and "pros" do you get by implementing a function like to_x in the shown way (compared to the alternative of not providing this function at all)?

Pros:

• it is simple
• you don't have to care for potentially unwanted side effects by modifying existing packages, classes or objects
• code which is implemented in terms of to_x works generically on all supported types

Cons:

• to_x violates the Open-Closed principle, which means, whenever the conversion logic needs to be extended for another type, you have to change to_x.

Is this a problem? This depends on how much genericity your code requires. When your goal is to put to_x into a package X, your generic code using to_x lives in another package Y, and you want to be able to introduce a new type d to work with Y without changing X, then this may become a problem, especially when all those packages have different maintainers. In this case, an OO solution might be better suited. But if that's not your goal, and you just want to use to_x as some "syntactic sugar" for the conversion for a fixed number of types, then go ahead.

The OCP is not an end in itself, it is a means to an end, and for lots of real-world situations following it can lead to overengineering.

Answer 2

Statically typed functional programming languages often solve this situation with a type class. For example, Haskell has a Show that converts type instances to a string, similar to your to_x if x was a string. The nice thing about type classes is you don't require one central function that knows about all the individual implementations.

So ad hoc polymorphism is not only a good practice in functional programming, many FP languages have more expressive ways of doing ad hoc polymorphism.

However, you're not using one of those languages, so you should use the most expressive mechanism that your language provides. That means something like Caleth's answer. Object-oriented classes do not conflict with a functional style. They can work very nicely together.

Avoiding OO classes at all costs is more of a procedural style than a functional style. If that's what you're going for, that's fine, but you're going to have situations like your to_x where it doesn't seem to fit well with the language.

Answer 3

This is actually a pretty common thing to do in Python libraries. I'd say it's an approach that works in practice but not in theory. The biggest challenge it is not always obvious what types such methods take but you've mitigated that with typing hints to some level.

One thing you can do to eliminate the if-else chaining is to use a dict, like so:

converters = {
    a: from_a_to_x,
    b: from_b_to_x,
    c: from_c_to_x,
}


def to_x(data: Union[a, b, c]) -> x:
    try:
        return converters[type(data)](data)
    except KeyError:
        raise TypeError("Unknown data type")

Alternately, if you need to support subtypes, you can do this:

def to_x(data: Union[a, b, c]) -> x:
    for typ, converter in converters.items():
        if isinstance(data, typ):
            return converter(data)

    raise TypeError("Unknown data type")

The challenge with that, though is the order of the dict items might change your results.

The big advantage of this is that you can 'register' new converters at runtime instead of having to modify the code. Of course, your type hints won't be of much use if you allow for that. Personally I still prefer this over the if-else chain, all things equal.

Answer 4

Polymorphism like this is painful in Python because it lacks function overloading.

Ideally you'd have overloads of to_x for each of a, b and c, and Union would dispatch to the correct overload when you use it, but that isn't Python.

If you want to avoid lots of isinstance, you could add to_x to each of a, b, c as a member

from typing import Union
from package import from_a_to_x, from_b_to_x, from_c_to_x

a.to_x = from_a_to_x
b.to_x = from_b_to_x
c.to_x = from_c_to_x

def to_x(data: Union[a, b, c]) -> x:
    return data.to_x()

# can call data.to_x() instead of to_x(data)

But you seem to be insistent on an isinstance approach. Might I suggest iterating your ifs

_to_x = [ 
    (a, from_a_to_x),
    (b, from_b_to_x),
    (c, from_c_to_x)
]

def to_x(data: Union[a, b, c]) -> x:
    for typ, func in _to_x:
        if isinstance(typ, data):
            return func(data)
    raise TypeError("Unknown data type")

Answer 5

Sounds like a job for functools.singledispatch. I just learned about type classes literally 20 minutes ago, and went looking to see if python could support them. I found singledispatch (which is almost like typeclasses, almost) and this post.

https://docs.python.org/3/library/functools.html#functools.singledispatch

This decorator allows you to write your library-specific implementations, while allowing them to be extended:

from typing import Union
from functools import singledispatch
from maaniB import from_a_to_x, from_b_to_x, from_c_to_x
from package_a import a
from package_b import b
from package_C import c

@singledispatch
def to_x(data) -> x:
    raise NotImplementedError(f"Unhandled data type: {type(data)}" 

to_x.register(a, from_a_to_x)
to_x.register(b, from_b_to_x)
to_x.register(c, from_c_to_x)

The great part about this is that it allows me, a consumer, to implement my own types, without any input from you, the library maintainer:

from maaniB import to_x

@to_x.register
def int_to_x(i):
    return whatever(i)

class MyOwnWidget:
    def to_x(self):
        return f"MyOwn({self})"
    
to_x.register(MyOwnWidget, MyOwnWidget.to_x)

If you can use python >=3.8, you can also use singledispatchmethod.

Why is this better?

To understand the advantages of this approach, we first have to briefly consider what the alternatives are. The first is inheritance-based polymorphism. If you wanted, you could write the elif isinstance tree in a OOPy way.

In python, the difference between Widget(foo).to_x() (instance method with self) and Widget.to_x(foo) (the unbound method) is very small. So as long as you control type a, you can just write a A.to_x() method. If there is significant similarity, you can often accomplish this behavior with a mixin. This is "early binding". The one great thing about this approach is that it facilitates generic interfaces and static type checking.

You can provide a AbstractBaseClass and let the user implement their own to_x but this tends to lead to the banana-gorilla-jungle problem with more sophisticated interfaces.

However, it sounds like from your setup that package_a through _c are not actually libraries you control. Hence why you end up with the function with the manual type dispatch. As Doc Brown points out, this could be fine if the set of types you wish to handle is small - e.g. you just want to deal with numpy arrays and pandas dataframes. By using runtime dispatch, you don't have to modify the actual library code. This is "late binding". This works great when you are converging on a single concrete type (say you are returning str), but it gets more tricky if the return type is parametric. Say your to_x is tasked with converting all custom types to Union[str, int, float, dict, list] so you can JSON serialize it later. But you don't know whether the root object is a Dict or List until you get your actual object. You need to essentially loosen the return type so that the type checks out. But now you have this abomination like -> Union[Dict[str, Jsonable], List[Dict[str, Jsonable]] (where Jsonable is roughly Union[list, dict, int, float, str, type(None)]. Also lets say a new type arrives and the sensible interface is to return a primitive, now you have -> Union[Jsonable, Dict[str, Jsonable], List[Dict[str, Jsonable]], it's a hot mess. It would be nice if each implementation could have its own return type, based on the input type. And as mentioned, late binding suffers from downsides relating to OCP.

singledispatch mostly avoids issues with OCP; it's not quite the same type class behavior as more FP-heavy languages since you have to explicitly register functions. But I would contend that it gives you the best of both worlds - the stronger type guarantees of early binding, but the dynamicism of late binding.

Answer 6

Ad hoc polymorphism is used in FP, but your example isn't really a case where this is the way to go.

Rather, it is a very clear-cut example of an algebraic data type. In Haskell, you'd do it like this:

import MyPackage (fromAtoX, fromBtoX, fromCtoX)
import PackageA (A)
import PackageB (B)
import PackageC (C)

data ABC = Aey A
         | Bee B
         | Sea C

toX :: ABC -> X
toX (Aey a) = fromAtoX a
toX (Bee b) = fromBtoX b
toX (Sea c) = fromCtoX c

Those multiple clauses you can have in Haskell are nothing but alternative syntax for the perhaps more familiar-looking

toX datas = case datas of
     Aey a -> fromAtoX a
     Bee b -> fromBtoX b
     Sea c -> fromCtoX c

Python's Union is essentially expressing the sum type ABC, and the isinstance cases correspond to the different case-matches / clauses. The difference is that in Python the type is always part of a value's runtime representation, so you might as well use it for the branch selection, whereas in Haskell types are normally erased at runtime but you can explicitly distinguish different cases. This has several advantages (including that the compiler can easier check correctness&completeness, as well as optimising the decisions, and you can add further special cases without needing additional types), but it does require that when calling toX, you need to explicitly wrap the value in its dedicated constructor, like

main :: IO ()
main = do
   let myA = ...
   let myX = toX (Aey myA)
   print myX

Real ad-hoc polymorphism is also possible in Haskell, done with type classes (this is perhaps surprising from an OO background); in your example it would be

class XAble d where toX :: d -> X

instance XAble A where toX = fromAtoX
instance XAble B where toX = fromBtoX
instance XAble C where toX = fromCtoX

In this case, there is no need to wrap the values, however this is in practice quite different from how your Python example is structured, because the type class is open, i.e. anybody might later on add new types that can be converted to X, whereas in your case you restrict the allowed types to exactly A, B and C – again, that is like a Haskell sum type, not like a Haskell typeclass.