Duck checker in Python: does one exist?

Question

Python uses duck-typing, rather than static type checking. But many of the same concerns ultimately apply: does an object have the desired methods and attributes? Do those attributes have valid, in-range values?

Whether you're writing constraints in code, or writing test cases, or validating user input, or just debugging, inevitably somewhere you'll need to verify that an object is still in a proper state--that it still "looks like a duck" and "quacks like a duck."

In statically typed languages you can simply declare "int x", and anytime you create or mutate x, it will always be a valid int. It seems feasible to decorate a Python object to ensure that it is valid under certain constraints, and that every time that object is mutated it is still valid under those constraints. Ideally there would be a simple declarative syntax to express "hasattr length and length is non-negative" (not in those words. Not unlike Rails validators, but less human-language and more programming-language). You could think of this as ad-hoc interface/type system, or you could think of it as an ever-present object-level unit test.

Does such a library exist to declare and validate constraint/duck-checking on Python-objects? Is this an unreasonable tool to want? :)

(Thanks!)

---

Contrived example:

rectangle = {'length': 5, 'width': 10}

# We live in a fictional universe where multiplication is super expensive.  
# Therefore any time we multiply, we need to cache the results.

def area(rect):
    if 'area' in rect:
        return rect['area']
    rect['area'] = rect['length'] * rect['width']
    return rect['area']

print area(rectangle)
rectangle['length'] = 15
print area(rectangle) # compare expected vs. actual output!

# imagine the same thing with object attributes rather than dictionary keys.

Answer 1

Today, python offers a wide spectrum of tests than can be done for testing duck-type coding.

Type Annotations

Implementations

For starters, static, compile time type-checking ability was added to Python 3 in 2015.

Now, there are multiple third-party libraries that provide an implementations for the new type-checking hooks.

1. MyPy — data type-checking only (fulfills Python 3 type annotation capability)
2. PyContracts — same Python 3 type as in MyPy above, plus allows conjunctive value constraint(s), which is like what you are talking about.

MyPy is pretty slick.

It even supports overloaded function parameter constraints, so that a function can have different return value type if it takes different parameter value types, for instance, using @overload decorator.

Examples

If in your example above, 'length' and 'width' values were passed as discrete parameters, like:

def area(length: 'int,>0', width: 'int,>0') -> int:
    pass

then, PyContracts could easily do individual type/value checks.

PyContracts can work with the dictionary you specified too, as-is.

def area(rect: 'dict(str: int)') -> int:
    pass

---

Problems with PyDBC

Though it was slick for its time, nowadays, the drawbacks I see with PyDBC are:

1. There is a slight performance hit to running these precondition/postcondition tests. They're dynamic, run time tests. Python 3 type annotations are done using static analysis.
2. Rewriting code at runtime, for one purpose, in a one-off way, that is scattered throughout your program, can be too brittle to be worth the bargain. Works fine with one use. What if some other technique/tool does that too? One can get into conflicts where the only way to get out of them is to drop one tool or another after depending upon it a long time. Cleaner ways to do this have evolved for Python since PyDBC was invented.
3. Explicit declarative constructs, like Python function decorators and Python 3 type annotations, are going to be instantly recognizable/understood by any programmer familiar with Python. They won't spot the implicitly invoked functions. Once they are told, they may forget, may copy or rename the primary function and forget the precondition/postcondition sidekicks. Automated tools are going to treat those extra 2 functions as independent, when they intent is that they are not.
4. The metaclass attribute feature of Python 2 no longer works in Python 3. It is ignored. Instead, metaclass= is specified in class declarations in Python 3. Nowadays, new projects start in Python 3 and Python 2 is legacy.
5. Python 3 has reworked the type system slightly going from early Python 2 to recent Python 3. A Python 2 metaclass-based solution may not work with Python 3 without rewriting/retesting it.
6. Metaclasses are not module-wide now, can only be specified per class, and there can only be one metaclass per class. So it is a preciously applied feature of a class.

---

Unit Testing

To take things farther, into the realm of checking post-conditions, which is appropriate for testing in the fashion of TDD or BDD, one might be better off writing unit tests—particularly if the desire is to test side effects.

TDD unit tests with unittest

Python comes with a unittest package that works fine for verifying external behavior effects of a class are correct.

https://docs.python.org/3/library/unittest.html

It's very handy to use. If you want to test all of your boundary cases (like dimensions<=0), normal case, and any exceptions it supports in the future, it's the right place for it.

BDD testing

There are some third-party programming tools for BDD (behavior-driven design). Which ones are popular/supported will probably change over time.

In my opinion, BDD test scenarios tend to be easier to read/understand. They're readable, possibly even slightly extendable, by non-Python programmers. The step language is pretty simple.

doctest comments/tool

Alternatively, one could include executable examples in the area function's doc string and test it by running doctest—a standard feature of Python for far more than a decade.

https://en.wikipedia.org/wiki/Doctest

By using doctest, tests would be part of the documentation, inline and generated, as both useful examples and convenient tests.

Those are appropriate ways to check that external state/logic of the function is working properly, given fully met preconditions. They tend to offer different side benefits as well.

---

Decorators

One can also use Python 3 decorators to declare preconditions and post conditions. The @precondition would be checked on entry. The @postcondition would be checked on exit.

This would allow you to enforce design by contract (DBC) with very little code at all.

To illustrate this listing 10, the @require decorator sample function, in this Dr. Dobb's magazine article, which shows how to implement a powerfully declarative, yet fully reusable, precondition-checking decorator in just 10 lines of regular Python code.

http://www.drdobbs.com/web-development/python-24-decorators/184406073

You could write a post condition verification decorator based on this @require example, and name them @precondition and @postcondition.

Just be careful. The example uses eval, and there is no parameter sanity-checking or constraining, so someone might put something. Really awkward. Of course, that's a possibility other places as well, but I feel like I should raise awareness since eval/exec uses should always kept in mind as a concern.

---

Python 3 includes a library called functools that actually offers 2 capabilities handy for your problem.

https://docs.python.org/3/library/functools.html

1. It has a decorator that provides a LRU (least recently used) cache for memoization of a decorated function's return values: @functools.lru_cache
2. It eases writing decorator functions.

On the 1st point, that strikes your need to test the memoization feature; supplied, pretested functionality. Memoization is a very handy efficiency tool. In addition to reusing existing code, using lru_cache decorator is going to make your area function much more transparent.

Anyone would be able to see that there is memoization (results caching) going on, without needing to read all of its source code, just by seeing the lru_cache decorator. They would know it's tested code too, not some embedded, one-off logic.

For the second, if you needed something more robust, complex, or implementation specific than the Dr. Dobb's example, functools may speed your efforts.

Particularly, you should look at the @functools.wraps decorator. It not only saves a couple of lines of code, but makes the metadata more intuitively correct an useful.

---

Error-Proofing

The following suggestions made above will make the example correct, less error-prone as-is not to mention if modified, easier to read/understand its usage and behavior, well-tested.

• Removing the area result-caching from the rect: dict parameter gets rid of calculation/state error. Replacing it with the standard library's @functools.lru_cache decorator has none of the drawbacks and virtually all of the benefits. The performance savings is still there. A function should do one thing, and do it well. Callers could store the value returned by the lru_cache memoized area function in an object if desired. In fact, an object could store it in an instance variable. It could even do so in a property read by a getter, fulfilling the DRY (don't repeat yourself) principle. So nothing good is being lost or precluded by extracting the memoization, making it an extrinsic quality. Separation of concerns is key to designing good, reusable, reliable, testable functions.
• For generic handling, like record-processing, a dict (or tuple/obj) makes sense. For something like this, explicit/discrete parameters are more easily understood/tested because they're self-documenting. The new type annotation syntax is simple & efficient for authors to read/change anytime, and tools to verify when developers feel like it.
• Docstring with doctest compatible examples and unittest test cases will make sure the function works, and stays working.
• Writing @precondition and @postcondition decorators is easy to do and can make the code more readable.

Answer 2

It sounds like you want class invariants. That link (a deferred PEP) is the first Google search result for "class invariants in Python". The second is a link to an existing library: PyDBC. I haven't used it before, but it appears to do what you're after.

import os
os.environ['PY_DBC'] = 'true'
import dbc

class Rect:
    __metaclass__ = dbc.DBC
    def __init__(self, length, width):
        self.length = length
        self.width = width
        self.area = length*width
    def __invar(self):
        if hasattr(self, "area"): # __invar gets called during __init__ too :(
            assert self.area == (self.length * self.width)

r = Rect(4, 5)
r.length = 123

Answer 3

Remember that duck typing is really just an (implicit) promise to implement certain methods, with the agreement that the call will fail at run time rather than at compile time. Because of that, a ducktyper would really just be an explicit promise - which isn't really that different from an interface.

I know that zope has an implementation of interfaces - http://wiki.zope.org/zope3/WhatAreInterfaces

Although I haven't used it personally