Decorator Factories (and decorator factory factories)

Decorators are a simple, handy way of implementing certain features in any language, but many people find them perplexing. When you start talking about functions that generate decorators, it gets a little twisty; when you then make helpful functions to create functions that create decorators, you might have to stop for a minute to unclench your brain. Really, though, the concept is quite simple if you take it step by step, so here it is (after the jump), hopefully demystified.

Decorators


For the uninitiated, a decorator is a function that replaces another function, adding extra functionality along the way. Here is a trivial example in Python:
def squared(f):
    def inner(*args):
        result = f(*args)
        return result**2
    return inner

def timestwo(n):
    return n*2

timestwo = squared(timestwo)
squared() accepts a function and returns a function, which is the fundamental characteristic of a decorator. We then decorate timestwo() by passing it through the decorator and replacing the reference to the original function with the result. After all this has executed, timestwo is now actually the closure inner, which keeps a reference to the original timestwo in the decorator's scope so it can call it and mess with the result.

In Python you can use nice decorator syntax:

@squared
def timestwo(n):
    ...
Which is synonymous with the manual call to the decorator and reassignment.

Decorator Factories


So let's say you want a more flexible decorator that can raise to any power, not just 2. We could individually define squared(), cubed(), timed() and so on, but it's much easier just to create a function that returns the decorator we need. We'll make a decorator factory, raisetopower, that returns a decorator, decorator, that returns a function, inner, which will replace the decorated function. Again, we'll use closures all the way down, so we keep all the information we need in scope:
def raisetopower(exponent):
    def decorator(f):
        def inner(*args):
            result = f(*args)
            return result**exponent
        return inner
    return decorator
Since raisetopower(2) will return what is essentially squared() from above, we could do:
squared = raisetopower(2)
cubed = raisetopower(3)
But it is much easier just to do:
@raisetopower(2)
def timestwo(n):
    return n*2
This may look strange to some, but bear in mind that it is functionally identical to:
timestwo = raisetopower(2)(timestwo)
And since raisetopower(2)==squared, we've recreated the same situation we had above, only we can now pass in any exponent we want.

Decorator Factory Factories


Naturally the same technique, creating and returning a closure wrapping the original function, can be nested indefinitely; at some point it becomes unreadable. Personally, I would never have multiple calls on the same line, so I draw the line at actually calling decorator factories. I have, however, found uses for decorator factory factories, which you would need when you wanted a reusable decorator factory with some information frozen inside. For example, in txrestapi, I needed decorators that tied a function to both a request method and a regular expression, like:
@GET('^/.*$')
def onGetAnything(self):
    ...
GET itself is just a decorator factory, but I defined GET (and POST, and PUT, and DELETE) using a decorator factory factory:
def decorator_factory_factory(method):
    def decorator_factory(regex):
        def decorator(f):
            def inner(*args, **kwargs):
                # do stuff with f(*args, **kwargs)
                # and method and regex
            return inner
        return decorator
    return decorator_factory

GET = decorator_factory_factory('GET')
PUT = decorator_factory_factory('PUT')
Now, I could have used the function to do:
@decorator_factory_factory('GET')('^/.*$')
def onGetAnything(self):
    ...
But as I said above, you reach a point where it's totally unreadable, so draw the line appropriately.

So that's that: decorators, and functions that create decorators, and functions that create functions that create decorators, all unpacked. Really it's just the same technique, replacing a function with a closure wrapping that function, over and over.

4 comments

  • Anonymous  
    September 9, 2010 at 7:39 PM

    Great article!
    It helped me to finally understand the difference between decorators with and without parameters in python.

  • Kevin H  
    January 19, 2011 at 5:24 PM

    Indeed. A fantastic explanation!

  • Anonymous  
    May 12, 2013 at 3:20 AM

    So... why can't you just put multiple parameters in your decorator factory, instead of adding another level?

    For that matter... why can't you just add a second parameter to your decorator, rather than creating a "factory"? Some thing like:

    def decorator(exponent, f):
    [...]

    And then you can do "timestwo = decorator(2, timestwo)", or something like that

  • Ian McCracken  
    May 15, 2013 at 11:14 AM

    You certainly can do that. It totally depends on the situation. There are cases where you might want to create enough reusable decorators that a factory makes more sense.

    For example, say you're writing the (useless) library "pyexponent", which provides decorators for raising things to various powers. You might certainly provide your example, @raiser(2), but you might also want to provide squared, cubed and so on, for those coders who I guess understand math so poorly that they need a library to square things. There's also an argument to be made that the name of the decorator has semantic value and can be considered documentation of a kind; it might be way more readable just to say squared=raised(2) at the top of your file and use @squared throughout.

    Certainly for a case where the arguments to your decorator factory aren't as trivial as an integer, refer to some initialized value or something expensive to create, you would want to freeze that in a closure rather than recreate it every time -- it may not even be possible the other way. For example:

    @storemetric(timer_func, open('/tmp/time-metrics', 'w'))
    def some_func():
    ...

    vs.

    @timed
    def some_func():
    ...

    The latter would, in this scenario, keep the open file around for writing metrics (file object stored in the closure), saving a ton of time. The decorator would also then transparently handle flushing, reopening the file, etc. That's not possible in the former scenario.

    Point is, this is just a standard technique for freezing parameters in a function instance for reuse later. There may be cases where it makes sense, and others where it would just be simpler not to do it.

Post a Comment