Decorator Factories (and decorator factory factories)
written by Ian McCracken
at Wednesday, September 8, 2010
Decorators are a simple, handy way of implementing certain features in any language, but many people find them perplexing. When you start talking about functions that generate decorators, it gets a little twisty; when you then make helpful functions to create functions that create decorators, you might have to stop for a minute to unclench your brain. Really, though, the concept is quite simple if you take it step by step, so here it is (after the jump), hopefully demystified.
Decorators
For the uninitiated, a decorator is a function that replaces another function, adding extra functionality along the way. Here is a trivial example in Python:
def squared(f): def inner(*args): result = f(*args) return result**2 return inner def timestwo(n): return n*2 timestwo = squared(timestwo)
squared()
accepts a function and returns a function, which is the fundamental characteristic of a decorator. We then decorate timestwo()
by passing it through the decorator and replacing the reference to the original function with the result. After all this has executed, timestwo
is now actually the closure inner
, which keeps a reference to the original timestwo
in the decorator's scope so it can call it and mess with the result.In Python you can use nice decorator syntax:
@squared def timestwo(n): ...Which is synonymous with the manual call to the decorator and reassignment.
Decorator Factories
So let's say you want a more flexible decorator that can raise to any power, not just 2. We could individually define
squared()
, cubed()
, timed()
and so on, but it's much easier just to create a function that returns the decorator we need. We'll make a decorator factory, raisetopower
, that returns a decorator, decorator
, that returns a function, inner
, which will replace the decorated function. Again, we'll use closures all the way down, so we keep all the information we need in scope:def raisetopower(exponent): def decorator(f): def inner(*args): result = f(*args) return result**exponent return inner return decoratorSince
raisetopower(2)
will return what is essentially squared()
from above, we could do:squared = raisetopower(2) cubed = raisetopower(3)But it is much easier just to do:
@raisetopower(2) def timestwo(n): return n*2This may look strange to some, but bear in mind that it is functionally identical to:
timestwo = raisetopower(2)(timestwo)And since
raisetopower(2)==squared
, we've recreated the same situation we had above, only we can now pass in any exponent we want.Decorator Factory Factories
Naturally the same technique, creating and returning a closure wrapping the original function, can be nested indefinitely; at some point it becomes unreadable. Personally, I would never have multiple calls on the same line, so I draw the line at actually calling decorator factories. I have, however, found uses for decorator factory factories, which you would need when you wanted a reusable decorator factory with some information frozen inside. For example, in txrestapi, I needed decorators that tied a function to both a request method and a regular expression, like:
@GET('^/.*$') def onGetAnything(self): ...
GET
itself is just a decorator factory, but I defined GET
(and POST
, and PUT
, and DELETE
) using a decorator factory factory:def decorator_factory_factory(method): def decorator_factory(regex): def decorator(f): def inner(*args, **kwargs): # do stuff with f(*args, **kwargs) # and method and regex return inner return decorator return decorator_factory GET = decorator_factory_factory('GET') PUT = decorator_factory_factory('PUT')Now, I could have used the function to do:
@decorator_factory_factory('GET')('^/.*$') def onGetAnything(self): ...But as I said above, you reach a point where it's totally unreadable, so draw the line appropriately.
So that's that: decorators, and functions that create decorators, and functions that create functions that create decorators, all unpacked. Really it's just the same technique, replacing a function with a closure wrapping that function, over and over.
September 9, 2010 at 7:39 PM
Great article!
It helped me to finally understand the difference between decorators with and without parameters in python.
January 19, 2011 at 5:24 PM
Indeed. A fantastic explanation!
May 12, 2013 at 3:20 AM
So... why can't you just put multiple parameters in your decorator factory, instead of adding another level?
For that matter... why can't you just add a second parameter to your decorator, rather than creating a "factory"? Some thing like:
def decorator(exponent, f):
[...]
And then you can do "timestwo = decorator(2, timestwo)", or something like that
May 15, 2013 at 11:14 AM
You certainly can do that. It totally depends on the situation. There are cases where you might want to create enough reusable decorators that a factory makes more sense.
For example, say you're writing the (useless) library "pyexponent", which provides decorators for raising things to various powers. You might certainly provide your example, @raiser(2), but you might also want to provide squared, cubed and so on, for those coders who I guess understand math so poorly that they need a library to square things. There's also an argument to be made that the name of the decorator has semantic value and can be considered documentation of a kind; it might be way more readable just to say squared=raised(2) at the top of your file and use @squared throughout.
Certainly for a case where the arguments to your decorator factory aren't as trivial as an integer, refer to some initialized value or something expensive to create, you would want to freeze that in a closure rather than recreate it every time -- it may not even be possible the other way. For example:
@storemetric(timer_func, open('/tmp/time-metrics', 'w'))
def some_func():
...
vs.
@timed
def some_func():
...
The latter would, in this scenario, keep the open file around for writing metrics (file object stored in the closure), saving a ton of time. The decorator would also then transparently handle flushing, reopening the file, etc. That's not possible in the former scenario.
Point is, this is just a standard technique for freezing parameters in a function instance for reuse later. There may be cases where it makes sense, and others where it would just be simpler not to do it.