Decorator Classes
The next step with Python decorators is a class-based decorator. As usual I’ll take a quick look at things on the Web and then go my own way. Results are ‘interesting’: I need more reading, and what I’ve learned is captured in a test.
Scanning the RealPython decorators tutorial, I learn that a class intended to be used as a decorator needs to implement __call__, so that it can act as a function. The RealPython tutorial is using functools by this point, and while I think I’d like to see how they help, I want to do my own wrappers for a while. I’ll keep searching a bit.
I find something on builtin.com. Don’t love it, but I glean that, like the implementation via a function, a class-based decorator will be called with the wrapped object if it is used without parameters, and if it is used with parameters, it is expected to return the wrapper function, which will be called with the decorated object.
I’m forming a model that says it’ll be just the same as with a function, just that we have a __call__ method, which is enough to allow Python to treat class-based and function-based decorators the same way.
I want to scan a bit more. I’m not quite ready to try to code one up.
Found something on Geeks for Geeks. Not quite pleasing but it’s confirming what I’m thinking.
As I write the above, I am tempted to start testing and coding now. I think I could write a series of spikes and tests that would get me where I need to be, but I’d still like to find an example that satisfies me a bit more than these examples have.
This discussion on python.org was quite interesting, but not quite on point regarding class-based decorators.
Oh! I found something on python-course.eu. It starts very basically introducing nested functions, and works its way up.
The course stops about where they always do, so I think I’m going to just dive in, or perhaps tiptoe in. However, I had not found that site before and it looks very useful. I make a note to explore it further, in my copious free time.
Let’s Just Try It
It took me a few tries, but I have this:
class deco_class_1:
def __init__(self, thing):
print(f'init {thing}')
self.thing = thing
def __call__(self):
print(f'call {self.thing.__name__}')
return self.thing()
class TestDecoratorClasses:
def test_first_example(self):
@deco_class_1
class Decorated:
def __init__(self):
print(f'init {self}')
self.foo = 42
print('create')
d = Decorated()
print('created')
assert d.foo == 42
Note that we are not creating our decorator class with parameters in this case. The test runs, and the output is enlightening:
decorator init <class 'test_decorator_classes.TestDecoratorClasses.test_first_example.<locals>.Decorated'>
create
call Decorated
Decorated init <test_decorator_classes.TestDecoratorClasses.test_first_example.<locals>.Decorated object at 0x10648f860>
created
When the @deco_class_1 is encountered, an instance is created and it is passed the class being decorated (not an instance thereof). Our code prints ‘decorator init’ etc., saves the class away. Whether we are decorating a function or a class, whatever we get will be callable.
Then we print ‘create’ and make an instance of our decorated class. At that point, our decorator instance is called, printing ‘call’ etc., and returning the result of calling the thing passed in, i.e. the Decorated class. this creates and returns the instance.
We print ‘created’ and assert.
Most of this I just typed in, copying the sense of the things I had seen, but I had return self.thing, not return self.thing().
The resulting error message was:
> assert d.foo == 41
E AttributeError: type object 'Decorated' has no attribute 'foo'
That (finally) told me that I was returning the class and the code expected an instance. So I added the parens and we went green. Let me think about this.
Reflection
In my function-based tests so far, I didn’t have to call the wrapped class, I merely returned it. Why the difference?
Ah. The difference is that in the case of the function-based ones what we have is callable, whether it is a wrapper or the original. And Python calls it. Here, we have made our class callable … and Python calls it … but Python thinks it is calling the original wrapped function, so we must call do the call on its behalf.
OK, I think I’ve got it. Now let’s think about a class-based decorator that takes parameters. We can assume—I think we know—that Python will treat it the same as before: seeing the parameters, Python will instantiate the class with those parameters, and will then call the class passing in the wrapped object (and its parameters, which we have not dealt with anywhere, make a note of that).
So let’s do a simple case. This time, just so you can watch me fumble, I’ll show more detail of my steps.
def test_parameterized_decorator(self):
@deco_class_parm(37)
class Decorated:
def __init__(self):
print(f'Decorated init {self}')
self.foo = 42
d = Decorated()
assert d.foo == 42
My thinking here is that I’ll sprinkle print statements as before and work up to what to do. If this gets too detailed, skim.
I start with just this much:
class deco_class_parm:
def __init__(self, param):
print(f'decorator init {param}')
self.param = param
I get the message ‘decorator init 37’, and the test fails:
> @deco_class_parm(37)
E TypeError: 'deco_class_parm' object is not callable
No big surprise there, we haven’t built __call__ yet.
class deco_class_parm:
def __init__(self, param):
print(f'decorator init {param}')
self.param = param
def __call__(self):
pass
This, if I’m not mistaken, will object to there not being enough parameters on __call__.
def test_parameterized_decorator(self):
> @deco_class_parm(37)
E TypeError: deco_class_parm.__call__() takes 1 positional argument but 2 were given
Yes, though it is not obvious yet what we’re getting. I imagine it will be the class Decorator.
def __call__(self, callable):
print(f'call {callable.__name__}')
The prints are much as expected:
decorator init 37
call Decorated
And the error isn’t surprising, since our method __call__ returns None:
> d = Decorated()
E TypeError: 'NoneType' object is not callable
Let’s call and return our callable.
def __call__(self, callable):
print(f'call {callable.__name__}')
return callable()
Prints are not too surprising:
decorator init 37
call Decorated
Decorated init <test_decorator_classes.TestDecoratorClasses.test_parameterized_decorator.<locals>.Decorated object at 0x102a38890>
But the test fails.
> d = Decorated()
E TypeError: 'Decorated' object is not callable
I do not understand this, but I bet that this will pass the test:
def __call__(self, callable):
print(f'call {callable.__name__}')
return callable()()
Wrong! I lose that bet:
def __call__(self, callable):
print(f'call {callable.__name__}')
> return callable()()
E TypeError: 'Decorated' object is not callable
Let’s remove the extra parens and look at the prints again:
def __call__(self, callable):
print(f'call {callable.__name__}')
return callable()
decorator init 37
call Decorated
Decorated init <test_decorator_classes.TestDecoratorClasses.test_parameterized_decorator.<locals>.Decorated object at 0x102455520>
We have created a Decorated instance. But maybe we weren’t supposed to call this time.
def __call__(self, callable):
print(f'call {callable.__name__}')
return callable
The test passes, printing:
decorator init 37
call Decorated
Decorated init <test_decorator_classes.TestDecoratorClasses.test_parameterized_decorator.<locals>.Decorated object at 0x102b3f110>
I don’t quite understand this. I see what it does: we get our class called to create itself with its parameters. On __call__, we’ll be passed the thing being decorated, in this case a class, which is callable, since a class is just a fancy function that creates an instance. Whatever we return from the call is itself going to be called. And it had better return an instance of some class, and that class had better understand all the attributes of the input callable class.
So that’s what it does, but what good is our own decorator class? What should it be doing?
A Wild Experiment
I do not have a good sense of how to use this yet, but let’s try something. We’ll create a dynamic subclass of our input class, and we’ll give it a new method to return our parameter. This will be enlightening, and possibly explosive. I extend the test to expect the class to understand bar():
d = Decorated()
assert d.foo == 42
assert d.bar() == "mumble"
Fails now, of course.
class deco_class_parm:
def __init__(self, param):
print(f'decorator init {param}')
self.param = param
def __call__(self, callable):
class decorated(callable):
def __init__(self):
super().__init__(self)
def bar(self):
return 'mumble'
print(f'call {callable.__name__}')
return decorated
This is way out into fantasy land, and the test still fails. I’m not giving up yet. The error:
def __init__(self):
> super().__init__(self)
E TypeError: TestDecoratorClasses.test_parameterized_decorator.<locals>.Decorated.__init__() takes 1 positional argument but 2 were given
This test passes:
def test_parameterized_decorator(self):
@deco_class_parm(37)
class Decorated:
def __init__(self, unexpected):
print(f'Decorated init {self}\n{unexpected=}')
self.foo = 42
print("create")
d = Decorated()
print("created")
assert d.foo == 42
assert d.bar() == "mumble"
assert False
I just gave up and added a parameter unexpected to the Decorated class’s __init__. The print is this:
decorator init 37
call Decorated
create
decorated init <test_decorator_classes.deco_class_parm.__call__.<locals>.decorated object at 0x106f92030> ()
Decorated init <test_decorator_classes.deco_class_parm.__call__.<locals>.decorated object at 0x106f92030>
unexpected=<test_decorator_classes.deco_class_parm.__call__.<locals>.decorated object at 0x106f92030>
created
So it appears that the same object was passed twice to the init of our decorated class, and we did not expect that.
I am not fully enlightened and I am out of spoons. We have a test that is running, though we do not quite understand what is going on. The test will preserve the facts of the matter, and I will do some noodling and reading, to see if I can sort out what’s going on. But I need a mental break: too many invisible mental balls in the air. Commit: inconclusive tests of class-based decorators.
Summary
On the one hand, what I’ve clearly done here is bash on an example with no real idea of quite what’s needed. Freely granted. But what better way is there to find out what’s going on but to try things? So far, I have not found an example tutorial or example with a class-based decorator definition that takes parameters.
I suspect that the issue revolves around just what we can validly return from our __call__ method. I do think that the subclass was properly returned, since the call to bar() passes, so we’re pretty close to something usable.
Possible parameters on the wrapped class remain to be dealt with but I’ve seen an example somewhere that just accepts and passes on *args,**kwargs, which is Python for “whatever arguments there are”, and when the time comes, we’ll try that. Right now, I’m mostly wondering what that extra copy of the decorated object was.
I think I might do well with better names. ‘Decorated’ and ‘decorated” aren’t quite as different as they might be. And I’m sure I’ll want more prints and more info in them. But mostly, I need a bit of mental down time and then some more reading.
Until then, stay warm, and I’ll see you next time!