The Python “decorator” is less bizarre than metaclasses and might be just the thing for our needs. Let’s try one.
Python introduced the notion of decorators. A decorator is a function or an object that can be invoked to wrap or otherwise change a function or class definition. From the outside, they are rather nice, signified by an at-sign as below:
class Mumble: @property def foo(self): return self._foo
That code allows the developer to write
mumble.foo rather than
mumble.foo() when fetching the
foo from an instance of Mumble class.
class InvadersFlyer(Flyer): @abstractmethod def interact_with_missile(self, missile, fleets): pass
interact_with_missile as “abstract”, which requires any class derived from InvadersFlyer to implement that method.
Generally speaking, decorators are used to wrap the function or class that they prefix in another function or class. The built in decorator
@Timer, which can be used to print the time used of whatever code it decorates, does its job by wrapping the original function inside another function that captures the current time, executes the function, gets the time again and prints a message.
There are two ways to build a decorator. You can define it as a function, or define it as a class. I have chosen the latter approach, as you’ll see below.
Our mission is to write a decorator, currently called @ignoring_decorator in the tests that follow, that we can use like this:
class Base(ABC): def foo(self): raise NotImplementedError def bar(self): raise NotImplementedError def baz(self): raise NotImplementedError def qux(self): raise NotImplementedError @ignoring_decorator(["bar"]) class FooDecorated(Base): def __init__(self): print("Foo init") def foo(self): return "foo" def baz(self): return "baz"
After the decorated class is created, we intend that a call to
bar will be ignored, despite the fact that it should raise a
NotImplementedError via the superclass Base. Note that this is not all the way to our desired spec for ignoring methods: we’ll finish up below after we look at this much.
Class-Based Decorators with Parameters
The decorator above,
@ignoring_decorator, takes a parameter, the list of method names to ignore. As we saw in the other examples above, not all decorators take parameters. And it turns out that they are implemented a bit differently depending on the existence of parameters. Urrrk. Here, we’ll describe only our current situation, a class-based decorator, taking parameters, applied to a class definition.
Our class will be named
ignoring_decorator. The recipe calls for two methods,
__call__. The init will be called with the parameters provided on the @ line, and the call will be called with the new class as its parameter. I’ll explain a bit more below. Our code looks like this:
class ignoring_decorator(object): def __init__(self, names): self.names = names def __call__(self, klass): if self.names: for name in self.names: setattr(klass, name, lambda *args: None) return klass
So, in the init, we just save our incoming list of names. A defensive programmer would check the parameter. I am not a defensive programmer and I trust my developers to test their code, so I do not, I just save the list.
__call__, we will receive the class being defined under the decorator. (If a function were being defined, we would be passed the function.) The class is fully assembled at the point of the call, with all its methods defined. That will be important in a moment.
Now, as written above, our decorator simply goes through the list of names to be made null, i.e. to just return None without complaint, and defines each name to be a function accepting any arguments, returning None.
Therefore, when we return the class—and we must do that—it will have null methods for all the names provided.
In essence, what happens is the same as this:
class FooDecorated(Base): def __init__(self): print("Foo init") def foo(self): return "foo" def baz(self): return "baz" decorate = ignore_decorator(["bar", "baz"]) // create instance decorate(FooDecorated)
our class ignore_decorator is callable, since it implements
__call__, so that works. You may find those lines hard to grasp at first—I did—but it’s really pretty simple.
- More commonly …
Our code in
__call__might create a new class of some kind, containing the class
klassthat was passed in, and return that wrapper class instead. Doing that is beyond our scope at the moment. Perhaps another time.
So we see how our decorator works: It saves up the list of names and when it is applied to the class, just defines those names right into the class’s attributes. The notation is neat and clean. Again:
@ignoring_decorator(["bar"]) class FooDecorated(Base): def __init__(self): print("Foo init") def foo(self): return "foo" def baz(self): return "baz"
There is just one problem: this doesn’t meet our desired spec. Our spec says that with the code above, the
baz function should not be set to null. The method in the class should be used.
Why? We want it that way to avoid what seems like a likely programmer error. We first set up our class to ignore both
baz, and then later we decide to implement
baz after all, and we forget to remove
baz from the list. We want the real method to override our null.
To make that happen, we’ll “just” have to check to make sure that we do not set an attribute that already exists.
Here’s our test:
class TestIgnoringDecorator: def test_class_decorator(self): foo = FooDecorated() print("foo created") assert foo.foo() == "foo" assert foo.bar() is None assert foo.baz() == "baz" with pytest.raises(NotImplementedError): foo.qux()
If we have
baz in the decorator list, the test fails:
Expected :'baz' Actual :None
We’re not surprised: we override our list unconditionally. Since we have the class as fully compiled in
__call__, we can “just” check for the attribute and only set the ones that are absent.
def __call__(self, klass): if self.names: for name in self.names: if name not in klass.__dict__: setattr(klass, name, lambda *args: None) return klass
We have to check
__dict__ directly, because
getattr would search upward and find the
baz in the superclass.
Our test runs. Commit: @ignore_decorator passes tests.
I really had to do a lot of reading and discovery to get this to work. I was unable to find an example of a decorator defined as a class, with parameters, applied to a class definition. Most of the articles on decorators start at the beginning, with the creation of the universe and never quite get to parameterized class-defined decorators applied to classes.
I managed to find a draft of a book that Bruce Eckel apparently never published that had a write up that got me close to what I needed. But I still wanted to put a lot of prints into my test runs, because it wasn’t clear to me what happened and when it happened. Pytest recompiles your tests, but not the classes associated with your tests, so that I couldn’t trace what was happening during the decoration process of my main test class FooDecorated. I wound up with a test that created a class inside the test, like this:
def test_local(self): @ignoring_decorator(None) class Fubar: def action(self): return 4422 print("calling") fubar = Fubar() assert fubar.action() == 4422
You can see one print still in there, and as I worked to figure out what was happening, there were others in the decorator class as well. Once I understood that the decorator parameters go to
__init__ in this case and the class goes in on the
__call__, I was on the way. As you can see, the actual code is quite straightforward, but it took a lot of fumbling and trying things.
I have read, but have not tested, that when you do not have parameters on the decorator, the sequence is different. I think the parameters then go in on the
__call__ but I am not sure.
Let’s sum up. We might be done for the morning, though it is early (0625).
Decorators are not as deep in the bag as metaclasses, but they are deeper than one usually goes. There are plenty of useful decorators already defined, and for most purposes, we can just use them without regard to what goes on behind the curtain. Since I actually feel confident in the metaclass solution, and this one is far less scary, I think we’ll give it a go.
I’ve done enough testing around this topic to be confident that hammering the class
__dict__ attributes in this way does what I want. It’s a kind of automated monkey-patching in a shiny wrapper. But I’m confident that it is solid.
I think that except for needing a decent name, the decorator approach is the one to use for our methods-to-ignore approach to simplifying the code in our Flyer objects. The decorator will declare which methods are not used, and we’ll thus only have to implement the ones that are used. And if we implement them as
raise NotImplementedError in the superclass, we’ll get a clear error if we do it wrong. And, I have promised to write a test for all combinations, and I might actually do it.
We should probably feel a bit of concern about the hammering. One possibility would be to build a superclass or wrapper class of some kind to contain a custom-made attributes dictionary. At least one of the earlier examples did something like that. We’ll leave that for another time, but it does seem that learning more about decorators would be interesting and possibly useful. For now, we’ll take a break.
I think I’ll start working toward using the decorator scheme in upcoming articles, beginning with a decorator with a better name, defined on the production side.
See you then!