Python Asteroids+Invaders on GitHub

Never heard of him. Let’s learn about decorators, using tests.

What I propose do do in this article, and as many others as it may require, is to learn about Python decorators, and explain both how I go about learning, and what I learn, as best I can. We’ll learn by writing tests and making them work.

I think that to do this, we should start in a new PyCharm project. A bit of fiddling, and cribbing from other projects, and I have a test file and configuration set up. My first test:

class TestDecorators:
    def test_hookup(self):
        assert 4 == 2+2

This is passing. We’re on the road. Let’s see, maybe we need a sort of plan. I’ll start with something easy, of course, and then do more complicated things, and different things. I’ll try both forms of decorator, the ones built with a function, and the ones built with a class, and I’ll try both kinds with and without parameters. And, assuming I can remember all these things, we’ll decorate both functions, and classes.

Let’s try wrapping a function. I propose a decorator @repeat, that given a function, returns a function that calls the input function twice. Then we’ll extend it to @repeat(n). I stole this idea somewhere but I’ll start with my own code. (Turns out I won’t finish with my own code, as you’ll see.)

Let’s try a test:

    def test_repeat_twice(self):
        def repeat(*args):
            print(args)
        count = 0
        @repeat
        def increment():
            nonlocal count
            count += 1
        assert count == 0
        increment()
        assert count == 2

We get some error and find out that args is the function increment. Let’s revise and extend:

    def test_repeat_twice(self):
        def repeat(f):
            def rpt():
                f()
                f()
            return rpt
        count = 0
        
        @repeat
        def increment():
            nonlocal count
            count += 1
        assert count == 0
        increment()
        assert count == 2

This passes. Let’s see what we have done. The repeat function takes a single parameter, f, which we have learned is the function to be repeated. repeat defines a new function, rpt that calls f twice. I am pleased to see that f is known inside rpt. I’m never quit clear on what Python’s rules are about that. Anyway the test passes. Let’s extend it a bit with a new function, also repeated, just to show that the wrapping works for different functions. We’ll do a string this time:

    def test_repeat_twice(self):
        ...

        s = ""
        
        @repeat
        def add_a():
            nonlocal s
            s += "a"
        add_a()
        assert s == "aa"

Still passes. Now let’s try to give repeat a parameter. We’ll do a new test for that, and things get weird:

    def test_repeat_with_parameter(self):
        def repeat(times):
            def rpt(func):
                @functools.wraps(func)
                def wrapped():
                    for i in range(times):
                        func()
                return wrapped
            return rpt

        count = 0

        @repeat(3)
        def increment():
            nonlocal count
            count += 1
        print("increment", increment)
        count = 0
        increment()
        assert count == 3

The parameterized case works quite differently: it’s one level deeper in wrappers. First of all, the parameters are sent in on the initial call to the decorator, where in the no-parameters version they send in the function. Then they send the function in to the first-level inner function that repeat returns, rpt in our case, where in the no-parms version they sent in nothing. The inner function, rpt, must now return the wrapped function that does the work. It pushes everything down one level.

In the code above, I’m using the functools.wraps to create the wrapper function, because I cribbed that code from RealPython or someplace like that. Let’s see about doing the wrap manually. I gather that wraps does some internal gyrations to make the returned function look somewhat normal. We’ll explore that in a moment. Let’s see if we can do the wrap now, by hand.

We totally can, just commenting out the wraps still passes the test:

    def test_repeat_with_parameter(self):
        def repeat(times):
            def rpt(func):
                # @functools.wraps(func)
                def wrapped():
                    for i in range(times):
                        func()
                return wrapped
            return rpt

With the wraps removed, increment prints like this:

increment <function TestDecorators.test_repeat_with_parameter.<locals>
.repeat.<locals>.rpt.<locals>.wrapped at 0x1025b1c60>

With wraps in place, we get this:

increment <function TestDecorators.test_repeat_with_parameter.<locals>
.increment at 0x1037ddc60>

Well, at least it’s called increment instead of repeat.<locals>.rpt.<locals>.wrapped at 0x1025b1c60>. I guess that’s better.

Commit: function-based function decorators, with and without parms. I might put this in github, haven’t decided yet.

Let’s reflect and see whether I can understand this.

Reflection

Today’s work has been for decorators implemented as functions. We’ll look at decorators implemented as classes later.

In a simple decorator with no parameters, the decorator function is called with the function to be decorated, and you wrap it in whatever you want to do with it, repeat it, log it, time it, pat it, prick it, mark it with a D, whatever.

In a parameterized decorator, the decorator function is called with the parameters, not with the function to be decorated. It returns another function that will be called with the function to be delegated, putting us one level further down. That returned function is passed the original function and wraps it (basically just like the simple case, just one call deeper) and returns that function, which will be used in place of the original.

And, because it cleans up prints and such, it is the convention to use the functools.wraps capability, which (I gather) just sorts out some naming. Read about wraps at your peril.

I would think we could and should use wraps in the simple case as well, but most of the examples I’ve found seem not to do that.

My current p-baked conclusion is that when you need a decorator of your own, you just follow the recipe and try not to think about it. The recipe works and a function returning a function that returns a function is more than a little hard to think about.

Summary

I don’t feel that I can explain these decorators in a truly coherent way, yet. The basic one is really pretty simple, and even there I’m not sure yet just what I’d say. We’ll keep working on this and see if we can get the definitive writeup of decorators. It could happen.

See you next time, for decorators implemented with classes.