P-228 - The Matrix

Python Asteroids+Invaders on GitHub

Let’s explore last night’s matrix idea. There’s got to be a pony lesson in here somewhere.

So as I wrote about early this morning, the Zoom discussion led me to think about a hybrid solution that might better balance the tension between the safety of abstract methods and the convenience of inherited pass methods.

The tension is this: when I define all the possible interact_with_thisandthat methods as abstract, it helps ensure that I put those methods in classes that need them, because I have to put them all in, even if they just pass. But if I define them as pass in the superclass but not as abstract, then my classes are less cluttered, and I don’t have to change them all the time, but I tend to miss out on the classes that do need a particular new method.

The idea, refined a bit by just having been thought about a bit, is to have an easy to write, easy to read “matrix” kind of structure that declares, in one place, whether or not a given pair need to interact. There are at least two ways the matrix might work.

In one case, the matrix would be used to drive one or more tests that use the matrix to verify by inspection that each object that should define a method does define it. In the other case, we could actually use the matrix at run time, to dispatch to the correct method directly.

I believe that there are reasons why we should go with the first case, the test that verifies the state of the code. First, we’d be foolish not to have the test even if we were going to use the table to dispatch our calls, because otherwise we might get a run-time error if a method were missing. Second, the dispatch would be far more complex and slower than what we have now, which runs at the speed of internal method dispatch.

So we’ll start with a matrix, aiming at the test. Let’s muse a bit about what the matrix might be.

Design Thinking

It seems clear to me that the structure we’d like to read would be a nice 2D array, with object class on both axes, and the action info in the cells. Unlike, say, FORTRAN, Python does not have two dimensional arrays. Generally you’re meant to have a list of lists. Since we will be indexing by object class (Invader vs PlayerShot), we might find it better to have a dictionary of dictionaries. I’m not aware of any other reasonable possibilities.

I’m more concerned that the thing should be readable. The idea is that it displays in a clear and comprehensive form, all the interactions, class vs class. As a table, it might look like this:

A few notes on that table:

1. It’s incomplete because the Invader class isn’t a flyer but does interact with the bumpers, sort of.
2. It’s not symmetrical in a couple of cases, which makes at least for a surprise if nothing else.
3. I’m not sure it’s correct. It’s so hard to type and as we’ll see in a moment, so hard to read in the source of this article, that I can’t concentrate well on both the formatting and the design issue.
4. It looks like it could be partitioned but in general I don’t think we can count on that.
5. There are objects that appear not to interact with anything. Others detect them, in some cases. In other cases they are completely independent of anything. The explosions are examples of that.
6. It’s hard to create and fill in the table. The source looks pretty nasty:

| | Bum| IExp| IFlt| IPla| ISho| PSho| SCon| SExp| TBum |
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| Bumper | _ | _ | Y | _ | _ | _ | _ | _ | _ |
| InvaderExplosion | _ | _ | _ | _ | _ | _ | _ | _ | _ |
| InvaderFleet | Y | _ | _ | _ | _ | Y | _ | _ | _ |
| InvaderPlayer | _ | _ | _ | _ | Y | _ | _ | _ | _ |
| InvaderShot | _ | _ | _ | Y | _ | _ | _ | _ | _ |
| PlayerShot | _ | _ | Y| _ | _ | _ | _ | _ | _Y|
| ShotController | _ | _ | Y | Y | _ | _ | _ | _ | _ |
| ShotExplosion | _ | _ | _ | _ | _ | _ | _ | _ | _ |
| TopBumper | _ | _ | _ | _ | _ | _ | _ | _ | _ |

As a documentation tool, a markdown table is nice on the output side and takes ages to create and to get right. We could probably generate one as a report by raking through the subclasses and their methods. But proof-reading the thing is difficult as you have to take it cell by cell. We’re looking for a good way to express the relationships easily in the code, resulting in something that’s easy to read.

Conclusions?

None yet but it leaves me with the concern I brought up with the Zoomies last night: I don’t know a good way to express the matrix in readable source code. I know ways to express it, just not in a way that helps me create it or helps me understand it.

Well … I don’t know yet. Perhaps soon, perhaps not so soon, I might know. I’m not giving up on ever knowing, even if I don’t know right now.

I wonder if we could do it line by line somehow. Let’s imagine a dictionary of dictionaries. The top key is the “left-hand” class in an interaction. Invader vs Shot, it’s the invader. The dictionary inside is indexed by the right-hand class, and by the rules must include all the subclasses of whatever interface we’re checking. We could generate the code for that, probably. Let me try it with a test.

    def test_print_dictionaries(self):
        subs = InvadersFlyer.__subclasses__()
        names = [sub.__name__ for sub in subs]
        names.sort()
        for name in names:
            self.print_dict(name, names)
        assert False

    def print_dict(self, name, names):
        print(name + ": {")
        for name in names:
            print("    " + name + ":error,")
        print("},")

I often do little tests like that, to produce some output. Then I remove their final assert or comment them out, but I keep them in case I want them later. This one produces the following code:

Bumper: {
    Bumper:error,
    InvaderExplosion:error,
    InvaderFleet:error,
    InvaderPlayer:error,
    InvaderShot:error,
    PlayerShot:error,
    ShotController:error,
    ShotExplosion:error,
    TopBumper:error,
},
InvaderExplosion: {
    Bumper:error,
    InvaderExplosion:error,
    InvaderFleet:error,
    InvaderPlayer:error,
    InvaderShot:error,
    PlayerShot:error,
    ShotController:error,
    ShotExplosion:error,
    TopBumper:error,
},
InvaderFleet: {
    Bumper:error,
    InvaderExplosion:error,
    InvaderFleet:error,
    InvaderPlayer:error,
    InvaderShot:error,
    PlayerShot:error,
    ShotController:error,
    ShotExplosion:error,
    TopBumper:error,
},
InvaderPlayer: {
    Bumper:error,
    InvaderExplosion:error,
    InvaderFleet:error,
    InvaderPlayer:error,
    InvaderShot:error,
    PlayerShot:error,
    ShotController:error,
    ShotExplosion:error,
    TopBumper:error,
},
InvaderShot: {
    Bumper:error,
    InvaderExplosion:error,
    InvaderFleet:error,
    InvaderPlayer:error,
    InvaderShot:error,
    PlayerShot:error,
    ShotController:error,
    ShotExplosion:error,
    TopBumper:error,
},
PlayerShot: {
    Bumper:error,
    InvaderExplosion:error,
    InvaderFleet:error,
    InvaderPlayer:error,
    InvaderShot:error,
    PlayerShot:error,
    ShotController:error,
    ShotExplosion:error,
    TopBumper:error,
},
ShotController: {
    Bumper:error,
    InvaderExplosion:error,
    InvaderFleet:error,
    InvaderPlayer:error,
    InvaderShot:error,
    PlayerShot:error,
    ShotController:error,
    ShotExplosion:error,
    TopBumper:error,
},
ShotExplosion: {
    Bumper:error,
    InvaderExplosion:error,
    InvaderFleet:error,
    InvaderPlayer:error,
    InvaderShot:error,
    PlayerShot:error,
    ShotController:error,
    ShotExplosion:error,
    TopBumper:error,
},
TopBumper: {
    Bumper:error,
    InvaderExplosion:error,
    InvaderFleet:error,
    InvaderPlayer:error,
    InvaderShot:error,
    PlayerShot:error,
    ShotController:error,
    ShotExplosion:error,
    TopBumper:error,
},

Those little dictionaries might not be too difficult to edit into something like this:

Bumper: {
    Bumper:NO,
    InvaderExplosion:NO,
    InvaderFleet:YES,
    InvaderPlayer:NO,
    InvaderShot:NO,
    PlayerShot:NO,
    ShotController:NO,
    ShotExplosion:NO,
    TopBumper:NO,
},

… and so on. Starting with “error” would at least require us to think a little bit about what to type in its place, and we could use a simple YES / NO or True / False or 1, 0, whatever made sense.

However …

What about updating? When we add a new class to InvadersFlyer, and we’re not done yet, we have to add a new top element to the dictionary and a new line item to each existing dictionary, leaving all the other values YES / NO as before and with the new line item saying “error”.

Probably not too difficult, our test could know where the source for the dictionary exists and read it and produce a new copy. We probably can’t make our python tests update the source directly … well, probably we could, but should we? We could at least output a new version if the test found something missing. That would be “easy enough”. A few hours’ work, perhaps.

If these dictionaries were all in a single source module, we could at least edit them in one place. There might be a high probability of a merge conflict if we had a number of team members working on separate new objects or new interactions at the same time. We could deal with that, probably either by making an early call on a new object, and quickly updating, suggesting that the team pull the new module, or wait until we’re ready and do the same except now we pull.

It might be feasible. I can almost feas it.

Decision?

My decision for now is to wait on this idea. And all this thinking leads me to another place, a different framing of my problem.

Reframing

THe matrix idea seems not to bear weight because if is hard to express in readable usable code. Let’s think about the existing abstract method scheme. It has these properties:

1. Because all the possible interact methods are demanded by the abstract method, PyCharm will help us by filling them all into the class, and that give us a perfect opportunity to tick through them and think about each one. It would be nice if PyCharm didn’t implement them as pass but as something that wouldn’t compile., but it’s still not a big hassle to tick through them. It is a bit distracting because usually we’re in the middle of some very trivial early test for the class, and thinking about the big picture is a bit off the table.

2. Adding the new class breaks a lot of other classes, because now they all have to implement the new method, generally as the default pass. That’s distracting, again because we are on some trivial highly focused test and the big picture is off the table.

Because of these issues, when a given object interacts with no one, I’ve been tempted to make interact_withgivenobject non-abstract in the superclass, so that everyone just inherits the pass that they want anyway. There’s less editing, less code in the various subclasses … and, unfortunately, less thinking, because I am not required to tick through each class, and let it implement the required methods, giving me at least the opportunity to think for a moment about that specific class and whether it interacts.

Making the methods abstract requires every subclass to explicitly implement them. There’s no doubt in my mind that that’s safer than inheriting null behavior, because when the object should do something, we don’t get a chance to spot it.

I conclude that I should stop doing the inheritance trick and should make all the interact_with_givenobject methods abstract, and go ahead and implement them in all the classes. It’s tedious but better.

PyCharm has a very nice dialog when it offers to implement abstract methods. It looks like this:

My practice has typically been to select all the options and let ‘er rip. That stuffs pass in everywhere and I then just ignored everything that I’m not thinking about.

But if I would use that dialog on each class after adding a new class, it would just list the individual method and it wouldn’t be that difficult to think, for each class, “does this class interact with givenobject?”. If I’d just slow down a bit and smell the roses, it could work for me instead of against me.

Therefore

I declare that my best idea so far is to use the abstract method approach and stop using the shortcut approach of not declaring new methods abstract just to save a little editing, because it also saves me from thinking about things that may need thinking about.

Therefore, soon, I’ll go find the non-abstract methods and see how awful it is to implement them. And if I go in small steps, maybe one method at a time, my commits will be quick even if they do affect several classes, and I can just shout “out of the pool” and everyone else on the team can pull and merge the new otherwise unaffected classes.

Besides, there’s no one else on the team.

Summary

Sometimes it takes a while for an idea to even seem possibly acceptable. When I was strongly focused on my scheme of “every object for herself”, I couldn’t even consider Hill’s notion of the two-dimensional approach. Today I managed to consider it. This time, rather than reject it as a foreign object in my mind, I reject it for more nearly valid reasons: I haven’t found a way to do it whose cost is commensurate with the benefits.

Instead, I’ve come up with some personal habit changes that will provide greater benefit from what I now do, with just a little bit of adjustment.

Would I have decided to do a better job with the abstract methods without working through these other options? I don’t know and I don’t care. Because now I understand at least a few new possibilities that might have been useful in this program, and still might be, depending on how things go. I’m a little more adept at generating code with code. And I’ve got a small commitment to a small improvement in how I work.

Well worth the thinking. And the writing down of it, too, I suspect.

See you next time!