I’ve challenged myself to work in small steps, supported by tests, integrating new code as soon as I can manage it. I think I see a path. Spoiler: Works very nicely! Lesson: Need less tension, not more!

If the next bunch of articles go like the following, I’ll be satisfied that I’ve met the challenge:

  1. Commits come no further apart than 2X20, 2 times 20 minutes, ideally well under 20 minutes;
  2. The new code is well-tested, whether by existing or new tests;
  3. The new code is quickly integrated, also 2X20;
  4. Everything that can reasonably use the new code does use it.

The idea is to do this all the way up to actually running cliient-server, everything but the actual network code itself. I’ll of course try to do the network code in the same fashion, but I’m so very inexperienced with that particular area that I want to leave myself a bit of wiggle room.

I think I have a notion that just might work. The overall flow of things needs to go something like this:

  1. The server receives a JSON string representing a collection of requests for entity action;
  2. Somewhere that JSON is decoded into a structure for driving the world;
  3. The world sees a series of requests;
  4. Each request includes an entity identifier and a series of actions;
  5. Each action consists of a verb and zero or more parameters;
  6. Each action invokes a primitive action in the world, step, take, and so on.
  7. In due time, the current knowledge state for each entity in the collection is formed;
  8. That collection is converted to JSON and sent back to the client.
  9. The client decodes the JSON and sends the correct knowledge to each corresponding Bot.

My plan notion is to define the request structure that the world wants to use, test drive that into working, then in or near DirectConnection, begin to use that structure. At some point, that side of things should begin to move from “notion” to “nearly OK idea”, and then I think I’ll be able to elaborate the capability on the world side, plug in a quick JSON encode/decode if I want to, and begin to convert the client side Bot and the nascent Cohort idea according to what the World needs.

What about the request structure? Well, naively, it’s a list of requests. Each request has an entity identifier, the entity we’re trying to process, and a list of actions. Each action has a verb (take, etc) and possibly parameters. In principle, we could just have it be a long nested list

[[id_1, [[take], [step]]], [id_2, [[step]]]]

It should be pretty clear what’s wrong with that notion: it’s too easy to get it wrong and too hard to see what’s wrong when you do. The done thing seems to be to make nested dictionaries, instead, and that’s what we’ll be trying.

Let’s turn to code. I think I’ll create a new test file for the server side, oh let’s call it

class TestWorldRequests:
    def test_hookup(self):
        assert False

And today we see why I always start with the hookup. I created the file and that pseudo-test and the tests did not run. I have to start them the first time after opening PyCharm and I hadn’t done that. So I ran them and they ran green. My test was not found! Why? Well, I had named the file TestWorldRequests and pytest prefers something like test_world_requests. Rename the file, the test fails.

If I had not done that hookup, I would probably have noticed sooner or later, but since I am not religious about making every test go red before it goes green, I might not have noticed for a while. As things stand, we’re good to go.

Now I’m going to sketch a test for World, and my plan is that it needs to process a request in the form of nested dictionaries.

The reason I wanted to start with code was so that I could type something like this:

    def test_imaginary_request(self):
        single_request = {
            "entity": 100,
            "actions": [
                {"verb": "take"},
                {"verb": 'step'}
            ]
        }
        id = single_request['entity']
        assert id == 100
        first_action = single_request['actions'][0]
        assert first_action['verb'] == 'take'

This passes. Commit: initial test for dictionary structure.

Now I want to write a test against World now, using this new thing. I think that might be able to be done in one go.

Rules Committee
According to the challenge I’ve set myself, I need to commit every 2x20 minutes or ideally much sooner. So I need to make this test pretty simple. We’ll just place a bot somewhere and have her take a step.
    def test_one_step(self):
        world = World(10, 10)
        bot_id = world.add_bot(5, 5)
        rq = {
            'entity': bot_id,
            'actions': [
                {'verb': 'step'}
            ]
        }
        world.execute(rq)
        world_bot = world.entity_from_id(bot_id)
        assert world_bot.location == Location(6, 5)

OK, all we need, I think, is execute in World.

    def execute(self, request):
        id = request["entity"]
        entity = self.entity_from_id(id)
        self.step(entity)

Test passes! Commit: Fake it till you make it, step works. Time, counting thinking and writing: 16 minutes.

Now of course we can elaborate the execute to be less fake:

    def execute(self, request):
        id = request["entity"]
        entity = self.entity_from_id(id)
        actions = request["actions"]
        for action in actions:
            self.execute_action(entity, action)

    def execute_action(self, entity, action):
        self.step(entity)

Still fake but more like what we need. Commit: loop over actions.

    def execute(self, request):
        id = request["entity"]
        entity = self.entity_from_id(id)
        actions = request["actions"]
        for action in actions:
            self.execute_action(entity, action)

    def execute_action(self, entity, action):
        verb = action["verb"]
        match verb:
            case 'step':
                self.step(entity)

Green, commit: match on verb step.

Reflection

I have been feeling a bit of pressure, tension, anxiety, about the time. I felt like I had to hurry. However, we have four commits, 22, 7, 4, and 2 minutes ago.

But I don’t like the tension. Maybe I’ll get used to it, but hurrying isn’t good. Slow is smooth, smooth is fast. We may have to adjust the challenge, maybe just kind of focus on small steps and report the results without punishing the programmer, who, after all, is doing the best he can within his limited capability.

That said, we have a request that can be executed, given a suitable dictionary. One of the things we’ll want to do Real Soon Now is to create a builder for those dictionaries.

But part of the objective of this challenge is to get the new code integrated as soon as we can.

Let’s see if we can cause DirectConnection to do step commands using a request dictionary. We have this:

class DirectConnection:
    def step(self, cohort, client_bot_id):
        self.world.command('step', client_bot_id)
        result_dict = self.world.fetch(client_bot_id)
        cohort.update(result_dict)

Well, in for a penny, here goes at 0905 hours:

    def step(self, cohort, client_bot_id):
        rq = dict()
        rq['entity'] = client_bot_id
        step_action = {'verb': 'step'}
        rq.actions = [step_action,]
        self.world.execute(rq)
        result_dict = self.world.fetch(client_bot_id)
        cohort.update(result_dict)

I really had high hopes for that. Tests fail.

Oh. Duh. No such thing as rq.actions.

    def step(self, cohort, client_bot_id):
        rq = dict()
        rq['entity'] = client_bot_id
        step_action = {'verb': 'step'}
        rq['actions'] = [step_action,]  # <===
        self.world.execute(rq)
        result_dict = self.world.fetch(client_bot_id)
        cohort.update(result_dict)

Green. Commit: DirectConnection uses World’s new execute for step operation

I think the commit was about about 0912, so seven minutes after 0905.

Reflection

On the one hand, we have done very little. We have a nearly trivial dictionary structure that can only contain an id and a list containing a step command. On the other hand, we have it wired all the way from DirectConnection through World, keeping the world properly up to date. And on the gripping hand, it seems clear that we can quite readily extend our structure to support all the commands that we have—and probably any command that we might ever have.

We did it in five commits over a period of about 45 minutes, so an average of 9 minutes between commits.

The new capability is fully integrated into the existing code. When I run the game, displaying the field where 20 bots are arranging things by color, this new capability is bearing the weight of all the steps that all those bots take.

Looking Forward

Commands will clearly be easy to add. On the world side, it seems easy to add a new method to take a list of requests, probably using an outer dictionary { ‘requests’: […]}. If we want to encode the dictionary in JSON and decode it, we can pretty clearly do that.

A bit less clear is this issue: we want Bot.do_something() to return a single request, and currently it calls DirectConnection once per command. But if the world is ready for a list, that shouldn’t be too difficult. I think there should be no difficulty getting the pieces done in small steps, but at this writing I don’t see how to make a smooth transition from DC’s current single step notion to a list notion. No, wait maybe I do.

Maybe it goes the same as we did today: a new method on DC that takes a list, which it iterates similarly to how the World handles its list. (We note that World does not, as yet, have a way to handle multiple requests in one execute, but we can clearly get there.) So, similarly, we give DC its new list operations, fake it, build it up in a few steps over time.

Summary

This went quite nicely, I think. The increased tension is my main concern, but I think I’ve re-learned a few things that will let me relax. “Fake it till you make it”, or, as I like to call it “Programming by wishful thinking”, should help us get the first bit working, and then we can make it less fake.

I feel quite good about where we plugged the new execute into World, and I think we can find similar places, and then those places will be built up to be more and more robust.

And, of course, there’s work to be done so that the program doesn’t explode when we get a bad or missing key in our dictionaries, but all that is just work that can be done in small steps quite easily.

I’m glad I set myself this challenge. I think it has caused me to find a couple of nice places to stand to get the job done, and, with a little experience, I think I can reduce the tension that I felt.

Harping
Why do I keep harping on the tension, you may be wondering. Some of you may feel tension much of the time when you’re programming, though I hope that’s not the case. In my experience, when I feel tension, I hurry, and hurrying causes me to make mistakes, and to take longer steps than would be ideal, with the result that things break in ways that I don’t expect and can’t instantly fix, and that makes me more tense and … SCREEEE … everything falls down.

Tension makes me a poorer programmer than I am capable of being. And that is neither pleasant nor effective. So I want never to feel tense, and when I do, I try to treat it as a sign to slow down. The challenge pushes me to speed up.

I need to find a way to deal with that failure loop. Maybe the rule is that a rollback restarts the clock? We would probably roll back the production code, keeping the tests. Worth considering. We’ll see.

Bottom Line
Darn good start today, lots of commits and a very substantial set of moves in what seems to be a good direction.

I am pleased. See you next time!