2x20: Small Steps

The reason for setting myself this public 2X20 challenge isn’t so that I can be embarrassed in front of everyone. It’s because I need more practice on small steps. The downside of thinking one is particularly smart. Results: Good, but by the skin of my teeth.

2x20

I know quite a few developers who are better than I am, especially in particular areas, but there are plenty of folx I know who are better than I am at one or more of the things I think I’m pretty good at. Even so, down deep, I think I’m pretty smart, in spite of the fact that I also know darn well that I am quite stupid about a lot of things, including, sometimes, programming.

I enjoy the problem-solving, puzzle-solving aspect of programming. There’s real pleasure in figuring out a structure of programming statements that solves some problem. For me, it’s the same pleasure that I get from working out how to build or repair something¹ … and it’s a lot less physical effort.

The upshot of my inner belief that I can sort things out, plus decades of writing lots of code in between tests, plus countless other personality defects, is that I tend to take steps that are often quite a bit too big. When I do that, I will often have made two or more mistakes, with the result that it is four or eight or more times harder to make the code work than if I had made only zero or one mistakes at a time.

When I pair with GeePaw Hill, or observe him programming, I see him finding steps that are much smaller than my natural size, and I see how well it works. When I work that way myself, I see it working well for me. So I want to build up the habit, and the skill required to see a tiny step that leads in the direction we want to go.

Thus the 2X20 challenge to myself, to do the current client-server-focused work in the Robot World such that I never need more than 40 minutes to get to green. Why 40? Because my intuitive grasp of things is that after 20 minutes without a green bar and, ideally, a commit, I’m probably in trouble. Then why not 1x20? Well, because while I am dedicated to showing you all the mistakes I make along the way, I don’t really enjoy the failure aspects of things. So, for now, I’m giving myself double the ideal time. Perhaps, if I manage this, I’ll issue a new challenge, 2x10.

Then again, perhaps not.

One legitimate reason to allow myself extra time is that I write these articles contemporaneously with my programming, and also at the same time. It takes time to put together all these words, and the clock is running.

But the point, really, is to practice finding smaller and smaller steps, in hopes that it will become more natural to me and that I’ll get better at it.

Let’s get to it.

Review

Yesterday we chose to give the World a new method, which has evolved to look like this:

class World:
    def execute(self, request):
        id = request["entity"]
        entity = self.entity_from_id(id)
        actions = request["actions"]
        for action in actions:
            self.execute_action(entity, action)

    def execute_action(self, entity, action):
        verb = action["verb"]
        match verb:
            case 'step':
                self.step(entity)

This code posits a request which, for now, is a dictionary with two keys, ‘entity’ and ‘actions’. The entity key points to the id of an entity. (We should perhaps change the key name.) The actions key points to a list of, well, actions. We see from the execute_action method that the action is also dictionary-like, with at least one key, ‘verb’.

We put this code in via TDD, with a simple test or two, and then we put it into production, creating a request in DirectConnection:

class DirectConnection:
    def step(self, cohort, client_bot_id):
        rq = dict()
        rq['entity'] = client_bot_id
        step_action = {'verb': 'step'}
        rq['actions'] = [step_action,]
        self.world.execute(rq)
        result_dict = self.world.fetch(client_bot_id)
        cohort.update(result_dict)

All this, if it’s fair to call about 20 lines of code “all this”, all this was accomplished in five commits over the course of about 45 minutes, about 9 minutes between commits on the average. I called the day a win.

GeePaw Hill asked me yesterday if I planned to make the individual commands typed, that is, each of a different class. I replied that I probably wouldn’t, and referred to my builder, which is an experiment in building up a structure like a list of these requests with code that is less dictionary-focused on what we see above.

But, I said, we’ll see what the code wants and respond accordingly. Certainly it won’t be fun to write out those request by hand very often … but the production code itself will probably not do much of it. We’ll see.

One advantage to typed actions would be that it would help ensure that my code doesn’t accidentally create a request that can’t be dealt with. As things stand in the execute code above, a malformed dictionary will cause an exception, and We Do Not Want Exceptions. We’ll need to deal with missing keys somewhat, but a better way of building requests will help as well.

I would like to get further along with the request idea before committing to a better structure, but I do know that when I create custom collection classes instead of use language-provided ones, things tend to go better. However, using a builder will provide similar advantages. We’ll see.

Bridges? What?

I often think of small steps as being like I imagine bridge building to have been, back when my ancestors were living in the jungle. When they came to a crevasse and needed a bridge, they would probably start by slinging a single rope (or vine?) across, affixing it to a convenient tree or rock on each side. Then, i don’t know, maybe another one or two. Then they could begin affixing planks or whatever, until finally they had a reasonable walkway.

So, what we have with our single ‘step’ request is a vine slung across between the connection and the world. We can work on enhancing that vine, adding additional actions, and extending it to allow more than one request, in a list or Yet Another Dictionary or something. And on the Bot side, we will want to evolve from the current scheme, which makes single-step calls to DirectConnection. We’ll want the Bot to package up a request. And, when we elaborate our Cohort, the collection of Bots, we’ll want it to create the list or Yet Another Dictionary for a batch of requests.

Small Steps?

We can see some small steps easily: we can readily add the other action verbs to World’s execute, and we can readily modify the DirectConnection to call those. As things stand, they’d be single-verb requests, but they’ll work and exercise all the World’s code.

Then, and I don’t see this perfectly, probably we TDD a new entry point in DirectConnection, to take a list of requests. That should be quite simple. Then, and this is even more foggy, we rig Bot to produce a request dictionary, and we rig Cohort to produce a list of requests, in whatever form me have by then chosen.

I think there are at least two sessions of work there, maybe more. And it seems like the steps can all be small.

OK, we’re a ways in here, we can see we are probably in good shape, let’s do some work.

More Verbs

I think I’d like to start with the rest of the verbs that the Bots can use. We have ‘step’, and there is also ‘take’, ‘drop’ and whatever we do for setting direction. ‘turn’, I think. There’s a tricky bit in turn, if I’m not mistaken. If so, we’ll deal with it.

Let’s just pick one. ‘take’, because it doesn’t need a parameter. We’ll save that for when we’re warmed up. There are at least two ways to go:

1. Write a new test in TestWorldRequests, like the ‘step’ test below;
2. Change DirectConnection to make a ‘take’ request, let tests break, fix World.

The first way is more conservative. I want to do the second way. I think it’s small enough. Let’s try it.

It’s 1031. We start with this:

class DirectConnection:
    def take(self, client_bot):
        self.world.command('take', client_bot.id)
        self.update_client(client_bot)

We replicate the step code:

    def take(self, cohort, client_bot_id):
        rq = dict()
        rq['entity'] = client_bot_id
        take_action = {'verb': 'take'}
        rq['actions'] = [take_action,]
        self.world.execute(rq)
        result_dict = self.world.fetch(client_bot_id)
        cohort.update(result_dict)

We don’t have a cohort for take. I didn’t see that coming. In Bot, we have this:

    def perform_actions(self, actions, connection):
        for action in actions:
            match action:
                case 'take':
                    connection.take(self)
                case 'drop':
                    connection.drop(self, self.holding)
                case 'step':
                    self._old_location = self.location
                    cohort = Cohort(self)
                    connection.step(cohort, self.id)
                case 'NORTH' | 'EAST' | 'SOUTH' | 'WEST':
                    connection.set_direction(self, action)
                case _:
                    assert 0, f'no case {action}'

We can create the cohort in take, as with step.

        case 'take':
            cohort = Cohort(self)
            connection.take(cohort, self.id)

Two tests fail. We have to fix them to do the cohort trick. They still fail. IWefix World.execute:

    def execute_action(self, entity, action):
        verb = action["verb"]
        match verb:
            case 'step':
                self.step(entity)
            case 'take':
                self.take_forward(entity)
            case _:
                raise Exception(f'Unknown action {action}')

Green! Note the added default case to raise an exception if we don’t find the verb. I added that first, then the new verb. Tests are green and it is 1043. Twelve minutes, counting writing. Commit: converted take to use a request.

I nearly panicked when I realized that Bot had to change to use the Cohort. And again when then I had to change the tests to create a Cohort. I think there’d have been a bit less panic had I started with a test for World, but I got away with it.

Got away with it.

I don’t think “getting away with it” is quite the ideal way of programming. Here I am, trying to practice the best habits I know of, and yet, there I go again, trying to take a shortcut. I’ll do it the other way next time. Maybe.

We’re in an odd state with Cohort, which does serve as a stand-in for a real Cohort class that manages a group of Bots, though this one only handles one Bot. I’m a bit concerned that we’ll have to unwind some of these places where we create a Cohort, but for now, maybe it’s OK.

Except … let’s look at one of the remaining actions that aren’t converted yet, compared to the new scheme:

class DirectConnection:
    def take(self, cohort, client_bot_id):
        rq = dict()
        rq['entity'] = client_bot_id
        take_action = {'verb': 'take'}
        rq['actions'] = [take_action,]
        self.world.execute(rq)
        result_dict = self.world.fetch(client_bot_id)
        cohort.update(result_dict)

    def drop(self, client_bot, block_id):
        self.world.command('drop', client_bot.id, block_id)
        self.update_client(client_bot)

It’s tempting to fix take without the cohort rigmarole back in Bot. We’re going to lose all these detailed methods anyway, so the passing of the cohort, while we learned something from it, shouldn’t happen at all at the level of executing a request, much less these verbs, which will disappear entirely, if I’m not mistaken.

I think we’ll TDD the new World verb this time. We only have this one real test

class TestWorldRequests:
    def test_one_step(self):
        world = World(10, 10)
        bot_id = world.add_bot(5, 5)
        rq = {
            'entity': bot_id,
            'actions': [
                {'verb': 'step'}
            ]
        }
        world.execute(rq)
        world_bot = world.entity_from_id(bot_id)
        assert world_bot.location == Location(6, 5)

The drop test will be a bit tricky to craft but it’ll be worth having. Let me try Wishful Thinking on this one.

    def test_drop(self):
        world = World(10, 10)
        bot_id = world.add_bot(5, 5)
        block_id = world.add_block(6, 5)
        rq = {
            'entity': bot_id,
            'actions': [
                {'verb': 'take'}
            ]
        }
        world.execute(rq)
        world_bot = world.entity_from_id(bot_id)
        assert world_bot.holding.id == block_id

This much runs: the bot received the block. Now some more test. I think this is close to what I need:

    def test_drop(self):
        world = World(10, 10)
        bot_id = world.add_bot(5, 5)
        block_id = world.add_block(6, 5)
        rq = {
            'entity': bot_id,
            'actions': [
                {'verb': 'take'}
            ]
        }
        world.execute(rq)
        world_bot = world.entity_from_id(bot_id)
        assert world_bot.holding.id == block_id
        assert world.map.at_xy(6, 5) is None
        rq = {
            'entity': bot_id,
            'actions': [
                {'verb': 'drop', 'param1': block_id}
            ]
        }
        world.execute(rq)
        assert world.map.at_xy(6, 5).id == block_id\

I’m not sure about the parameter, but I think it’s supposed to be an id. and I’m not sure whether I need the id in that last assert but I think I do.

Note

We see here that I’m not sure what’s an id and what’s an entity. This confusion will come back to bite me, and clearly indicates that the code needs to be more clear, and probably more consistent.

Anyway we should be getting the exception now, and we are. So we need the new case:

    def execute_action(self, entity, action):
        verb = action["verb"]
        match verb:
            case 'step':
                self.step(entity)
            case 'take':
                self.take_forward(entity)
            case 'drop':
                param1 = action["param1"]
                self.drop_forward(entity, param1)
            case _:
                raise Exception(f'Unknown action {action}')

Doesn’t work! I think we need the entity not the id. Change the test:

    def test_drop(self):
        world = World(10, 10)
        bot_id = world.add_bot(5, 5)
        block_id = world.add_block(6, 5)
        rq = {
            'entity': bot_id,
            'actions': [
                {'verb': 'take'}
            ]
        }
        world.execute(rq)
        world_bot = world.entity_from_id(bot_id)
        assert world_bot.holding.id == block_id
        assert world.map.at_xy(6, 5) is None
        block = world.entity_from_id(block_id)
        rq = {
            'entity': bot_id,
            'actions': [
                {'verb': 'drop', 'param1': block}
            ]
        }
        world.execute(rq)
        assert world.map.at_xy(6, 5) == block

Green. We can commit: test for drop in execute works.

It has been 30 minutes since the last commit. There might be five minutes of overhead in writing. I’d be hard-pressed to argue that I spent ten minutes not programming, so we ran over the 1x20 deadline but not 2x20. Let’s reflect:

Reflection

The bulk of the time went into writing the test. There are only three lines of new code in World, with 24 lines of test. Most of the time was spent working out how to access the info needed in the asserts, and not being clear on when I needed an id and when I needed a WorldEntity. But, looking back, I don’t think much time was wasted. It just took that long.

I never felt pressure or concern that I was losing track. I pretty much just ticked through it. At my pace of thinking, writing, and coding, that’s how long it took.

Ten minutes per line of production code? Really? I guess so. Of course I could have just typed it in: the code was obvious other than the snap decision to call the parameter ‘param1’, which I’m already regretting.

Let’s finish up the job, see what tests break and fix them up.

class Bot:
    def perform_actions(self, actions, connection):
        cohort = Cohort(self)
        for action in actions:
            match action:
                case 'take':
                    connection.take(cohort, self.id)
                case 'drop':
                    connection.drop(cohort, self.id, self.holding.id)
                case 'step':
                    self._old_location = self.location
                    connection.step(cohort, self.id)
                case 'NORTH' | 'EAST' | 'SOUTH' | 'WEST':
                    connection.set_direction(self, action)
                case _:
                    assert 0, f'no case {action}'

class DirectConnection:
    def drop(self, cohort, client_bot_id, holding_id):
        rq = dict()
        rq['entity'] = client_bot_id
        take_action = {'verb': 'drop', 'param1': holding_id}
        rq['actions'] = [take_action,]
        self.world.execute(rq)
        result_dict = self.world.fetch(client_bot_id)
        cohort.update(result_dict)

A test breaks, just one. I am a bit concerned about the game, but one thing at a time here.

    def test_bot_drops_and_world_receives(self):
        world = World(10, 10)
        connection = DirectConnection(world)
        client_bot = connection.add_bot(5, 5)
        world.add_block(6, 5)
        block = world.map.at_xy(6, 5)
        assert isinstance(block, WorldEntity)
        cohort = Cohort(client_bot)
        connection.take(cohort, client_bot.id)
        assert client_bot.has_block()
        assert not world.map.at_xy(6, 5)
        print(client_bot.holding)
        connection.drop(cohort, client_bot.id, block)
        test_block = world.map.at_xy(6, 5)
        assert isinstance(test_block, WorldEntity)

Fixing the test up in a nearly obvious way makes it pass. I’ll run the game, with some trepidation, and also fear.

And it fails:

    connection.drop(cohort, self.id, self.holding.id)
                                     ^^^^^^^^^^^^^^^
AttributeError: 'int' object has no attribute 'id'

So holding is already an id. We really need to sort out id versus entity and get stuck in to doing it one way instead of swapping back and forth.

This is bad. I have no real recourse but to debug it. It all comes down to confusion between when we are passing an id and when an entity. However, counting a break, fixing the bug results in 39 minutes between the prior commit and this one.

Whew! Just under the 2x20 bar.

Well past time to be done, let’s reflect and then back away slowly.

Reflect

We have take and drop working according to the new request scheme. Two features in 2 1/2 hours. Not impressive. Small steps were simply not happening. First time was a lazy choice of two options. The other option, doing World and Bot separately, would probably have provided for an intervening commit, maybe even beating 1x20.

The second time, drop, was the first action with a parameter, and in retrospect it’s easy to see that I’m not clear in my mind when we pass an entity and when we pass an id. In particular I’m not clear on whether the Bot is holding an object id or an object … though I should be, because we don’t even have a Block class on the client side.

I think we need to take a pass through things renaming some parameters and methods to better indicate what they do. And it is possible that our id should be something other than an int type, which might make some type hints useful. I guess we could hint int but that would be particularly tacky.

I did panic a bit when the game didn’t work, but I kind of knew in my heart that it was likely to break. Sorting it out was a bit tricky, because there was code assuming entity was passed and code assuming id was passed and code converting from one to the other in both directions.

A wild idea has appeared!

What if we had a “smart id”? It would contain the id int, but would also support all the methods from WorldEntity that we need to use, fetching, possibly caching, and forwarding to the entity.

Mm, probably not. I’ll keep it in mind but we should review the code for better naming and perhaps type hinting. The WorldEntity is pretty lightweight and there may be a decent chance that, when we get this request stuff in place, there will be fewer places that swap back and forth.

That said, we’re kind of stuck with passing a naked integer or equivalent back and forth between client and server. Sure we could give it a type and all that but it has to be just an identifier, I think, to preserve the separation between client knowledge and world knowledge.

We’ll keep it in mind but probably no.

Bottom line, we nearly blew the walls out of 2x20 but squeaked under the limit both times. Next time, I’ll see if there are even smaller moves to be made. Right now, I don’t see them. It seems like today’s work was about the size it had to be.

I’m almost certainly wrong about that. There were probably some intervening tests that could have been written, and possibly I could have written a new method for drop, tested it into working and only then plugged it in.

A decent day, but not a great one. Still, getting out alive is the main thing, and by that standard, I’m OK so far.

See you next time!