The Repo on GitHub

How much does the way I work apply to Real Work? What is the future of the Bots World? And let’s improve the messages a bit.

Discovery

You’ve probably noticed that my style of programming is very much focused on “discovery”. I don’t seem to do a lot of large-scale design. Instead, I have lots of small ideas, small understandings of what “good code” looks like. I build off in the direction of the next thing we need, observe the pattern of the resulting code, and improve it. In working this way, I defer addressing issues that we all know will turn up.

In the Bot World, I’ve deferred the client-server connection way beyond any reasonable person’s point of panic. In the ill-fated Extreme Programming Adventures in C#, I deferred the undo operation until well past any rational person’s opinion on when it needed to be addressed. I do that on purpose.

What purpose is that, you ask? Well, what I want to understand, and to demonstrate, is that if the design we have is good for the features we have now, we can almost certainly refactor to accommodate whatever new requirement is likely to come along. I do emphasize almost certainly and is likely. I suspect that if the surprise requirement for my C# text editor had been “Must compile the edited text as a C# program without use of any commercial or open source libraries”, I might have been in trouble.

Although … I suppose we could have started a small compiler project and simply passed the text over to the new module, then extended the module … “This may take a while, chief, but we can do it.”

So I program as I do because I like to explore the edges of the programming space, to discover and show what we can do while working very incrementally, in tiny steps, with plenty of design thinking, but very limited design decisions. What does that say about what a Real World Programmer should do? Should a team of Real World Programmers proceed quite as incrementally as I do, with as little commitment to an overall design as I make? I would not draw that conclusion.

If I were again condemned to work in a Real World Programming Team, we would do enough design thinking to identify serious risks to the effort, risks that might stop us dead, risks that might slow us down, risks that need to be addressed early. We would propose work to mitigate those risks, via experiments or other forms of discovery. We would build the system with “seams” built in where deferred components would someday go. And then, in the light of those things, we would build the software as I build it here, bit by bit, very small steps, everything supported by tests, with continual refactoring to keep the design fresh.

That’s what I’d do, to the extent that I can write it in a paragraph. I suspect I could write it in a book-sized form if anyone wanted it.

What should you and your team do? That is up to you. My job here is to do things, show you what happens, draw my conclusions about what I should do. I want my experience to become a component of your thinking, not so that you can do as I do, but so that you can better identify what you should do.

In the Real World, I wouldn’t be quite so free-wheeling and discovery-oriented as I am here. (Well, actually, I might be, but it might be a bad idea.) In your Real World, you get to do you, using whatever learning and guidelines and wisdom you’ve formed over all these years.

I hope that sometimes, I can help some of you learn to do things just a bit better than before.

Future of Bots World

It’s about time to make the World more interesting, and to devise smarter Bots to deal with the world. After a bit more work on the World’s messaging, I have a few ideas that might be fun.

Pits and Blocks
Suppose the World included Pits, locations where there is a hole in the ground. If a Bot steps into a Pit it gets stuck. Perhaps the only way it can get out is for another Bot to help it. Perhaps if a Bot drops a Block into a Pit, it fills it up, and the location becomes normal ground.

Or perhaps, like ants, Bots can bridge a pit with their bodies. Maybe they can transform (Ground, Pit, Ground) into (Bot, Bot, Bot) such that other Bots can cross the bridge. Perhaps there is some objective that is isolated from the Bots by Pits, and the Bots must organize to reach it.

More Intelligent Bots
I would prefer to keep the Bots very simple, driven by fairly simple state machines rather than complex Python code. But as soon as we add something like Pits to the World, the machine we have will certainly have to be improved substantially. Ideally—by my design preferences—the machine states and conditions would remain quite simple, making decisions based only on current state and limited vision and scent information.

I think it would be interesting and fun to work with the state machine logic of the Bots … and perhaps we would come up with a “little language” or some other way of writing the machine that is even less code-oriented than we have now.

Pheromones
Ants, I’m told, base their behavior on the ability to identify a fairly wide range of scents, and the ability to tell which direction has more of a given scent than the other directions. They use this ability in very simple ways, but the result is that they will all start biting enemies, or all begin to go to a food source and bring food home. We could add the notion of pheromones to our world.
Inter-Bot Communication
As the Bots wander around, they get a continual sequence of “vision” elements, which identifies the contents of the cells immediately surrounding the Bot, and the “scents” associated with the cell they are on. Currently, they only make immediate decisions based on that limited information. They do not remember it, nor do they share it with others.

What if they did share it? Suppose there is a radio system between Bots and what if they all continually radio what they see and smell, and they all record a memory of all the radio messages they have had. If we did that, they might be able to make larger-scale decisions, perhaps still without very complex logic.

Bot View vs World View
Our Game class displays the World’s view of the situation, showing all the Bots and Blocks at once. If we allow the Bots to build up a picture of the world, as described above, we’d probably want to have another kind of view, from the Bots’ viewpoint: what do our Bots know about the World.
Client-Server
Someday, I suppose, I should get to a full client-server setup. The fact is, I am not very interested in this. I think it’s quite doable, I think it’ll be tedious, and I think there is little purpose for it, because no one is ever going to connect to this World with their own Bot code. That aspect of this project seems to me to have become moot. Still, it might be a learning experience. Maybe even a good one.

Overall, I think that some of these ideas might be fun, and I’m sure that as we try to program them, we’ll encounter interesting design issues. So we may find ourselves trying some of them, unless I get a better idea.

Messages

Working as I usually do, I don’t concern myself much with exceptions and error conditions. My programs are integrated, they have no external users, and if there is a mistake that causes an exception, I fix the error. However, in this Bot World situation, in principle, the client side Bots could send any old malformed piece of junk over to us, and we shouldn’t every crash the server. Instead, we should just return a sharply-worded note to the offending requestor.

We have a start at messages now. I’d like to take it further, because I think I’ll learn some useful things in doing so.

As a partial driver for this, I think I’d like to “improve” the little language that we use to pass requests to the World. As we make any changes like that, we’ll almost certainly get something wrong, and I’d like to have that dealt with via these terse notes rather than crashes of the program.

And I have an idea. Generally speaking, when our code can produce an exception. we code something like this:

try:
    do_something(a1, a2, a3)
except Exception as error:
    self.add_message("blah blah")
finally:
    perhaps_some_recovery()

I find that code to be irritating. So I’d like to find a way to code something more like this:

do_protected(do_something, "blah blah", a1, a2, a3)

Or perhaps:

with Protection("blah blah", perhaps_some_recovery):
    do_something(a1, a2, a3)

I’m aware that there are ways to build these with things, called “context managers”, and I think they might be just the thing to allow our error-handling code to look a bit better.

This being the case, I think we’ll work on messages a bit more: there are useful things to learn.

We’ll drive this effort with tests. We need to handle a lot of mistakes, and the best way I see to do that is to build up a lot of tests that do things that just won’t work.

Let’s try one.

    def test_no_verb(self):
        world = World(10, 10)
        requests = [{'vorb': 'add_bot'}]
        messages = world.execute_requests(requests)['messages']
        assert messages == []

In setting up this test, which is still not doing what I want, dealing with the typo ‘vorb’ for ‘verb’, I discovered two exceptions, one when I put a comma where the colon goes, which makes the action a set instead of a dictionary, and another the missing ‘entity’ key, which is presently required. We currently have this exception:

    def execute_actions(self, actions_list):
        for action in actions_list:
>           action_with_parameters = self.assign_parameters(**action)
E           TypeError: World.assign_parameters() missing 1 required positional argument: 'entity'

We’re using Python’s clever ability to strip specific keys out of a dictionary and turn them into parameters:

    def assign_parameters(self, entity, **parameters):
        if entity:
            self.ids_used.add(entity)
            parameters['entity_object'] = self.entity_from_id(entity)
        return parameters

This works a treat as long as there is a key ‘entity’ and it hurls if there isn’t. Let’s handle this with a try/except, which we’re not going to like, so we’ll see whether it drives us to do something better.

One alternative, of course, would be to check explicitly for entity. Even just mentioning it makes me like that better.

    def assign_parameters(self, **parameters):
        entity = parameters.pop('entity', 0)
        if entity:
            self.ids_used.add(entity)
            parameters['entity_object'] = self.entity_from_id(entity)
        return parameters

This results in this exception:

            case _:
>               raise Exception(f'Unknown action {action_dictionary}')
E               Exception: Unknown action {'vorb': 'add_bot'}

We should be able to just add the necessary message right here.

            case _:
                self._add_message(f'Unknown action {action_dictionary}')

And this test passes:

    def test_no_verb(self):
        world = World(10, 10)
        requests = [{'vorb': 'add_bot'}]
        messages = world.execute_requests(requests)['messages']
        assert 'Unknown action' in messages[0]['message']

We really need some central location for message texts, don’t we? We should do that Real Soon Now, before there are too many.

Green. Commit: convert exception to message.

Now there is no need for the ** on the call and definition of assign_parameters.

    def execute_actions(self, actions_list):
        for action in actions_list:
            action_with_parameters = self.assign_parameters(action)
            self.execute_action(**action_with_parameters)

    def assign_parameters(self, parameters):
        entity = parameters.pop('entity', 0)
        if entity:
            self.ids_used.add(entity)
            parameters['entity_object'] = self.entity_from_id(entity)
        return parameters

I noticed this code:

    def execute_action(self, entity_object=None, **action_dictionary):
        match action_dictionary:
            case {'verb': 'add_bot',
                  'x': x, 'y': y, 'direction': direction}:
                self.add_bot_action(x, y, direction)
            ...

Could I have just used entity=None in what I just did, rather than the explicit check and remove? Let’s try that, it would be better.

    def execute_actions(self, actions_list):
        for action in actions_list:
            action_with_parameters = self.assign_parameters(**action)
            self.execute_action(**action_with_parameters)

    def assign_parameters(self, entity=None, **parameters):
        if entity:
            self.ids_used.add(entity)
            parameters['entity_object'] = self.entity_from_id(entity)
        return parameters

Green. Better. Commit: improve handling of missing entity.

Now we can do a new test where I think we should get an interesting explosion.

    def test_no_entity(self):
        world = World(10, 10)
        requests = [ {'verb': 'step'}]
        messages = world.execute_requests(requests)['messages']

This hurls:

    def step_action(self, bot):
>       self.map.attempt_move(bot.id, bot.forward_location())  # changes world version
E       AttributeError: 'NoneType' object has no attribute 'id'

Now in execute_action we do allow entity_object to be None:

    def execute_action(self, entity_object=None, **action_dictionary):
        match action_dictionary:
            case {'verb': 'add_bot',
                  'x': x, 'y': y, 'direction': direction}:
                self.add_bot_action(x, y, direction)
            case {'verb': 'drop',
                  'holding': holding}:
                self.drop_forward_action(entity_object, holding)
            case {'verb': 'step'}:
                self.step_action(entity_object)
            case {'verb': 'take'}:
                self.take_forward_action(entity_object)
            case {'verb': 'turn',
                  'direction': 'NORTH' | 'EAST' | 'SOUTH' | 'WEST' as direction}:
                self.turn_action(entity_object, direction)
            case {'verb': 'turn', 'direction': bad_direction}:
                raise AttributeError(f'unknown direction {bad_direction}')
            case {'verb': 'NORTH' | 'EAST' | 'SOUTH' | 'WEST' as direction}:
                self.turn_action(entity_object, direction)
            case _:
                self._add_message(f'Unknown action {action_dictionary}')

However, it really can’t be allowed to be None: we’ll send it messages. Hm not if we have removed entity_object from the dictionary.

I try this:

    def execute_action(self, entity_object=None, **action_dictionary):
        action_dictionary['entity_object'] = entity_object
        match action_dictionary:
            case {'verb': 'add_bot',
                  'x': x, 'y': y, 'direction': direction}:
                self.add_bot_action(x, y, direction)
            case {'entity_object': None }:
                verb = action_dictionary.get('verb', 'missing verb')
                self._add_message(f'verb {verb} requires entity parameter {action_dictionary}')
            case {'verb': 'drop',
                  'holding': holding}:
                self.drop_forward_action(entity_object, holding)

This is a bit of a hack: I put the object back into the dictionary. We’ll improve that if this works.

Down after the only verb that does not need an entity object, I added a case for it being None. Two tests break, because my first one is getting this message also. I modify that to have a non-null entity object:

    def test_no_verb(self):
        world = World(10, 10)
        requests = [{'entity_object': "fake", 'vorb': 'add_bot'}]
        messages = world.execute_requests(requests)['messages']
        assert 'Unknown action' in messages[0]['message']

The other failing test is this one:

    def test_zero_id(self):
        WorldEntity.next_id = 100
        world = World(10, 10)
        command = {'entity': 0,
                   'verb': 'step'}
        rq = [ command ]
        with pytest.raises(AttributeError) as error:
            result = world.execute_requests(rq)
        assert str(error.value) == "'NoneType' object has no attribute 'id'"

Right, we get a message now. What is it?

    def test_zero_id(self):
        WorldEntity.next_id = 100
        world = World(10, 10)
        command = {'entity': 0,
                   'verb': 'step'}
        rq = [ command ]
        messages = world.execute_requests(rq)['messages']
        assert 'requires entity parameter' in messages[0]['message']

That’s as expected. Now the test we’re actually working on needs that same check.

    def test_no_entity(self):
        world = World(10, 10)
        requests = [ {'verb': 'step'}]
        messages = world.execute_requests(requests)['messages']
        assert 'requires entity parameter' in messages[0]['message']

Green. Commit: operations requiring entity now file a message.

Make it work, then make it right. So let’s make this right:

    def execute_action(self, entity_object=None, **action_dictionary):
        action_dictionary['entity_object'] = entity_object
        match action_dictionary:
            case {'verb': 'add_bot',
                  'x': x, 'y': y, 'direction': direction}:
                self.add_bot_action(x, y, direction)
            case {'entity_object': None }:
                verb = action_dictionary.get('verb', 'missing verb')
                self._add_message(f'verb {verb} requires entity parameter {action_dictionary}')
            case {'verb': 'drop',
                  'holding': holding}:
                self.drop_forward_action(entity_object, holding)
            ...

It’s a hack to put the object back. Better to pull it out if it’s there and leave it in. And I’d like at least a comment, as you’ll see below.

    def execute_actions(self, actions_list):
        for action in actions_list:
            action_with_parameters = self.assign_parameters(**action)
            self.execute_action(action_with_parameters)  # Note: removed **

    def execute_action(self, action_dictionary):  # Note: removed ** and entity_object parameter
        entity_object = action_dictionary.get('entity_object', None)  # added
        match action_dictionary:
            case {'verb': 'add_bot',
                  'x': x, 'y': y, 'direction': direction}:
                self.add_bot_action(x, y, direction)
            # -----------------------------------------------
            # operations below here all require entity_object
            case {'entity_object': None }:
                verb = action_dictionary.get('verb', 'missing verb')
                self._add_message(f'verb {verb} requires entity parameter {action_dictionary}')
            case {'verb': 'drop',
                  'holding': holding}:
                self.drop_forward_action(entity_object, holding)

I think that’s nearly good. Commit: improve handling of entity object

I think we’ve done enough for this morning.

Reflective Summary

After some large-scale musing, we got down to adding error messages where we formerly had exceptions. We’ve added just two, but the second one in particular covers a number of potential exceptions, one for each verb that requires a subject entity object. We covered all those with a single check in our match/case statement. The other change, unknown verb, covers an infinity of cases, the infinite number of strings that aren’t valid verbs. So we’ve done an amazing amount of work this morning, by some standards.

We kind of worked in a circle on the entity parameter in an action. At first I thought that I needed to look explicitly for the ‘entity’ key in the input, but then I realized I could default it in the call and deal with it that way. So now if we get a non-zero ‘entity’ key, we convert it to an entity and store it in the input for later use. (The name needs changing to ‘entity_id’ or ‘bot_id’ or something. We’ll get to that, I think.)

I think you could make the case that we should break out the verbs that require an entity into a separate match/case from those (the one) that do not. Right now, there is just the one that doesn’t, but structurally it might be better to keep them separate.

We still need more destructive tests, but so far we have only removed exception raises, and have not needed to add any. I do think we’ll want at least one try/except, to handle true surprises, but I think we can get this thing pretty bullet-proof using the scheme we have here.

As always—well, almost always—well, usually—well, sometimes—happens, we leave the world improved and a bit more capable than when we arrived. A satisfactory morning. See you next time!