Circling Client-Server

The Repo on GitHub

Continuing changes toward client-server. We’ll discuss my lack of a real plan. We’ll discuss why what I want to begin on today is particularly irritating. Much thinking, little code. As it should be.

Over the past few sessions, I’ve been pushing the code more in the direction of client-server, with a particular focus, batching¹ up the communication. In principle, each call between bot and world could be its own network operation, with some kind of co-routine / threading kind of support built in². I believe, based on a small calculation, that doing that would limit the cycle time of the game far too much, so that one batch of commands per cycle is what I’m aiming for. I could be wrong³.

It may seem that I should have more of a plan than I seem to have: some overall scheme of first doing this, then doing that, then finally wrapping up this and tying it up with this nice bow. I wouldn’t mind having a plan like that, but in fact I do not have one. I wouldn’t deny that if we were to sit down and talk about it, we could probably draw some kind of pictures of the system, with circles and arrows and a paragraph on the back of each one explaining what each one is, but I’m not there yet. Maybe later today I’ll do that just for fun.

But I do have a general understanding of how this little program works, and a sense of the places where things will have to change to support network client-server arrangements. These come down, mostly, to places where an object on one side of he divide sends a message or returns a result directly to an object on the other side. Every one of those places needs to be dealt with, moving whatever is going on there into part of a communications message sent across the air, wire, fiber, what ya got, and a reply sent back.

And there have been small gestures made in that direction. I call the court’s attention to Exhibit A, the replacement of direct calls from the state machine, into strings indicating what the Bot should send to the World:

    def do_something(self):
        self.update_knowledge()
        self.state = self.state.update(self._knowledge)
        self.do_state_actions()
        self.move()

    def do_state_actions(self):
        for action in self.state.action(self._knowledge):
            match action:
                case 'take':
                    self.world.take_forward(self)
                case 'drop':
                    self.world.drop_forward(self, self.inventory[0])
                case _:
                    assert 0, f'no case {action}'

We originally made that change while the code still had access to the world member, although now it has not. We made it to bring together separate calls to world into one place, which I perceived to be a step in the direction of batching up the commands or at least converting them to strings for transmission. Doing so also gives a bit of focus on what information will need to be sent, all in this one place.

It’s not much. It’s not a grand plan. It is just a small step in what I perceive to be the direction of client-server.

Would it be better to have a better plan? I can think of at least one way in which it might: it would help my colleagues understand why certain things are the way they are. As things stand now, when you look at that code, you are likely to wonder “Why didn’t he just send the message? Why is he creating this string?” And you would be right to wonder that. We may even find, going forward, that the Machine should just send a message to the World. What, even though it can’t? Yes, because these strings are really irritating.

Why? I’ll tell you why!

If we have a handle on an object and want to ask it to do something, we can just send it a message, bank.deposit(my_account,350) and it’s all good. In a client-server situation, we have the added chore of devising a language that the two sides will agree on, and formatting up a string, like ‘DEPOSIT,345-9876,350.00’ and sending it over to the other side, where it will be decoded somehow and turned into something like self.deposit("345-9876","350.00) and executed.

We are faced with this somewhat large problem, which is the creation of a textual language that communicates everything a client needs to say to a server and everything the server needs to reply back, and we don’t even know at this point what the server will and will not do, what facilities it will offer the client, and what information it will need in order to do what it does.

Let me assure you …

In my defense, I think I could do a pretty decent job, inside a day, of specifying the language between the Bot client and World server, with the ability to be extended as needed built in. Oh, there would be gaps, especially for things we haven’t even thought of, but the basics would be there to express most anything necessary. And, who knows, I might work on that. But what I really want to do here is to work to improve the program slowly, in the direction of client-server, without a grand plan.

If we were in a company, and we had a grand plan to convert our batch robot system to a client-server one, to better serve the robot game market, and we created a design for the client-server language, there’s a good chance that the development team would be handed that spec (naturally it was created by the newly formed Client-Server Architecture Unit, who write no code) and the team would be told to implement that spec to drive the formerly batch world. And they would go away and sometime in the indefinite future, they would still not be done.

Meanwhile, in another country, or three, various client-facing teams would be told to write new client software, using the aforementioned spec, to call the headquarters server, as soon as it’s done, so be ready by some arbitrary date.

This plan would be an unmitigated disaster. Heads would roll, and executives would get much higher pay because of the hard decisions they had to make about which employees to remove and which ones would merely be punished.

Now, I can’t solve the problems of that company right here, they are far too far down the river for my poor help, but what I can do is avoid a Big Bang approach to client server, instead making lots of small changes that will move the design closer and closer to client-server, until, if I’m successful, one day there are two clear spots into which to insert a network request and response and the program will be client-server.

If I’m not successful, you’ll get to watch. But don’t get your hopes up: this is going to work out just fine.

Just what are you doing?

Well, I’m spotting places where c-s things seem to need to happen, and I’m pushing them together, trying to make them more similar to each other, and trying to get them closer together in the code. And I have a bit of a scheme building up in my head. Recall the two diagrams from yesterday:

World⇔Bot

And

World⇔BotProxy⇔ClientProxy⇔Network⇔WorldProxy⇔Bot

I think that a fuller picture might be like this:

World⇔BotProxy⇔ClientProxy⇔(s)⇔Network⇔(s)⇔WorldProxy⇔Bot

That shows where the conversion from object-talk to network string talk (s) takes place: on the network-facing side of the WorldProxy and ClientProxy.

Remember the Making App idea?

The little picture above shows how it’ll work in production, in what GeePaw Hill calls the “Shipping App”. In the Making App, if and when we do one, the connection will be more like this:

World⇔BotProxy⇔ClientProxy⇔(s)⇔WorldProxy⇔Bot

Will the world have a BotProxy? I think it might, but it will be up to the World to decide that. Well, the developers will help, but the code on that side will tell us what it wants to be.

Can you wrap up that “idea”?

Yes, I’ll try. What I’m doing with these small changes is trying to encircle the issues of client server, and herd them together where we can better see what they are, and how to deal with them. I am trying to proceed in small steps such that each step can be pushed into the code base as soon as possible, without harm to the continuing operation of the program, and without harm to our ability to improve the objects on either side of the divide.

I don’t think that can be done perfectly. I expect to learn, to discover, to change my mind. And I am quite sure that I’ll like that better than crafting a detailed plan and getting half way through it only to discover that it needs changes that I’ll have to negotiate with three teams in three different countries.

Where Were We?

Oh, right, there is programming to be done. Yesterday, we did something. What was it?

Oh, yes, right, a start at Population, an object that I think we’ll use on the client side to hold our bots, and WorldProxy, which we’ll send messages to on the client side, as if we were talking to the “real” World, where they’ll be batched up and sent over to the server in the fullness of time.

Those two classes look like this, very small, just starting:

class Population:
    def __init__(self, bot_count=0):
        self._bots = []
        for i in range(bot_count):
            self._bots.append(Bot(i, i))

    def __iter__(self):
        return iter(self._bots)

    def add(self, entity):
        self._bots.append(entity)

class WorldProxy:
    def __init__(self, width=20, height=20):
        self._population = Population()
        self._world = World(width, height)

    def add(self, entity):
        self._world.add(entity)
        entity.world = self
        self._population.add(entity)

    def run_cycle(self):
        for bot in self._population:
            bot.do_something()

    def step(self, bot):
        self._world.step(bot)

The WorldProxy tests include one round trip:

    def test_round_trip(self):
        proxy = WorldProxy()
        bot = Bot(5, 5)
        bot.direction_change_chance = 0
        proxy.add(bot)
        proxy.run_cycle()
        assert bot.location == Location(6, 5)

As that stands, the WorldProxy creates a World instance to talk with. Let’s instead have him create a ClientProxy to talk with, and have it, for now, create the world. There will need to be factories or factory methods for these, I’m sure.

I’m going to just modify these tests, though I am sure we’ll have … no … let’s do TestClientProxy from the get-go. I don’t think it will take longer and then we’ll have better alternatives for testing details versus integration.

Here are my first two tests:

class TestClientProxy:
    def test_exists(self):
        proxy = ClientProxy()

    def test_add_bot(self):
        proxy = ClientProxy()
        bot = Bot(5, 5)
        proxy.add(bot)
        entity = proxy._world.map.entity_at(5, 5)
        assert entity.name == 'R'

And my current class:

class ClientProxy:
    def __init__(self, width=20, height=20):
        self._world = World(width, height)

    def add(self, entity):
        self._world.add(entity)

Tests are green. Commit: initial ClientProxy and tests.

Reflection

It’s pretty clear that we can plug the CientProxy into the WorldProxy and hard-wire message forwarding between the two, and, if we choose, even begin translating those messages to and from a string format. I am concerned about the names a bit because WorldProxy and ClientProxy don’t seem symmetric. The thing is, we have no name for the assemblage of Bots over on the client side. We have individual Bots but what is it that we are doing over there? Is it a planet? I don’t know.

Furthermore, right now, the World receives messages from individual Bots and sends messages back to them. That’s why there’s a BotProxy item in the word pictures above, because, I think, the World mostly thinks in terms of individual Bots and other entities that it is keeping track of. It doesn’t really think in terms of a larger assemblage on the world side either.

Hm. Maybe it’s not a World server at all? Maybe it’s an Entity server? Or two servers in parallel, like two channels, one for addressing the world, hey, add this Bot, but after that, the Bot talks to a Bot kind of channel? I need to talk about this with my friends.

Be all that as it may, with one more change, I think we’ll have a round trip through both proxies, and the change goes like this:

class WorldProxy:
    def __init__(self, width=20, height=20):
        self._population = Population()
        # self._world = World(width, height)
        self._world = ClientProxy(width, height)

That is actually a change to the WorldProxy to use a ClientProxy. Well, you can see that, can’t you. Anyway, that connects the two proxies, and the test fails because ClientProxy does not understand step. I did not foresee that, I just went ahead and typed in my change to plug in the ClientProxy. Should be easy enough:

class ClientProxy:
    def __init__(self, width=20, height=20):
        self._world = World(width, height)

    def add(self, entity):
        self._world.add(entity)

    def step(self, bot):
        self._world.step(bot)

Green. Commit: first round trip through both WorldProxy and ClientProxy.

We begin to see the pattern more clearly now. On the Bot side of things, we have a world proxy whose API is the same as world, talking to a client proxy whose API is currently the same, talking to the World, whose API is the same.

We could, in principle, convert each of those API elements to a string and pass it along to be decoded and so on. More likely, what we’ll do is to accumulate the calls to World API and pack them all up and send them over to be unpacked.

Or maybe we’ll do something else. The main point is this:

When we plug a working WorldProxy and ClientProxy pair into the real program, and it works⁴, we will have the network client-server concern cornered. My guess⁵ is that we are only a few days’ work away from that state. My biggest concern is that currently our tests are all about a single Bot talking to the World and we still need to interpolate a batching object in there, the one that runs the entire cycle. We’ll find a way.

Maybe that should be our next task.

Summary

Today what you’ve seen is a bit of “Design Thinking”: I blathered about the design, as if talking to my Rubber Duckie or one of my colleagues, about what we might have to do. The conversation would have been better with a colleague but none were available. After the thinking, I wrote less than 25 lines of code and test, mostly test, to get a rudimentary ClientProxy in place.

That’s all still running just in the test file, but it is exercising Population, WorldProxy, and ClientProxy, using at least one real Bot and one real instance or World. It is running end to end … missing only the network connection, which is not small but is a well-understood problem. By someone. Not necessarily by me. Yet.

I think these very small tests and classes are showing that we will be quite able to break this thing apart into client and server sides, without massive changes to the operational classes.

And if I’m right, that will be a very good thing. And, if I’m wrong³, you’ll see it right here. See you next time!