Robot 30

More on server code. I have an idea. Should I try it? Answer: no.

Today I mostly want to refine my server object a bit, and decide how to apply what I’m learning. And I have an idea that might be worth exploring.

If I recall correctly—always a dicey proposition—I was able to open multiple clients against a single socket server and send messages back and forth. If that’s true, then it seems to me that I could do a fairly simple test where I instantiate a World and multiple robots, connecting via sockets, and cause the robots to do some operation frequently, to see what might happen.

If I were to do this, I think we’d find that my iPad could readily serve a rather large number of robots, at least if we assume that the robots are operating at human transaction rates.

But would we really learn anything we don’t already know, or anything that we couldn’t find out without going to that extreme? There’s the rub. It’s easy to time the socket-checking loop, and it’s easy to time how many transactions per second the World can do: we’d just do some long operation, like the scan/look, in a loop, and time how many we can do. Unless there’s massive latency in the socket code itself, we’d have our answer right there.

I’m here to distract myself from reality, and to have fun doing it, and perhaps to be helpful to a few people, so I’ll do the experiment only if it scores sufficiently high on those dimensions. We’ll see.

In any case, today I just want to work a bit more on the server object we were working on yesterday. Let’s get started.

Review

Here are my tests. There’s one new one that I didn’t show yesterday, that I wrote during that long dark debugging time of the soul.

        _:test("FakeServer", function()
            local server = FakeServer()
            local result,error = server:accept()
            _:expect(result, "first result nil").is(nil)
            _:expect(error, "first err timeout").is("timeout")
            server:ready()
            client,error = server:accept()
            _:expect(client:is_a(FakeClient)).is(true)
            result,error = server:accept()
            _:expect(result).is(nil)
            _:expect(error).is("timeout")
        end)
        
        _:test("indirect readiness", function()
            local socket = FakeServer()
            local server = Server(socket)
            socket:ready()
            server:process()
            _:expect(#server:clients()).is(1)
            local client = server:clients()[1]
            local r,e = client:receive()
            _:expect(r).is(nil)
            _:expect(e).is("timeout")
            server:ready(1)
            _:expect(client._ready).is(true)
            local msg = ""
            local proc = function(client)
                local m,e = client:receive()
                msg = m
            end
            server:processClients(proc)
            _:expect(msg).is("message\n")
        end)
        
        _:test("Server accumulates clients", function()
            local count = 0
            local proc = function(client)
                local r,e = client:receive()
                if r then
                    count = count + 1
                end
            end
            local socket = FakeServer()
            local server = Server(socket)
            server:process()
            _:expect(#server:clients()).is(0)
            server:processClients(proc)
            _:expect(count).is(0)
            socket:ready()
            server:process()
            _:expect(#server:clients(), "clients").is(1)
            server:ready(1)
            server:processClients(proc)
            _:expect(#server:clients(), "clients").is(1)
            _:expect(count).is(1)
            server:processClients(proc)
            _:expect(#server:clients(), "clients").is(1)
            _:expect(count).is(1) -- still one, client should be already processed
        end)

The first test drove out the FakeServer object, and the second two are just testing the operation of the “real” Server object, which has two methods:

• process, which checks to see whether there is a new connection to be added;
• processClients, which currently executes a function, providing each client as an argument.

The names don’t seem great, so we might improve that as part of the code review below. And there is an issue we might care about, which is that processClients processes all clients, not just those who are ready to provide a line. Since it’s likely that most robots are idle most of the time, this loop would inquire about many robots who would just return a timeout, indicating that they have nothing to say.

Let’s look at the code. There isn’t much:

Server = class()

function Server:init(socket)
    self._socket = socket
    self._clients = {}
end

function Server:clients()
    return self._clients
end

function Server:process()
    local client, err = self._socket:accept()
    if client then table.insert(self._clients, client) end
end

function Server:processClients(callBack)
    for i,client in ipairs(self._clients) do
        callBack(client)
    end
end

function Server:ready(clientNumber)
    if clientNumber <= #self._clients then
        self._clients[clientNumber]:ready()
    end
end

Note that the ready method is just there for testing. We use it to cause a FakeClient to return a line upon receive, rather than a timeout error.

There’s not much to hate here. I think I’d like it better if process were named acceptClients or acceptNewClients. And we presently have no provision for removing a client, for example if it has closed.

I believe I’ve mentioned that there is a function on socket called select, which given a collection of sockets, returns the ones that are ready to be read. (It can also return, from another collection, those which are ready to receive, which doesn’t concern us.) We could, in principle, use this function quite trivially, like this:

function Server:processClients(callBack)
    local readyClients = self:selectReadyClients()
    for i,client in ipairs(readyClients) do
        callBack(client)
    end
end

function Server:selectReadyClients()
    return socket:select(self._clients)
end

We can’t do that here in our test bed, because we’re not using real clients. There are a couple of magical incantations that could be implemented on our FakeClient to allow it to be used in socket:select, but I really don’t want to go there.

Let’s rename that process method and leave the selectReadyClients in there, with a second copy of the method covering the socket access:

function Server:acceptClients()
    local client, err = self._socket:accept()
    if client then table.insert(self._clients, client) end
end

function Server:processClients(callBack)
    local readyClients = self:selectReadyClients()
    for i,client in ipairs(readyClients) do
        callBack(client)
    end
end

function Server:selectReadyClients()
    return socket:select(self._clients)
end

function Server:selectReadyClients()
    return self._clients
end

Putting that second version of selectReadyClients replaces the first one, so the tests will run benignly. We are green. Commit: rename method. prepare for socket:select.

With this in hand, let’s reflect about our server situation.

Reflection

On the one hand, I think this little server object captures the essence of our service problem, and that it would serve just fine, no pun intended. On the other hand, it would certainly need fleshing out, with things like protection from internal errors and throws, dealing with network errors, and, in particular, dealing with closed sockets.

On the gripping hand, I feel satisfied with what I’ve learned here, and my advice to the team is that we can proceed with confidence that the client-server code will be readily completed to our liking. From my personal home viewpoint, I think I’m “done” with this question for now.

In a real product effort, I would recommend that we take another couple of days to bring the Server up to a reasonably solid state, and then plug it into our walking skeleton, so that it can be demonstrated and tested in the evolving real environment.

In a large-enough effort, one might imagine one part of the team producing versions of the world server and another part versions of the robo client, but I think that would be a terrible idea. Why terrible?

It would be a terrible idea because the game only makes sense to the extent which the world and robot evolve together. It’s not helpful if the robot makes a call that the world does not implement, nor if the world implements some feature that the robots do not use. Even worse, both sides would be implementing blindly according to their interpretation of a written spec, and would surely run into coordination problems when things don’t hook up correctly.

I think it would be far better to continue as we are now, with features being developed one at a time in both World and Robot, by the same individuals. It’s true that someday there may be hundreds of developers all over the world building Robots to join in our game, but during development, we really need at least one functioning Robot to work with the World, developing both.

For development, I would not propose working with a World server running separately from the Robots, because we’d be continually bouncing the server to bring up our new changes. Even if we did decide always to run across sockets, I think we probably will profit from having our World Making and Robot Making apps be the same.

There may come a day when it will seem like a good idea to separate them. Just now, I can’t think what situation would make me want to do that, unless and until we start multiple different Robot efforts. Even then, I think that each effort would want its own World code for as long as possible.

Once the World is stable, it probably makes more sense to run a standard multi-robot server, perhaps even different ones for different kinds of worlds. But that day, in my view, is far away, and I suspect that even when there are many Robot creation efforts, those efforts will prosper from having their own built-in local World most of the time.

I could be wrong, but that’s what I think.

Deeper Reflection

I am left with a deeper question: what should we do now?

At this writing, I don’t see any unsolved, interesting problems. I freely grant that “interesting” is a pretty personal characterization, but my house my rules.

If a reader gets this far and would like to see how I would address some aspect of the program, please email me or tweet me up with your idea. I promise to consider them, and to work on the ones that strike me as having value.

Otherwise, I think I’ll transition into a new phase, for this program.

The original Robot Worlds specification is intended as a learning platform for programming learners in South Africa. I think that, starting in the next article, I’ll think about how the spec and problem fit with learning programming. I’ll be doing this in the hope that my thoughts will be of value to the excellent folks who are providing this learning to South African learners.

And, of course, when interesting coding issues come to mind, I’ll be working on them and writing them up.

Stay tuned … and provide feedback if you wish.