Let’s see if we can get the new connection object to run as if it were on a thread, even though we don’t have threads. That should be amusing.

Things have been going quite smoothly in these past few articles working on the socket connection that is needed for a full Robot-To-World program. (I don’t intend to implement the game that way, or that far, but I want to have it clear that that’s because I don’t want to, not that I can’t.)

Anyway, after quite a few false starts that I haven’t shared, and perhaps a few where I have, this time around it seems to be going well. If I’m going to run into serious trouble, we’re getting close. Today and tomorrow, I’d guess, will tell the tale.

Let’s look forward a bit at what needs to be done.

Planning-ish

Where we are …

The WorldConnection object, which could probably be called Server, now has the ability to call back to its owner when a connection is added, and when a client object provides a line of input. It’s tested, so far, using fake objects that respond to receive. We’ve not yet plugged in attempted connections with socket:accept. Instead our tests are calling the addClient method directly. This method calls back on each addition, if a callback function is provided.

And our object has a method processClients, which loops over all existing clients and if they have something to say, calls back to a provided function for each successful receive.

Where we want to be …

We need to wire up an accept test on the server, dealing with attempted connections. I expect this to be easy.

Then we need to set up the WorldConnection so that it appears to be running in its own thread. The plan for that is to wire it into the Codea tween:update function, which is called once per frame, 30, 60, or 120 times a second, which should be just fine for our purposes. On each tick, we’ll process any new clients, and then give each client a chance to provide a message for us to receive. I expect this to be pretty straightforward as well.

It seems unlikely that things will go as smoothly as I expect, but I am forever an optimist, except when I’m not.

Accepting New Clients

Let’s see how things work now.

We’re doing all our testing with a FakeServer and Fake clients. Here’s our most robust test so far:

        _:test("acceptance callback", function()
            local result = ""
            local acb = function(client)
                result = result..client._msg.." connected "
            end
            local server = FakeServer()
            local wc = WorldConnection(server, nil, acb)
            local a = Fake("aaa")
            local b = Fake("bbb")
            wc:addClient(a)
            wc:addClient(b)
            _:expect(result).is("aaa connected bbb connected ")
        end)

We provide both callbacks, one for connection and one for messages, and, calling addClient, we see that the connections are reported. (We have a separate test for sending messages from the clients.). What we need is a new method on the server, processConnections, similar to our processClients, except that processConnections will call accept on the server, and expect a client back if there is one.

That’s as described here, from the LuaSocket spec:

server:accept()
Waits for a remote connection on the server object and returns a client object representing that connection.

If a connection is successfully initiated, a client object is returned. If a timeout condition is met, the method returns nil followed by the error string ‘timeout’. Other errors are reported by nil followed by a message describing the error.

In the case of the Fake client, when we call its ready method, it will return a message on the next call to receive. We’ll do similarly with the FakeServer: when we tell it ready, it will return a new Fake client, otherwise returning nil,”timeout” per the spec.

Let’s test:

Implementing Accept

Here’s the test. I think it’s right, and I’m sure it’ll drive me to right.

        _:test("Server accept", function()
            local result = ""
            local count = 0
            local acb = function(client)
                count = count + 1
                result = result.."client "..count.." connected "
            end
            local server = FakeServer()
            local wc = WorldConnection(server, nil, acb)
            wc:processConnections()
            _:expect(result).is()
            server:ready()
            wc:processConnections()
            _:expect(result).is("client 1 connected ")
            wc:processConnections()
            _:expect(result).is("client 1 connected client 2 connected ")
        end)

It’s rather more intricate than I’d like but most of the hassle is in setting up. The execution is just those three calls to processConnections. Which, I think, is the first error I’ll get:

4: Server accept -- TestWorldConnection:135: attempt to call a nil value (method 'processConnections')

Yep. Let’s go ahead and implement it. I’m not going for micro-TDD here, I’m spreading my wings a bit. We’ll see if they melt and I plummet into the sea.

function WorldConnection:processConnections()
    local client,err = self._server:accept()
    if client then self:addClient(client) end
end

Now I expect an error on FakeServer accept.

4: Server accept -- TestWorldConnection:63: attempt to call a nil value (method 'accept')

Perfect. Implement:

function FakeServer:init()
    self._ready = false
end

function FakeServer:accept()
    if self._ready then 
        self._ready = false
        return Fake("accept")
    else
        return nil, "timeout"
    end
end

I went wild there. Two methods. Init a ready flag and respond to accept differently depending on its value, clearing it if I use it. I could have—some would say should have—driven out each of those lines with a separate test, but I’m still stretching my wings.

Now I expect an error on ready, but I get two errors:

4: Server accept  -- 
Actual: , 
Expected: nil
4: Server accept -- TestWorldConnection:149: attempt to call a nil value (method 'ready')

Ha. The first one is an error in the test. I’m checking for empty result and should be empty string:

            local wc = WorldConnection(server, nil, acb)
            wc:processConnections()
            _:expect(result).is("")

Now I should just get the one I expect … and I do. Implemented readily:

function FakeServer:ready()
    self._ready = true
end

I remain optimistic and I expect the test to run green. If it does I’ll have to break it. So let’s break it now:

            local wc = WorldConnection(server, nil, acb)
            wc:processConnections()
            _:expect(result).is("")
            server:ready()
            wc:processConnections()
            _:expect(result).is("client 31 connected ")
            wc:processConnections()
            _:expect(result).is("client 31 connected client 2 connected ")

Now it’s checking for 31 but I really am hoping for 1, not 31. Test.

4: Server accept  -- 
Actual: client 1 connected , 
Expected: client 31 connected 
4: Server accept  -- 
Actual: client 1 connected , 
Expected: client 31 connected client 2 connected

Woot! Take that Daedalus! That’s how you do that!

Change the test back to just 1s, and test for green.

Of course it isn’t green. The test is wrong again. Fix it one more time. I take back the Woot!

            local wc = WorldConnection(server, nil, acb)
            wc:processConnections()
            _:expect(result).is("")
            server:ready()
            wc:processConnections()
            _:expect(result).is("client 1 connected ")
            wc:processConnections()
            _:expect(result).is("client 1 connected ")
            server:ready()
            wc:processConnections()
            _:expect(result).is("client 1 connected client 2 connected ")

This runs green, and I reinstate my Woot. Commit: Woot! processConnections receives clients on accept and adds them.

Let’s reflect.

Reflection

So … did I take too big a bite there? Or did it cost me time? Well, if I recall correctly—after all it was all of two minutes ago—all my mistakes were in the test, not in the code. That suggests that the test was too big, since it had both a simple error () for (“”) and a logic error, expecting a result before ready was called the second time.

And then, in my excitement, when I forced the error, I just glanced at it and thought it was right, leading to a surprise when the test failed that final time.

So … while I don’t think it cost me much time, I think that if I had built up the test a bit more incrementally, I’d have done better. My wings didn’t melt, but they did get kind of squishy.

Lesson learned? Probably not, but I hope you’ll try smaller steps and see whether you can learn what I obviously cannot.

Now let’s go the whole way and run this thing on the timer.

Running on the Timer

What we want is for a function or method to be called on a timer, such that each tick the function or method does processConnections and processClients. Let’s work up to it slowly, with tests.

I’m torn here. There’s more boilerplate in these tests than is ideal. They’d be better if we would move some of the setup up to before, but there isn’t all that much that’s completely in common, and (various other excuses) and I just don’t want to do that right now.

So I’ll create some more duplication in the next tests.

No, wait, let’s split the difference. I’ll create a new test function for these new ones, and that’ll make this part easier and then maybe some of the old ones can move.

New frame.

function test_WorldConnectionTimer()
    
    _:describe("WorldConnectionTimer", function()
        
        _:before(function()
        end)
        
        _:after(function()
        end)
        
        _:test("HOOKUP", function()
            _:expect("Foo", "testing fooness").is("Bar")
        end)
        
    end)
end

Expect hookup to fail. It does. Let’s start with a test duplicating some of the old functionality. It’ll make the next test more clear, I think, and it’s worth doing something simple to get the hookup right.

Hm, it’s a bit more intricate than I’d like, because of the need to provide the callback functions. I’ll just press on …

Here’s my first cut, pretty large:

        _:test("Connections and Messages by Hand", function()
            result = ""
            count = 0
            local acb = function(client)
                count = count + 1
                result = result.."client "..count.." connected "
            end
            local rcb = function(client,msg)
                result = result.."("..client._msg..")"
            end
            local server = FakeServer()
            local wc = WorldConnection(server, rcb, acb)
            wc:process()
            _:expect(result).is("")
        end)

I’ve invented a new method for wc, process, which will be the test’s first error:

1: Connections and Messages by Hand -- TestWorldConnection:190: attempt to call a nil value (method 'process')

And we implement:

function WorldConnection:process()
    self:processConnections()
    self:processClients()
end

I expect the test to run to its current end. It does.

Now I want the Fakes to generate their own messages if they’re not given one. I’ll just code that and hope.

function Fake:nextMsg()
    msgIndex = msgIndex +1
    if msgIndex > #msgs then
        msgIndex = 1
    end
    return msgs[msgIndex]
end

function Fake:init(msg)
    self._msg = msg or self:nextMsg()
    self._ready = false
end

That should give them messages to send. We’ll see when we test. Extend the test:

            local wc = WorldConnection(server, rcb, acb)
            wc:process()
            _:expect(result).is("")
            server:ready()
            wc:process()
            _:expect(result).is("client 1 connected ")

So far so good and I expect this to run. But then I have a problem. Test.

Green. But now I need to ready a client and I have o method to do so. I can either rip the guts out of the WorldConnection or give it an auxiliary method. Let’s do the latter:

            local wc = WorldConnection(server, rcb, acb)
            wc:process()
            _:expect(result).is("")
            server:ready()
            wc:process()
            _:expect(result).is("client 1 connected ")
            wc:readyClient(1)
            wc:process()
            _:expect(result).is("client 1 connected (aaa)")

This will drive out readyClient:

1: Connections and Messages by Hand -- TestWorldConnection:211: attempt to call a nil value (method 'readyClient')

And:

function WorldConnection:readyClient(n)
    -- testing only
    if n <= #self._clients then
        self._clients[n]:ready()
    end
end

I expect this test to run, or to learn something.

1: Connections and Messages by Hand -- TestWorldConnection:209: attempt to index a nil value (local 'client')

Well, I guess learning something is OK. What’s at 209?

            local rcb = function(client,msg)
                result = result.."("..client._msg..")"
            end

Hm, we did the callback but passed in no client or message. And why am I pulling out the _msg if there’s one sent in. Let’s see what’s going on here.

function WorldConnection:processClients()
    for i,c in ipairs(self._clients) do
        data,err = c:receive()
        if data then
            self._clientCallback(client,data)
            result = data
        end
    end
end

I find this odd. The bug is that client is in fact nil but I thought I had fixed that. And why did no tests detect it until now? Don’t know, but a test did find it, so fix:

function WorldConnection:processClients()
    for i,client in ipairs(self._clients) do
        data,err = client:receive()
        if data then
            self._clientCallback(client,data)
            result = data
        end
    end
end

And let’s improve the callback:

            local rcb = function(client,msg)
                _:expect(client).isnt(nil)
                result = result.."("..msg..")"
            end

This might serve. Wow the test fails in an interesting way:

1: Connections and Messages by Hand  -- 
Actual: accept, 
Expected: client 1 connected (aaa)

Ah, I have tricked myself:

function FakeServer:accept()
    if self._ready then 
        self._ready = false
        return Fake("accept")
    else
        return nil, "timeout"
    end
end

Let’s remove that “accept” and test again.

1: Connections and Messages by Hand  -- 
Actual: ddd, 
Expected: client 1 connected (aaa)

OK we have to reset the message counter on each test cycle, and someone isn’t appending …

        _:before(function()
            msgIndex = 0
        end)

That should change the error to “aaa”, I expect.

1: Connections and Messages by Hand  -- 
Actual: aaa, 
Expected: client 1 connected (aaa)

I must have a typo or something …

OK, that was wild. The word local was missing here:

        _:test("Connections and Messages by Hand", function()
            local result = ""
            count = 0
            local acb = function(client)
                count = count + 1
                result = result.."client "..count.." connected "
            end
            local rcb = function(client,msg)
                _:expect(client).isnt(nil)
                result = result.."("..msg..")"
            end
            local server = FakeServer()
            local wc = WorldConnection(server, rcb, acb)
            wc:process()
            _:expect(result).is("")
            server:ready()
            wc:process()
            _:expect(result).is("client 1 connected ")
            wc:readyClient(1)
            wc:process()
            _:expect(result).is("client 1 connected (aaa)")
        end)

So we have someone with a global result, which we also used … search …

function WorldConnection:processClients()
    for i,client in ipairs(self._clients) do
        data,err = client:receive()
        if data then
            self._clientCallback(client,data)
            result = data
        end
    end
end

Oh a test hack left over. Remove that and fix what breaks. Expect my current test to run.

We are green. Extend the test a bit more …

            _:expect(result).is("client 1 connected (aaa)")
            server:ready()
            wc:readyClient(1)
            wc:process()
            _:expect(result).is("client 1 connected (aaa)client 2 connected (aaa)")

Green. One more step:

            local wc = WorldConnection(server, rcb, acb)
            wc:process()
            _:expect(result).is("")
            server:ready()
            wc:process()
            _:expect(result).is("client 1 connected ")
            wc:readyClient(1)
            wc:process()
            _:expect(result).is("client 1 connected (aaa)")
            server:ready()
            wc:readyClient(1)
            wc:process()
            _:expect(result).is("client 1 connected (aaa)client 2 connected (aaa)")
            wc:readyClient(2)
            wc:process()
            _:expect(result).is("client 1 connected (aaa)client 2 connected (aaa)(bbb)")

Green. Commit: WorldConnection:process works as advertised.

I’ve been heads down for about 90 minutes since the beginning of the article, with a short reflection back there a ways. Let’s reflect again.

Reflection

So, that actually went well. With only the sort of missteps we typically make, and no real confusion we have a very complex flow of client connections and messages apparently supported in the new process method. We’re left with the task of plugging the thing into the timer. Let’s clean up this test, move setup to before and then do the timer.

I want to shorten those connect messages for convenience in typing the results.

First this, to change the connection message to just the index in square brackets:

        _:test("Connections and Messages by Hand", function()
            local result = ""
            count = 0
            local acb = function(client)
                count = count + 1
                result = result.."["..count.."]"
            end
            local rcb = function(client,msg)
                _:expect(client).isnt(nil)
                result = result.."("..msg..")"
            end
            local server = FakeServer()
            local wc = WorldConnection(server, rcb, acb)
            wc:process()
            _:expect(result).is("")
            server:ready()
            wc:process()
            _:expect(result).is("[1]")
            wc:readyClient(1)
            wc:process()
            _:expect(result).is("[1](aaa)")
            server:ready()
            wc:readyClient(1)
            wc:process()
            _:expect(result).is("[1](aaa)[2](aaa)")
            wc:readyClient(2)
            wc:process()
            _:expect(result).is("[1](aaa)[2](aaa)(bbb)")
        end)

Now let’s move the whole first bit up to before:

    _:describe("WorldConnectionTimer", function()
        local result
        local server
        local wc
        
        _:before(function()
            msgIndex = 0
            result = ""
            local count = 0
            local acb = function(client)
                count = count + 1
                result = result.."["..count.."]"
            end
            local rcb = function(client,msg)
                _:expect(client).isnt(nil)
                result = result.."("..msg..")"
            end
            server = FakeServer()
            wc = WorldConnection(server, rcb, acb)
        end)
        
        _:after(function()
        end)
        
        _:test("Connections and Messages by Hand", function()
            wc:process()
            _:expect(result).is("")
            server:ready()
            wc:process()
            _:expect(result).is("[1]")
            wc:readyClient(1)
            wc:process()
            _:expect(result).is("[1](aaa)")
            server:ready()
            wc:readyClient(1)
            wc:process()
            _:expect(result).is("[1](aaa)[2](aaa)")
            wc:readyClient(2)
            wc:process()
            _:expect(result).is("[1](aaa)[2](aaa)(bbb)")
        end)

Tests are green. Commit: move boilerplate to before.

Now a timer version of the test:

        _:test("Connections and Messages on Timer", function()
            _:expect(result).is("")
            server:ready()
            _:expect(result).is("[1]")
            wc:readyClient(1)
            _:expect(result).is("[1](aaa)")
            server:ready()
            wc:readyClient(1)
            _:expect(result).is("[1](aaa)[2](aaa)")
            wc:readyClient(2)
            _:expect(result).is("[1](aaa)[2](aaa)(bbb)")
        end)

Now at this point, I’ve not started the timer, so this version should fail all but the first expectation … and it does.

Now for the timer. We’d better save and restore it in before and after:

        _:before(function()
            tweenUpdate = tween.update
            msgIndex = 0
...
        _:after(function()
            tween.update = tweenUpdate
        end)

OK, and in this test, first just replace it:

        _:test("Connections and Messages on Timer", function()
            local processWC = function()
                wc:process()
            end
            tween.update = processWC
            _:expect(result).is("")
            server:ready()
            _:expect(result).is("[1]")
            wc:readyClient(1)
            _:expect(result).is("[1](aaa)")
            server:ready()
            wc:readyClient(1)
            _:expect(result).is("[1](aaa)[2](aaa)")
            wc:readyClient(2)
            _:expect(result).is("[1](aaa)[2](aaa)(bbb)")
        end)

I am rather hopeful that the test will run. If not, I’m going to be very sad.

It doesn’t, but I’m not as sad as I thought. I realized what would happen even before I clicked to run it.

This test can’t work, because it’ll be over long before the timer even ticks once. I can’t test this way. I’ve got to allow time for the timer to tick, and only then move on to the next step in my test.

I don’t think I can do this test in CodeaUnit at all, because it expects to run to completion. I think I’m going to have to do it in Main, probably using touch to trigger next steps.

Or … what if we put the steps we want into little functions, and triggered them on the timer? Would that even work?

Let’s comment out part of the test and try that.

A Delayed Test

OK, this is even more tricky than I thought. What needs to happen is that the test runs a bit, and gets a right answer. Then we can send a ready message and wait another tick, after which the next expectation should succeed. And so on. So we need a series of functions, running one at a time on ticks, to do an expectation, or a ready, one after another …

Here are all the functions:

        _:test("Connections and Messages on Timer", function()
            local processWC = function()
                wc:process()
            end
            tween.update = processWC
            local e1 = function()
                _:expect(result).is("")
            end
            local f2 = function()
                server:ready()
            end
            local e2 = function()
            _:expect(result).is("[1]")
            end
            local f3 = function()
                wc:readyClient(1)
            end
            local e3 = function()
            _:expect(result).is("[1](aaa)")
            end
            local f4 = function()
                server:ready()
                wc:readyClient(1)
            end
            local e4 = function()
            _:expect(result).is("[1](aaa)[2](aaa)")
            end
            local f5 = function()
                wc:readyClient(2)
            end
            local e5 = function()
            _:expect(result).is("[1](aaa)[2](aaa)(bbb)")
            end
        end)

Now I want them in a collection:

            local funcs = {e1, f2,e2, f3,e3, f4,e4, f5,e5}

And on each tick, after I call process, I want to execute the next function in the table. Like this:

        _:test("Connections and Messages on Timer", function()
            local funcs
            local processWC = function()
                wc:process()
                f = table.remove(funcs,1)
                f()
            end
...

This is way too intricate. But it might work. Let’s find out what it does anyway. If this doesn’t work, the next attempt will be even more difficult than this one.

The test runs. Let me break one assertion:

            local e5 = function()
            _:expect(result).is("break[1](aaa)[2](aaa)(bbb)")
            end

Test. Doesn’t break. Makes me think nothing is happening. No … it still can’t work that way … because nothing happens until it ticks and the test has all dropped though by then.

This is weird because I am totally certain that it’s actually going to work just fine.

OK, move the test to Main. Now that it’s all in place I can almost do it by cut and paste.

function setup()
    CodeaUnit_Detailed = false
    --CodeaUnit_Lock = false
    if CodeaUnit then 
        _.execute()
    end
    xxxtest()
end

local funcs 
local tweenUpdate
local msgIndex
local result
local wc

function xxxtest()
    --error("who called me")
    print("in test")
    tweenUpdate = tween.update
    msgIndex = 0
    result = ""
    local count = 0
    local acb = function(client)
        count = count + 1
        result = result.."["..count.."]"
    end
    local rcb = function(client,msg)
        _:expect(client).isnt(nil)
        result = result.."("..msg..")"
    end
    server = FakeServer()
    wc = WorldConnection(server, rcb, acb)
    
    
    local processWC = function()
        wc:process()
    end
    tween.update = processWC
    local e1 = function()
        _:expect(result).is("x")
    end
    local f2 = function()
        server:ready()
    end
    local e2 = function()
        _:expect(result).is("x[1]")
    end
    local f3 = function()
        wc:readyClient(1)
    end
    local e3 = function()
        _:expect(result).is("x[1](aaa)")
    end
    local f4 = function()
        server:ready()
        wc:readyClient(1)
    end
    local e4 = function()
        _:expect(result).is("x[1](aaa)[2](aaa)")
    end
    local f5 = function()
        wc:readyClient(2)
    end
    local e5 = function()
        _:expect(result).is("x[1](aaa)[2](aaa)(bbb)")
    end
    
    funcs = {e1, f2,e2, f3,e3, f4,e4, f5,e5}
end

function touched(touch)
    if touch.state~=ENDED then return end
    local f = table.remove(funcs,1)
    if f then print("f",#funcs) f() end
end

OK, we start the test in setup and then every time we touch the screen, we run another of the functions (and print f and the number remaining). They should all fail because I put those x’s in there, and they do.

(What is slightly odd is that the _:expect still works: I had thought I’d have to replace those, but there’s enough of CodeaUnit lying about that I’m sure it’s working. I’ll remove the x’s and the test will run without demur.

OK, that’s more than enough. Let’s stop, and sum up.

Summary – Well, Almost …

Well, first commit: WorldConnection can run on tween timer. Not yet fully installed. Bizarre test in Main.

OK. I really had to jump through my own orifice for that last test, because it needed to allow the Codea runtime to run and yet run the individual test chunks at intervals. So right now you have to touch the screen ten times to make it go. I think I can fix that in some other clever way.

In fact, though I am tired, let’s do it:

local runcheck
runcheck = function()
    funcCount = funcCount + 1
    if funcCount%10 == 0 then
        local f = table.remove(funcs,1)
        if f then 
            print("f",#funcs) f() 
        else
            print("test done")
            runcheck = function() end
        end
    end
    
end

function draw()
    runcheck()
...

Now on every tenth draw I run one of the functions. So it prints f 9 f 8 and so on quickly and then test done.

OK the test is now automatic, but I still had to invert a torus to do it.

I’ll not explain all that in great detail. In essence I made a series of functions to either check state or send an enabling message to allow the next state to occur. As I think about it now, it might be OK to do the expectation and the next enabling message in the same call. Separate is surely better for clarity, such as we can get.

In any case, except that it’s not setting up the update on its own, we’re nearly good … oh … except I’ll be we’re leaving tweens broken. This’ll fix that:

local runcheck
runcheck = function()
    funcCount = funcCount + 1
    if funcCount%10 == 0 then
        local f = table.remove(funcs,1)
        if f then 
            print("f",#funcs) f() 
        else
            print("test done")
            runcheck = function() end
            tween.update = tweenUpdate
        end
    end
end

Commit: Bizarre test restores tween update.

Now then:

Summary

My steps were a bit big, a bit optimistic, but I never fell into confusion. I did get close, and smaller steps would have been just as fast and maybe faster to the end result.

Lesson: Don’t program like my brother’s brother!

I think we have a solid system test for running the WorldConnection on the tween update. We need a real method to plug it in and to ensure that tweens continue to run even with the connection plugged in.

We’ll do that next time. Coding it is easy … testing it may not be. We’ll see. I’m tired now, but I feel I’m on track to have the connection working well enough to support our robot world application … still very thin, not much error handling, but there’s a solid couple of ropes across the chasm and we can build on it. I think there’s little opportunity for Eddie to come in saying we’re in big trouble on the server.

See you next time!