Spacewar! 15 - Evolving a testing framework.

What’s up today?

Today as a special Tuesday surprise, Tozier will be joining me, and we’ll work on tests. My guess is that we’ll shave the testing framework yak a bit, having produced a number of tests that presently run in Main. There are a number of ways we can go, and I think we’ll choose the path of evolving a test framework as we go. That will be more fun, for folks like us, and we’ll learn a bit more about the intricacies of Lua.¹

Let’s review the testing code as it stands today:

-- S3 Spacewar

function setup()
    local offset = 200
    ship1 = Ship(vec2(WIDTH/2 + offset, HEIGHT/2), 0)
    ship2 = Ship(vec2(WIDTH/2 - offset, HEIGHT/2), 180)
    
    ship1.turn = 0.5
    ship2.turn = -0.4
    button1accel = Button(ship1, WIDTH-100, HEIGHT/5, "accelerate")
    parameter.action("Test Button", testButton)
--    testButton()
end

local testResult = ""
function testButton()
    local button = Button(nil, 100, 100, "nothing")
    local insideEndTouch    = { id=1, state=ENDED, x=100, y=100 }
    local outsideEndTouch   = { id=2, state=ENDED, x=151, y=100 }
    
    assertTrue(button:insideCircle(insideEndTouch), "should be inside")
    
    assertFalse(button:insideCircle(outsideEndTouch), "should be outside")
    
    assertEqual(button.capturedID, nil, "initial captured id should be nil")
    assertFalse(button.pressed, "button better not be pressed")
    
    testTouch(button, insideEndTouch,
        {function() assertEqual(button.capturedID, nil, "captured id should still be nil") end,
         function() assertFalse(button.pressed, "button still not pressed") end } )
    
    -- test that we do not lose a capture without an ENDED
    local insideBeganTouch  = { id=3, state=BEGAN, x=100, y=100 }
    local outsideBeganTouch = { id=4, state=BEGAN, x=151, y=100 }
    testTouch(button, insideBeganTouch,
        {function() assertEqual(button.capturedID, 3, "should capture the id") end } )
    testTouch(button, outsideBeganTouch,
        {function() assertEqual(button.capturedID, 3, "id should be 3") end ,
         function() assertFalse(button.pressed, "outside touch erroneously captured a press") end } )
    
    -- sequence test
    local closeThree = { id=3, state=ENDED, x=200, y=200 }
    local movingFinger = { id=5, state=BEGAN, x = 100, y=100}
    testTouch(button, closeThree,
        {function() assertEqual(button.capturedID, nil, "should have lost captured value") end} )
    testTouch(button, movingFinger,
        {function() assertEqual(button.capturedID, 5, "captured id should be 5") end,
         function() assertTrue(button.pressed, "now it is pressed") end } )
    movingFinger.state = MOVING
    testTouch(button, movingFinger, 
        { function() assertTrue(button.pressed, "moving inside should remain pressed") end } )
    movingFinger.x = 200
    testTouch(button, movingFinger,
        {function() assertEqual(button.capturedID, 5, "moving outside does not lose capture") end,
         function() assertFalse(button.pressed, "moving outside loses press") end
        }
    )
    
    print(testResult)
end

function testTouch(b, touch, tests)
    b:touched(touch)
    for _,test in ipairs(tests) do
        test()
    end
end

function touched(touchid)
    button1accel:touched(touchid)
end

function assertTrue( boolean, message )
    if not boolean then 
        testResult = testResult .. "\n" .. message .. "\n"
    else 
        testResult = testResult .. "."
    end
end

function assertFalse(boolean, message)
    return assertTrue( not boolean, message)
end

function assertEqual( actual, expected, message )
    return assertTrue(actual == expected, message)
end

This is basically everything from Main except the draw function. It’s not terribly badly factored, but of course none of this actually belongs in Main. It “should” be in a testing class of some kind. Doubtless the yak that really needs shaving is the one where this test class is a subclass of some basic testing framework class, with the asserts and such up there, yadda yadda. We’ll not get there today, unless we’re very lucky. But we’ll get to some better place.

Note, by the way, that little change at the top:

    parameter.action("Test Button", testButton)
--    testButton()

That’s an idea I stole yesterday from somewhere. It creates a button in the parameter space of the screen, and if you press the button, it calls the function testButton. The comment below it is the code that used to be there. Are you wondering why there are no parens after testButton in the parameter statement? That’s because testButton is the name of the function to be called, while testButton() would be an actual call. We want only to call when the button is pressed, and that’s how you do that. The screen now looks like this when we run:

IMAGE HERE

See the little line of dots in the output section? I pressed the button before I took the picture, and that’s the output of the tests. Note that we save the results, append a dot for each running test, and put in a diagnostic for a test that doesn’t work. I’ll add a failing test so you can see what happens:

    assertEqual(3, 2+2, "two plus two should be four")

IMAGE HERE

So that’s what happens: the output is a bit more compact. Less verbose. Doesn’t include so much information. Focuses on just the important things, the tests that are failing. Keeps you on your toes without droning on and on. You get the idea …

Let’s get to work

Our first move will be to blindly move all the testing-related code to a new class:

TestButton = class()

function TestButton:init(x)
end


local testResult = ""
function testButton()
    local button = Button(nil, 100, 100, "nothing")
    local insideEndTouch    = { id=1, state=ENDED, x=100, y=100 }
    local outsideEndTouch   = { id=2, state=ENDED, x=151, y=100 }
    
    assertTrue(button:insideCircle(insideEndTouch), "should be inside")
    
    assertFalse(button:insideCircle(outsideEndTouch), "should be outside")
    
    assertEqual(button.capturedID, nil, "initial captured id should be nil")
    assertFalse(button.pressed, "button better not be pressed")
    
    testTouch(button, insideEndTouch,
        {function() assertEqual(button.capturedID, nil, "captured id should still be nil") end,
         function() assertFalse(button.pressed, "button still not pressed") end } )
    
    -- test that we do not lose a capture without an ENDED
    local insideBeganTouch  = { id=3, state=BEGAN, x=100, y=100 }
    local outsideBeganTouch = { id=4, state=BEGAN, x=151, y=100 }
    testTouch(button, insideBeganTouch,
        {function() assertEqual(button.capturedID, 3, "should capture the id") end } )
    testTouch(button, outsideBeganTouch,
        {function() assertEqual(button.capturedID, 3, "id should be 3") end ,
         function() assertFalse(button.pressed, "outside touch erroneously captured a press") end } )
    
    -- sequence test
    local closeThree = { id=3, state=ENDED, x=200, y=200 }
    local movingFinger = { id=5, state=BEGAN, x = 100, y=100}
    testTouch(button, closeThree,
        {function() assertEqual(button.capturedID, nil, "should have lost captured value") end} )
    testTouch(button, movingFinger,
        {function() assertEqual(button.capturedID, 5, "captured id should be 5") end,
         function() assertTrue(button.pressed, "now it is pressed") end } )
    movingFinger.state = MOVING
    testTouch(button, movingFinger, 
        { function() assertTrue(button.pressed, "moving inside should remain pressed") end } )
    movingFinger.x = 200
    testTouch(button, movingFinger,
        {function() assertEqual(button.capturedID, 5, "moving outside does not lose capture") end,
         function() assertFalse(button.pressed, "moving outside loses press") end
        }
    )
    
    print(testResult)
end

function testTouch(b, touch, assertions)
    b:touched(touch)
    for _,assertion in ipairs(assertions) do
        assertion()
    end
end

function assertTrue( boolean, message )
    if not boolean then 
        testResult = testResult .. "\n" .. message .. "\n"
    else 
        testResult = testResult .. "."
    end
end

function assertFalse(boolean, message)
    return assertTrue( not boolean, message)
end

function assertEqual( actual, expected, message )
    return assertTrue(actual == expected, message)
end

Pressing the test button runs the tests. This surprised us for a moment until we realized that while we have all that code “in” a class, all the functions are still global. So the button found them. We need to make the tests all be methods of the class. As soon as we do this:

function TestButton:testButton()
...
end

Our parameter button does nothing. (Interestingly, it does not fail with a message, as an explicit call would. It just does nothing. We change it thus, still in main, and the tests run:

    parameter.action("Test Button", TestButton().testButton)

We need to do a few things, including:

Move the button into the test class;
Pull out some of the inner tests into separate tests;
Change more of the internal functions from global to be in the TestButton class;
Move the local result inside somehow;
Initialize it on the button press, because now if we press it multiple times we get a longer and longer string of dots;
Make the TestButton class find all those inner tests and run them.

The actual work will be to write a separate intentionally-breaking test, outside our main testButton function, and observe that it is not called. Then we’ll make the TestButton class smart enough to find all the tests inside itself. Then we can begin to pull out all the real tests one at a time. We add this would-be failing tests and it is not called. (We don’t really know that. We know it didn’t fail, and we think we know why. Are we correct? The answer may surprise you. At least if you’ve been paying attention.

function TestButton:testShouldFail()
    assertEqual(3, 2+2, "Two plus two is supposed to be four!")
end

Our button runs the tests, but we do not get the failing test. Our plan now is to point the button to a function in the TestButton class which collects up all the methods whose names begin with “test”, and call them. First, we change the button:

    parameter.action("Test Button", TestButton().runAllTests)

Now the button does nothing, because there is no runAllTests and parameter buttons appear not to care. We write runAllTests, make all the global functions in the class into methods, and use self: all over the place:

TestButton = class()

function TestButton:init()
end

function TestButton:runAllTests()
    self.result = ""
    for functionName, func in pairs(TestButton) do
        if string.find(functionName,"test") == 1 then
            func(self) 
        end
    end
    print(self.result)
end

function TestButton:testShouldFail()
    self:assertEqual(3, 2+2, "Two plus two is supposed to be four!")
end

function TestButton:testButton()
    local button = Button(nil, 100, 100, "nothing")
    local insideEndTouch    = { id=1, state=ENDED, x=100, y=100 }
    local outsideEndTouch   = { id=2, state=ENDED, x=151, y=100 }
    
    self:assertTrue(button:insideCircle(insideEndTouch), "should be inside")
    
    self:assertFalse(button:insideCircle(outsideEndTouch), "should be outside")
    
    self:assertEqual(button.capturedID, nil, "initial captured id should be nil")
    self:assertFalse(button.pressed, "button better not be pressed")
    
    testTouch(button, insideEndTouch,
        {function() self:assertEqual(button.capturedID, nil, "captured id should still be nil") end,
         function() self:assertFalse(button.pressed, "button still not pressed") end } )
    
    -- test that we do not lose a capture without an ENDED
    local insideBeganTouch  = { id=3, state=BEGAN, x=100, y=100 }
    local outsideBeganTouch = { id=4, state=BEGAN, x=151, y=100 }
    testTouch(button, insideBeganTouch,
        {function() self:assertEqual(button.capturedID, 3, "should capture the id") end } )
    testTouch(button, outsideBeganTouch,
        {function() self:assertEqual(button.capturedID, 3, "id should be 3") end ,
         function() self:assertFalse(button.pressed, "outside touch erroneously captured a press") end } )
    
    -- sequence test
    local closeThree = { id=3, state=ENDED, x=200, y=200 }
    local movingFinger = { id=5, state=BEGAN, x = 100, y=100}
    testTouch(button, closeThree,
        {function() self:assertEqual(button.capturedID, nil, "should have lost captured value") end} )
    testTouch(button, movingFinger,
        {function() self:assertEqual(button.capturedID, 5, "captured id should be 5") end,
         function() self:assertTrue(button.pressed, "now it is pressed") end } )
    movingFinger.state = MOVING
    testTouch(button, movingFinger, 
        { function() self:assertTrue(button.pressed, "moving inside should remain pressed") end } )
    movingFinger.x = 200
    testTouch(button, movingFinger,
        {function() self:assertEqual(button.capturedID, 5, "moving outside does not lose capture") end,
         function() self:assertFalse(button.pressed, "moving outside loses press") end
        }
    )
end

function testTouch(b, touch, assertions)
    b:touched(touch)
    for _,assertion in ipairs(assertions) do
        assertion()
    end
end

function TestButton:assertTrue( boolean, message )
    if not boolean then 
        self.result = self.result .. "\n" .. message .. "\n"
    else 
        self.result = self.result .. "."
    end
end

function TestButton:assertFalse(boolean, message)
    return self:assertTrue( not boolean, message)
end

function TestButton:assertEqual( actual, expected, message )
    return self:assertTrue(actual == expected, message)
end

We also had to move the result up into a member variable, you’ll notice. Now our should-fail test fails and the others all show the dots.

Frankly this was more editing than we were comfortable with, but the refactoring called for it, and fortunately we had tests that would have failed had we not done it right. By some miracle, we did it right. Now we have our tests moved to a testing class (named, by convention, Test[whatever]) and the class is finding everything that starts with “test” and running it. One concern would be that if we had a member variable called, oh, testCount or testResults, it would try to call it as if it were a test method. We will fix that when it happens.

So. Good for today: our tests are moved into a single test case. We can now pull out small tests from that one big one, one at a time. We’ll do that next time. See you in the sequel.

IMAGE HERE

I am reminded that in some Heinlein novel, Podkayne of Mars if I’m not mistaken, the heroine, a young girl, having only read the word “intricacies”, thought it was pronounced “intrissiseas”. Which it isn’t, as we all know. But I digress. ↩