Codea Stats 5: Some fun.

“If this is his idea of fun, he must be a laugh riot at parties.” – A Reader

Yes, well. Nonetheless, I plan to have some fun here and you can’t stop me. Here’s my cunning plan:

Create a new object, MatchMaker, that holds a pattern, and that can be passed tab contents, producing a smart collection of whatever object the MatchMaker matches, such as MethodDefinition or ClassDefinition objects.

In principle, when I thought of the idea, it sounded easy to me. But there are issues, including:

• How does MM know what class to create?
• How does it know what arguments the class wants in its creation?
• How does it know what kind of collection should be used to hold the created objects?

We’d like to finish up with something nice. Something that’s clear and habitable. Oh, yes, and useful.

Danger, Will Robinson

What we have here is a clear instance of going beyond the requirements of our little program. Now, if we were going to do a lot of these pattern recognition things, this object might pay off. I expect it will be easier to use, and it will hide more of the specifics of the pattern mechanism. But I’m doing it because I think it will be nifty, and that we’ll learn something, so this is the sort of thing that a real-world team needs to be a bit careful with.

That said, I expect we’re only talking about a couple of hours, counting article-writing time, which takes more of my elapsed time than the code ever does. At that scale, doing something like this can be pretty reasonable even on a real product.

Expectations

I think this thing may either replace some of the existing tests, or allow us to remove them. This happens when you change things. We’ll see if we can slide them in without too much test changing.

Let’s begin.

Begin With a Test

This is about as simple as I want to make it:

        _:test("MatchMaker finds classes", function()
            local mm = MatchMaker()
            local tab = readProjectTab("SampleStatsProject:SampleCode")
            mm:process(tab)
            local coll = mm:collection()
            _:expect(coll:count()).is(2)
        end)

Note that I’m not providing any info to the MM: I’m going to discover what it needs to know. I am going to do a kind of “fake it till you make it” approach. My plan is to write just enough code to make the test run, and then make the test a bit harder to pass. Rinse, repeat.

Test.

1: MatchMaker finds classes -- MatchMaker:14: attempt to call a nil value (global 'MatchMaker')

So far so good.

MatchMaker = class()

1: MatchMaker finds classes -- MatchMaker:16: attempt to call a nil value (method 'process')

MatchMaker = class()

function MatchMaker:process(tabContents)
    
end

1: MatchMaker finds classes -- MatchMaker:17: attempt to call a nil value (method 'collection')

function MatchMaker:collection()
    
end

I’m just putting in dummy methods, driving out the basic shape of the program. This single test keeps failing, telling me the next thing to do. I expect now that it will complain that what comes back from collection doesn’t understand count.

1: MatchMaker finds classes -- MatchMaker:18: attempt to index a nil value (local 'coll')

Yes, that’s it. I think I want to enhance the test to check the return type of coll.

        _:test("MatchMaker finds classes", function()
            local mm = MatchMaker()
            local tab = readProjectTab("SampleStatsProject:SampleCode")
            mm:process(tab)
            local coll = mm:collection()
            local isClass = coll:is_a(ClassCollection)
            _:expect(isClass, "class collection").is(true)
            _:expect(coll:count()).is(2)
        end)

I think that’s the trick. Test. Still get the nil fail. Nil doesn’t understand is_a, only classes do.

function MatchMaker:collection()
    return ClassCollection({})
end

That’s how we build a ClassCollection. I don’t like the look of that today. Run the test.

1: MatchMaker finds classes  -- Actual: 0, Expected: 2

Right kind of thing, wrong answer.

Now before we make this work, a bit of thinking, always useful in these situations. To date, we have been creating ordinary Lua arrays and then wrapping them at the last minute in suitable smart collections. In our MatchMaker, let’s create the collection empty, and add things to it.

We prepare for that thus:

function MatchMaker:init()
    self.coll = ClassCollection()
end

function MatchMaker:process(tabContents)
    
end

function MatchMaker:collection()
    return self.coll
end

We run the test again, still expecting 0 instead of 2. We do not. We get this:

1: MatchMaker finds classes -- TabTests:177: attempt to get length of a nil value (field 'classDefinitions')

I forgot to include the empty table on the create. But let’s do better, making ClassCollection a bit smarter:

function ClassCollection:init(classTable)
    self.classDefinitions = classTable or {}
end

Now test. Back to this as intended:

1: MatchMaker finds classes  -- Actual: 0, Expected: 2

The creation method just provides an empty table if ClassCollection is created with no parameters.

Now let’s get the count fixed. We need two things to pass.

function MatchMaker:process(tabContents)
    self.coll:add(123)
    self.coll:add(234)
end

I’m just adding numbers in for now. Faking it after all. I think ClassCollection doesn’t know add.

1: MatchMaker finds classes -- MatchMaker:33: attempt to call a nil value (method 'add')

Right, Provide it:

function ClassCollection:add(anItem)
   table.insert(self.classDefinitions, anItem) 
end

We don’t care what the item is. Should we? We’ll talk about that in a moment.

The test now runs, because there are two items in the class collection. No one checked to see if they might actually be ClassDefinition instances. Yet.

Let’s test the insides by asking the collection for a report.

        _:test("MatchMaker finds classes", function()
            local mm = MatchMaker()
            local tab = readProjectTab("SampleStatsProject:SampleCode")
            mm:process(tab)
            local coll = mm:collection()
            local isClass = coll:is_a(ClassCollection)
            _:expect(isClass, "class collection").is(true)
            _:expect(coll:count()).is(2)
            local rep = coll:report()
            _:expect(rep).is("    AAA\n    BBB\n")
        end)

Now we expect to get two classes AAA and BBB. Those four spaces at the beginning kind of bug me.

My target for a while has been to slowly enhance the MatchMaker by making the test harder, until we have actually processed the tab. It occurs to me that I could create a fake tab here, in line, with AAA and BBB and be done with it. But let’s stick with plan A.

function MatchMaker:process(tabContents)
    self.coll:add(ClassDefinition("AAA",""))
    self.coll:add(ClassDefinition("BBB",""))
end

Test runs. This is too good. Change the implementation to be wrong just to see the test fail.

function MatchMaker:process(tabContents)
    self.coll:add(ClassDefinition("AAA",""))
    self.coll:add(ClassDefinition("XXX",""))
end

1: MatchMaker finds classes  -- Actual:     AAA
    XXX
, Expected:     AAA
    BBB

Perfect. Put the BBB back and enhance the test.

I’m tempted to go whole hog and read the tab, but let’s do a subclass first.

        _:test("MatchMaker finds classes", function()
            local mm = MatchMaker()
            local tab = readProjectTab("SampleStatsProject:SampleCode")
            mm:process(tab)
            local coll = mm:collection()
            local isClass = coll:is_a(ClassCollection)
            _:expect(isClass, "class collection").is(true)
            _:expect(coll:count()).is(2)
            local rep = coll:report()
            _:expect(rep).is("    AAA\n    BBB(AAA)\n")
        end)

Fails of course. Fix the fake:

function MatchMaker:process(tabContents)
    self.coll:add(ClassDefinition("AAA",""))
    self.coll:add(ClassDefinition("BBB","AAA"))
end

Test runs. Enhance the test once more, to expect the actual report from the tab, which we have in another test.

        _:test("MatchMaker finds classes", function()
            local mm = MatchMaker()
            local tab = readProjectTab("SampleStatsProject:SampleCode")
            mm:process(tab)
            local coll = mm:collection()
            local isClass = coll:is_a(ClassCollection)
            _:expect(isClass, "class collection").is(true)
            _:expect(coll:count()).is(2)
            local rep = coll:report()
            _:expect(rep).is("    SampleCode\n    SubclassOfSample(SampleCode)\n")
        end)

Test fails of course:

1: MatchMaker finds classes  -- Actual:     AAA
    BBB(AAA)
, Expected:     SampleCode
    SubclassOfSample(SampleCode)

Nothing for it now but to process the tab.

function MatchMaker:process(tabContents)
    local pattern = PatternDictionary:classDefinition()
    local matches = tabContents:gmatch(pattern)
    for className, superclassName in matches do
        local instance = ClassDefinition(className, superclassName)
        self.coll:add(instance)
    end
end

The test should run … and it does!

Commit: MatchMaker processes class definitions.

Let’s reflect.

Reflection

So. That was fun, wasn’t it? We moved in many small steps to create a new MatchMaker class, whose instances can create a ClassCollection of ClassDefinition objects, given input tabs.

Note that the object is rigged so that you can process as many tabs as you want before pulling out the collection. We didn’t test that. Should we? Let’s do.

        _:test("MatchMaker finds classes", function()
            local mm = MatchMaker()
            local tab = readProjectTab("SampleStatsProject:SampleCode")
            mm:process(tab)
            local coll = mm:collection()
            local isClass = coll:is_a(ClassCollection)
            _:expect(isClass, "class collection").is(true)
            _:expect(coll:count()).is(2)
            local rep = coll:report()
            _:expect(rep).is("    SampleCode\n    SubclassOfSample(SampleCode)\n")
            local tinyTab = "Argle = class()"
            mm:process(tinyTab)
            _:expect(coll:count()).is(3)
            _:expect(coll:report()).is("    SampleCode\n    SubclassOfSample(SampleCode)\n    Argle\n")
        end)

I’ve added a tinyTab with a simple class definition. It shows up in the report as intended.

Quiz: why didn’t I refetch coll?

Commit: tested processing more than one tab.

Now where were we?

Oh, right. Our mission is to create a MatchMaker that can provide for all our various match types (we only have two for now). This one is seriously hard-wired to ClassDefinition and ClassCollection.

Now we want to extend it to deal with the MethodDefinition and MethodCollection.

Let’s suppose that we’ll have class methods:

MatchMaker:classDefinition()
MatchMaker:methodDefinition()

And those class methods will return a suitable MathcMaker.

Change the test to use the posited class method.

        _:test("MatchMaker finds classes", function()
            local mm = MatchMaker:classDefinition()
            local tab = readProjectTab("SampleStatsProject:SampleCode")
...

Write the method:

function MatchMaker:classDefinition()
    return MatchMaker()
end

Test should run. It even does. Now we start passing in pluggable parameters:

function MatchMaker:classDefinition()
    return MatchMaker(ClassCollection)
end

function MatchMaker:init(collectionType)
    self.coll = collectionType()
end

Now the collection type is defined based on what kind of definition we have in mind. We begin to see the pattern. Let’s do the instance type:

function MatchMaker:classDefinition()
    return MatchMaker(ClassCollection, ClassDefinition)
end

function MatchMaker:init(collectionType, definitionType)
    self.coll = collectionType()
    self.defn = definitionType
end

function MatchMaker:process(tabContents)
    local pattern = PatternDictionary:classDefinition()
    local matches = tabContents:gmatch(pattern)
    for className, superclassName in matches do
        local instance = self.defn(className, superclassName)
        self.coll:add(instance)
    end
end

We pass in the definition type and use it in process. Test still runs.

Now we’d better pass in the pattern.

function MatchMaker:classDefinition()
    return MatchMaker(ClassCollection, ClassDefinition, PatternDictionary:classDefinition())
end

function MatchMaker:init(collectionType, definitionType, pattern)
    self.coll = collectionType()
    self.defn = definitionType
    self.pattern = pattern
end

function MatchMaker:process(tabContents)
    local matches = tabContents:gmatch(self.pattern)
    for className, superclassName in matches do
        local instance = self.defn(className, superclassName)
        self.coll:add(instance)
    end
end

Now the MatchMaker is completely parameterized (for class definitions only) Commit: MatchMaker creates necessary parameters for class definitions.

Now I reckon we’d better make the method one work. I expect some fiddling there, dealing with the fact that the MethodDefinition expects three parameters, unlike ClassDefinition’s two.

A new test. We’ll just write the hard version out, expecting that all we should need to do–mostly–is a new set of parameters.

Oops. I just realized that we’ve not yet written a report method for the MethodCollection. Well, we’ll find that out soon enough, won’t we?

For how here’s my test, to be enhanced as needed:

        _:test("MatchMaker finds methods", function()
            local mm = MatchMaker:methodDefinition()
            local tab = readProjectTab("SampleStatsProject:SampleCode")
            mm:process(tab)
            local coll = mm:collection()
            local isClass = coll:is_a(MethodCollection)
            _:expect(isClass, "method collection").is(true)
            _:expect(coll:count()).is(5)
            local rep = coll:report()
            _:expect(rep).is("    report\n")
        end)

Run expecting no methodDefinition:

2: MatchMaker finds methods -- MatchMaker:30: attempt to call a nil value (method 'methodDefinition')

function MatchMaker:methodDefinition()
    return MatchMaker(MethodCollection, MethodDefinition, PatternDictionary:methodDefinition())
end

I think this might fail on report. We’ll see what it does.

Ah.

2: MatchMaker finds methods -- MatchMaker:64: attempt to call a nil value (method 'add')

We need an add method in MethodCollection.

function MethodCollection:add(anItem)
    table.insert(self.methodDefinitions, anItem)
end

This is why we like the tests. They remind us of things we would otherwise have to remember … or perhaps forget, to our later disappointment.

2: MatchMaker finds methods -- TabTests:142: bad argument #1 to 'insert' (table expected, got nil)

We need to create the table on the nil creation.

function MethodCollection:init(methodTable)
    self.methodDefinitions = methodTable or {}
end

2: MatchMaker finds methods -- MatchMaker:37: attempt to call a nil value (method 'report')

OK, now we need a report. I don’t even know what’s in there, five methods of some kind. Let’s see how we do this. ClassCollection does this:

function ClassCollection:report()
    local tab = "    "
    result = ""
    for _i,classDef in ipairs(self.classDefinitions) do
        result = result..tab..classDef:report().."\n"
    end
    return result
end

function ClassDefinition:report()
    local result = self.className
    if self.superclassName and self.superclassName ~= "" then
        result = result.."("..self.superclassName..")"
    end
    return result
end

We’ll follow that lead:

function MethodCollection:report()
    local tab = "    "
    local report = ""
    for _i, methodDef in ipairs(self.methodDefinitions) do
        report = report..tab..methodDef:report().."\n"
    end
    return report
end

function MethodDefinition:report()
    return self.className..":"..self.methodName.."("..self.args..")"
end

Let’s see what we get.

2: MatchMaker finds methods -- TabTests:136: attempt to concatenate a nil value (field 'args')

Hm yes I’m pretty sure we aren’t even setting that. Let’s patch report for now.

function MethodDefinition:report()
    return self.className..":"..self.methodName.."("..(self.args or "")..")"
end

We get a fairly nice result:

2: MatchMaker finds methods  -- Actual:     SampleCode:init()
    SampleCode:draw()
    SampleCode:touched()
    SubclassOfSample:init()
    SampleCode:extraMethod()
, Expected:     report

However, we know it’s not perfect. Here’s our sample page:

-- SampleCode
-- RJ 20211021

SampleCode = class()

function SampleCode:init(x)
    -- you can accept and set parameters here
    self.x = x
end

function SampleCode:draw()
    -- Codea does not automatically call this method
end

function SampleCode:touched(touch)
    -- Codea does not automatically call this method
end

SubclassOfSample = class(SampleCode)

function SubclassOfSample:init(ignored)
    
end

function SampleCode:extraMethod(a,b,c)
    -- blah
end

We want all those arguments. It’s a good start though.

Let’s see about what goes on in process:

function MatchMaker:process(tabContents)
    local matches = tabContents:gmatch(self.pattern)
    for className, superclassName in matches do
        local instance = self.defn(className, superclassName)
        self.coll:add(instance)
    end
end

We’ll need to generalize that for loop. I think we can just give it as many parameters as our largest definition class expects:

function MatchMaker:process(tabContents)
    local matches = tabContents:gmatch(self.pattern)
    for cap1, cap2, cap3 in matches do
        local instance = self.defn(cap1,cap2,cap3)
        self.coll:add(instance)
    end
end

Let’s see what this does and then if it works as I expect I’ll explain. If it doesn’t, well, I’ll try to explain that.

2: MatchMaker finds methods  -- Actual:     SampleCode:init(x)
    SampleCode:draw()
    SampleCode:touched(touch)
    SubclassOfSample:init(ignored)
    SampleCode:extraMethod(a,b,c)
, Expected:     report

OK, I’m calling that sweet. What happened? Well, the gmatch iterator returns some number of items, however many captures the pattern has. Right now, the most we ever get is three, so I ask for three and pass them to the class creation. If there are fewer, like, say, two, then cap3 is nil. And since the ClassDefinition only expects two, it ignores the extra nil parm. This report is correct. I’ll change the test to expect it.

I try this:

        _:test("MatchMaker finds methods", function()
            local expected = [[    SampleCode:init(x)
            SampleCode:draw()
            SampleCode:touched(touch)
            SubclassOfSample:init(ignored)
            SampleCode:extraMethod(a,b,c)
]]
            local mm = MatchMaker:methodDefinition()
            local tab = readProjectTab("SampleStatsProject:SampleCode")
            mm:process(tab)
            local coll = mm:collection()
            local isClass = coll:is_a(MethodCollection)
            _:expect(isClass, "method collection").is(true)
            _:expect(coll:count()).is(5)
            local rep = coll:report()
            _:expect(rep).is(expected)
        end)

The result fails but it’s not clear why at first glance:

2: MatchMaker finds methods  -- Actual:     SampleCode:init(x)
    SampleCode:draw()
    SampleCode:touched(touch)
    SubclassOfSample:init(ignored)
    SampleCode:extraMethod(a,b,c)
, Expected:     SampleCode:init(x)
            SampleCode:draw()
            SampleCode:touched(touch)
            SubclassOfSample:init(ignored)
            SampleCode:extraMethod(a,b,c)

I’ll insert some extra returns into this report:

2: MatchMaker finds methods  -- Actual: 
    SampleCode:init(x)
    SampleCode:draw()
    SampleCode:touched(touch)
    SubclassOfSample:init(ignored)
    SampleCode:extraMethod(a,b,c)
, Expected: 
    SampleCode:init(x)
            SampleCode:draw()
            SampleCode:touched(touch)
            SubclassOfSample:init(ignored)
            SampleCode:extraMethod(a,b,c)

Hm 12 spaces. I think I laid that out seriously wrongly:

        _:test("MatchMaker finds methods", function()
            local expected = [[    SampleCode:init(x)
    SampleCode:draw()
    SampleCode:touched(touch)
    SubclassOfSample:init(ignored)
    SampleCode:extraMethod(a,b,c)
]]
...

Test runs.

Commit: MatchMaker understands classDefinition and methodDefinition, recognizes both types correctly.

We’re where we intended to be. We even have a new “feature”, the method report sketch. Let’s sum up.

Summary

So that was fun. There were almost no missteps during the whole cycle. Why? Because we took “Many More Much Smaller Steps” than even my usual small steps.

We had in mind a very general class, MatchMaker, that can do the parsing and collection making for any pattern we may ever devise. But we developed a single version of it, for class definitions (because they are simpler). We wrote an easy test, made it work, extended the test, made it work, rinse repeat until we have a MathMaker that was entirely custom made for class definitions.

Then we generalized it bit by bit, by providing first the collection class to use, then the definition class to use, then the pattern to use.

At that point, we had a class, custom made for class definitions, but generalized inside. We wrote a test for the method definition case, chucked in the definitions for that, discovered methods that we needed on MethodCollection and MethodDefinition, provided those methods, and inch by inch, step by step, slowly we turned our single-purpose class into one that handles both our cases perfectly.

That is what I call fun!

See you next time!