I have in mind small things for today, starting with an interesting and confusing mistake left over from yesterday.

Yesterday’s version in the zip file had an interesting mistake. Right after the correct implementation of a method, there was an empty definition of the same method. The tests did not run. Somehow this means that I must have started a refactoring for that method, then decided that I couldn’t do it, and left the shell there. I often create a second method of the same name when I’m doing a new version of a method, to give me a convenient reference to the one that works.

Sure enough, we closed yesterday with me deciding that I couldn’t use scopeSet in getScopeSet, even after I had changed the definition of scopeSet to work well with the statistics capability. And then I didn’t run the tests one more time before shipping.

I’ve shipped the good version already, so if you’re just a bit behind on downloading the Codea Lua source that you probably have no use for, well, you’re OK now.

I am no longer confused: I see in the captain’s log where I made the mistake. Since Codea is what it is, I think all I can do is be more careful. I know of no good way to cause exporting a Codea project to run its tests first. In fact, I know of no way at all. I don’t think iOS Shortcuts is up to that, but I’ll double check. Later.

I found the problem last night, on my other iPad, where I was browsing the code and playing with the idea of making Lua tables act like sets, and with giving them the support functions we have on sets, like exists and every. There are at least two ways we could do that last bit.

In my browsing, I found an interesting mistake.

Mistake

Recall that XSet class is a provider of general set methods, and that it relies on the XSet instance’s data member to be an object that can at least do hasAt and elements, which is sufficient to allow al the set operations to run.

I think it’s interesting how closely this adheres to mathematical set theory. In set theory, everything about set operations is defined in terms of the null set, and the element operation, which is essentially a query: does this element exist?

Everything else is defined in terms of element, with a kind of a glitch. Suppose we wanted to define the notion that Y is a subset of X. In math we’d say something like this:

Set Y is a subset of Set X iff for every y element of Y, y is an element of X.

Mathematically, the “for every” is a mathematical notion and doesn’t imply that we can enumerate the set. (In fact, set theory works fairly well on sets that literally cannot be enumerated.) But here on our computer, we need to check that the condition is true for every element. So we need the elements iterator to allow us to implement a computer kind of set theory.

So that’s neat: we come down to the same simple notions at the core of our implementation as we’d find in the math. I take that as a clue that there’s something good about the design.

But I digress. We’re here to talk about a mistake. When we create an XSet, it creates the instance with an XGeneric as its data member. This is the bottom-level set implementation that we get if we don’t do something special. But we can do things that are special.

We have the CSVSet, which is a set that allows us to process a file of CSV lines as if it were a set. That’s pretty nifty and I’m proud of it. But there’s a mistake. Patience, we’re almost there.

To create an XSet with something other than an XGeneric data element, we use XSet:on(aDataSet). It looks like this:

function XSet:on(aDataSet)
    self.data = XGeneric()
    local result = XSet()
    result.data = aDataSet
    return result
end

on is used as a class method. self, in this code, refers to the XSet class. So setting its data element doesn’t make sense. It’s harmless, but it’s wrong. We can and should remove it.

I’ve changed, tested, and committed both the changes mentioned here so far. Reviewing the XStatistics class, there’s still stuff not to like.

function XStatistics:getScopeSet()
    local groupFields = self.control.group or {}
    local scopeSet = XSet()
    for i,field in ipairs(groupFields) do
        scopeSet:addAt(field,field)
    end
    return scopeSet
end

We could inline the local:

function XStatistics:getScopeSet()
    local scopeSet = XSet()
    for i,field in ipairs(self.control.group or {}) do
        scopeSet:addAt(field,field)
    end
    return scopeSet
end

That puts a rather complex expression inside the ipairs, and I’m not sure that I really like it. I don’t think I’ve done that before, and I’m sure I don’t do it often. I think we’ll not do that after all. I tried it on, it didn’t fit.

What we have going on here is in essence a reduce function, but it’s operating on a table rather than a set, so we can’t use our XSet:reduce. But we could … if tables were sets.

This is a bit incestuous, but in Lua we do have very nice table handling, and we can express both arrays (tuples) and hashed tables, which are a lot like records.

Let’s see if we can write some tests and make tables act like simple sets.

We already have a decent array implementation in Tuple, so let’s see about something for arrays that look like this:

{ last="Jeffries",first="Ron",city="Pinckney"}

        _:test("Table as XSet", function()
            local tab = {"a",x=5, "b","c","d",last="Jeffries"}
            local set = XSet:fromTable(tab)
            _:expect(set:at("1")).is("a")
            _:expect(set:at("x")).is("5")
            _:expect(set:at("2")).is("b")
            _:expect(set:at("3")).is("c")
            _:expect(set:at("4")).is("d")
            _:expect(set:at("last")).is("Jeffries")
        end)

Here I’ve made some decisions. First, the method that does the job will be called fromTable. Second, all the keys will be strings, even if they appear as numbers in the original table.

Test should fail looking for fromTable.

39: Table as XSet -- Tests:516: attempt to call a nil value (method 'fromTable')

Now the class method:

function XSet:fromTable(tab)
    return XSet:on(XGeneric:fromTable(tab))
end

We want a generic XSet here, I think. So we’ll leave it up to XGeneric to create the instance. Now should fail asking for XGeneric to understand.

Ow. Stack overflow. XGeneric inherits from XData which inherits from XSet. I had forgotten that little trick. Remind me to think more deeply about that. Later. Anyway we can implement fromTable on XGeneric:

function XGeneric:fromTable(tab)
    local xg = XGeneric()
    for scope,element in pairs(tab) do
        xg:addAt(tostring(element),tostring(scope))
    end
    return xg
end

OK, that’s nice. Now we can make simple sets more easily.

I think we can commit this: XSet:fromTable converts tables to XSets with string element and scope.

It may be worth noting that, as written, we can’t use this function to create a set with a set as an element: the tostring will just dump whatever the print string is for an XSet:

function XSet:__tostring()
    if self:isNull() then return "NULL" end
    return "XSet("..self:card().." elements)"
end

Let’s improve the test and change the rules that we’ll convert numbers to string but everything else is left alone. That will allow something odd: it will allow a set to contain a table. We may choose to deal with that, but why not allow it.

        _:test("Table as XSet allows XSet elements", function()
            local s1 = XSet():addAt("first","Ron"):addAt("last","Jeffries")
            local s2 = Tuple{10,20,30}
            local tab = {"hello",person=s1,tens=s2}
            local set = XSet:fromTable(tab)
            _:expect(set:at("1")).is("hello")
            _:expect(set:at("person"), "person set").is(s1)
            _:expect(set:at("tens"), "tens set").is(s2)
        end)

This should fail not matching person set and tens set.

40: Table as XSet allows XSet elements person set -- Actual: XSet(2 elements), Expected: XSet(2 elements)
40: Table as XSet allows XSet elements tens set -- Actual: XData???, Expected: XData???

The second message tells me that we could do better with the print string on a tuple. Let’s make the test run.

function XGeneric:fromTable(tab)
    local stringify = function(e)
        return type(e)=="number" and tostring(e) or e
    end
    local xg = XGeneric()
    for scope,element in pairs(tab) do
        xg:addAt(stringify(element),stringify(scope))
    end
    return xg
end

That’s rather nice if I do say so myself. Test runs. Commit: fromTable only converts numbers to strings, all else is accepted as is.

Now about that print string. What happened there?

I find this:

function XSet:__tostring()
    if self:isNull() then return "NULL" end
    return "XSet("..self:card().." elements)"
end

That makes me think that the display should not have returned what it did. I want to see that error again. I’ll break the test:

            _:expect(set:at("tens"), "tens set").is(36)

That gives me this:

40: Table as XSet allows XSet elements tens set -- Actual: XData???, Expected: 36

What the heck is a Tuple anyway?

Tuple = class(XData)

Now it happens that XData inherits from XSet so apparently this works but shouldn’t we really be creating an XSet containing a Tuple?

A review of usage tells me that no, Tuple is a stand-alone thing that acts like a set. I’m not sure I’m comfortable with that from a design viewpoint, but we can just add a tostring and should be happy.

function Tuple:__tostring()
    return "Tuple 1-"..self:card()
end

Now the message is:

40: Table as XSet allows XSet elements tens set -- Actual: Tuple 1-3, Expected: 36

Fix the test:

        _:test("Table as XSet allows XSet elements", function()
            local s1 = XSet():addAt("first","Ron"):addAt("last","Jeffries")
            local s2 = Tuple{10,20,30}
            local tab = {"hello",person=s1,tens=s2}
            local set = XSet:fromTable(tab)
            _:expect(set:at("1")).is("hello")
            _:expect(set:at("person"), "person set").is(s1)
            _:expect(set:at("tens"), "tens set").is(s2)
        end)

Green. Commit: Tuple has reasonable tostring.

However

My motivation here was to clean up this method:

function XStatistics:getScopeSet()
    local groupFields = self.control.group or {}
    local scopeSet = XSet()
    for i,field in ipairs(groupFields) do
        scopeSet:addAt(field,field)
    end
    return scopeSet
end

I wonder if I can do any better here now. Let me try this:

function XStatistics:getScopeSet()
    local groupFields = XSet:fromTable(self.control.group or {})
    return groupFields:reduce(XSet(), function(r,e,s)
        return r:addAt(e,e)
    end)
end

The tests all run. We can commit this. Is it better than what we had before? Well, if one is a wizard with reduce, it probably is, and if not, probably not.

This verges on too clever to live, but it’s my darling and I’m not going to kill it. Commit: XStatistics:getScopeSet uses reduce to produce the set.

Surprise Summary

At this point in the narrative, I decided to see whether I could work out how to “view” a table as an XSet. This led me down a primrose path of setting metatables and, at this point, I don’t quite see how to get what I want. I’ll do some kind of spike offline and then come back to this idea. I promise to report on what happens, but I don’t think it’ll be productive for you to watch me fumbling along with something I don’t understand.

That aside, we have two little fixes, and we have the ability to quickly convert a table to an XSet, which we used to good effect in getScopeSet. Well, I call it good effect. If you disagree, let me know. We can discuss whether and when things like reduce are best used.

I’m still inclined to provide those functions as part of table, but if instead I can make tables act like sets, I’ll get the same capability more in tune with the current application.

I hope you’re enjoying these odd articles. If not, please let me know. I have other odd ideas.

See you next time, I hope!

XSet2.zip