Dungeon 78
Let’s try a more conventional approach to our deferred collection. First, some words on why it’s a good idea.
A curious person might ask why I’m spending time on this deferred collection idea, when there’s a perfectly good implementation in the Tile object:
function Tile:addDeferredContents(anEntity)
self.newlyAdded[anEntity] = anEntity
end
function Tile:removeContents(anEntity)
self.contents[anEntity] = nil
end
function Tile:updateContents()
for k,c in pairs(self.newlyAdded) do
self.contents[k] = c
end
self.newlyAdded = {}
end
One clue is in the fact that addDeferredContents is at line 72, while removeContents is at line 326 and updateContents is at line 335. Sure, I could group them closer together, perhaps at the cost of it being even harder to find them, but even so they would be 12 lines among 350.
Another clue is that they refer to only two member variables of Tile, contents and newlyAdded. Tile has at least 5 other member variables all having nothing to do with contents.
The contents variable has a sort of primary meaning to Tile: it’s, well, the contents of the tile. Tile does care about that. But Tile doesn’t care about how contents wants to be updated, yet we have made it responsible for managing that behavior, and doing it properly.
All these clues tell us that there is an object waiting to be born. I’ve been calling that object “deferred collection”.
Once we have a solution to the deferred collection concern, we can put it in place and simplify Tile just a bit. Will it be worth the work I’m putting in on inventing it? Perhaps not today, perhaps not at all. But I’m in this for the long term, and the problem of managing this kind of collection has bitten me more than once, quite possibly in every game I’ve written in Codea.
And, of course, unlike a “real” programmer, I get to write about it. But the learning we get from doing things like this is generally worth it, in my view. We’ll go faster and better in the future by investing a bit in ourselves today.
Therefore, we’re going to write …
An Object-Oriented Deferred Collection
Yesterday’s code was quite fun, but very deep in the bag of tricks, with tricksy things like metatables and specialized metatable tricks to get Codea to do nifty things for us. That collection works–I’m pretty darn sure–but every time I had to look at it, I’d have to figure it out again. And every time I hire a new programmer to work on this game, they’d have to come to me being all what is this deferred collection thing.
That would get tedious. So let’s see about another way of doing it: a conventional Codea class.
We’ll have to make one sacrifice. I see no practical way of letting the programmer store directly into a deferred collection like this:
dc[k] = v
They’re going to have to say something like:
dc:add(k,v)
I think that’ll be OK, because we only access Codea tables near the bottom of our code anyway, so we’re accustomed to methods like add.
Let’s get to it. I’ll do this in a separate program, with TDD, like yesterday.
Oh, and one more thing. I’ve been calling it “deferred collection” this morning, and I’ll leave the text above that way. But since in Codea we talk of tables more than collections, I’ll name the thing DeferredTable. Collection is more a Smalltalk term.
My first test will allow me to initialize the table with some contents, as I did yesterday. This has no real applicability in our present program. I might mention that I’d like to write a test for the table being empty, but that’s not possible, yet.
_:test("DT{stuff} has stuff contents", function()
local dt = DeferredTable{a="alpha", b="beta"}
_:expect(dt.get("a")).is("alpha")
end)
This will fail lacking DeferredTable:
1: DT{stuff} has stuff contents -- Tests:16: attempt to call a nil value (global 'DeferredTable')
Now some code:
DeferredTable = class()
function DeferredTable:get(k)
end
This suffices to discover the wrong answer:
1: DT{stuff} has stuff contents -- Actual: nil, Expected: alpha
Let’s do “Fake it till you make it”:
function DeferredTable:get(k)
return "alpha"
end
Test runs. Ship it. OK, maybe not. Extend the test:
_:test("DT{stuff} has stuff contents", function()
local dt = DeferredTable{a="alpha", b="beta"}
_:expect(dt.get("a")).is("alpha")
_:expect(dt.get("b")).is("beta")
end)
We certainly are surprised to see this message:
1: DT{stuff} has stuff contents -- Actual: alpha, Expected: beta
OK, not very surprised. We wrote that test to drive out the rest of our initial object:
function DeferredTable:init(aTable)
self.contents = aTable or {}
end
function DeferredTable:get(k)
return self.contents[k]
end
I’m so confident in that that I pasted it into the article before running the test. (Actually, I usually do that, because I want you to see how often my tests really do catch me.)
Well, I wish I hadn’t said that. Here’s what happened:
1: DT{stuff} has stuff contents -- Tests:31: attempt to index a nil value (field 'contents')
How could two lines of code not work? And how is contents nil? It should have to be at worst empty. Ah. The bug is in the test. Standard error #whatever: dot for colon.
_:test("DT{stuff} has stuff contents", function()
local dt = DeferredTable{a="alpha", b="beta"}
_:expect(dt:get("a")).is("alpha")
_:expect(dt:get("b")).is("beta")
end)
This better run, or I’m going home. And it does. I’m home anyway.
I’m ruefully amused. What we see here is that whe we have tests, they find our mistakes … even the ones in the tests. Anyway, let’s extend this test to check for deferral.
_:test("DT{stuff} has stuff contents", function()
local dt = DeferredTable{a="alpha", b="beta"}
_:expect(dt:get("a")).is("alpha")
_:expect(dt:get("b")).is("beta")
dt:put("c","charlie")
_:expect(dt:get("c")).is(nil)
end)
This will fail lacking put.
1: DT{stuff} has stuff contents -- Tests:19: attempt to call a nil
value (method 'put')
We’ll give a null put – which will make this test pass!
function DeferredTable:put(k,v)
end
1: DT{stuff} has stuff contents -- OK
But now to update and check:
_:test("DT{stuff} has stuff contents", function()
local dt = DeferredTable{a="alpha", b="beta"}
_:expect(dt:get("a")).is("alpha")
_:expect(dt:get("b")).is("beta")
dt:put("c","charlie")
_:expect(dt:get("c")).is(nil)
dt:update()
_:expect(dt:get("c")).is("charlie")
end)
I expect lack of update.
1: DT{stuff} has stuff contents -- Tests:21: attempt to call a nil value (method 'update')
I’ll go ahead and make this work now.
function DeferredTable:init(aTable)
self.buffer = {}
self.contents = aTable or {}
end
function DeferredTable:put(k,v)
self.buffer[k] = v
end
function DeferredTable:update()
for k,v in pairs(self.buffer) do
self.contents[k] = v
end
self.buffer = {}
end
The test runs. This is nearly good. Let’s test remove, since we need it.
_:test("DT{stuff} has stuff contents", function()
local dt = DeferredTable{a="alpha", b="beta"}
_:expect(dt:get("a")).is("alpha")
_:expect(dt:get("b")).is("beta")
dt:put("c","charlie")
_:expect(dt:get("c")).is(nil)
dt:update()
_:expect(dt:get("c")).is("charlie")
dt:remove("a")
_:expect(dt:get("a")).is(nil)
end)
This will fail looking for remove:
1: DT{stuff} has stuff contents -- Tests:23: attempt to call a nil value (method 'remove')
And build it:
function DeferredTable:remove(k)
self.contents[k] = nil
end
Test runs. However, somehow I thought of a bug, and I think it’s a bug in yesterday’s version as well. Let’s do a special test for it to make it more clear.
_:test("Remove after add works", function()
local dt = DeferredTable()
dt:put("a", "alpha")
_:expect(dt:get("a")).is(nil)
dt:remove("a")
dt:update()
_:expect(dt:get("a")).is(nil) -- !! still nil
end)
Here we build a DT. We add something to it but do not update. Then we call for the thing we added to be removed. Then we update … and right now, it’ll come back:
2: Remove after add works -- Actual: alpha, Expected: nil
We have to remove from the buffer as well, because the most recent value could be in there!
function DeferredTable:remove(k)
self.buffer[k] = nil
self.contents[k] = nil
end
The test should run now.
2: Remove after add works -- OK
And it does. I’m sure that defect is in the tricksy version. I’ll have to remember to go fix it if I decide not to scrap it.
I noticed in coding this that I wanted to type
dt:add(k,v)
I’ll add that as a courtesy method in addition to put.
I chose not to test drive that but I’ll go change that last test to say add, so we have a check for the case.
_:test("Remove after add works", function()
local dt = DeferredTable()
dt:add("a", "alpha")
_:expect(dt:get("a")).is(nil)
dt:remove("a")
dt:update()
_:expect(dt:get("a")).is(nil) -- !! still nil
end)
Still runs, woot.
We could commit, but we need one more bit of capability, and it’s critical to our purpose here. We have to be able to loop over the collection.
And I’ve decided that when we initiate a loop over the collection, it will update before the loop starts. This will cover all the cases I have, because I don’t loop over the same collection twice at the same time. Probably. :)
So a test. My first plan is to have a method pairs that acts like the built-in pairs function:
_:test("Loop over DT", function()
local dt = DeferredTable{a=1,b=2}
local sum = 0
for k,v in dt:pairs() do
sum = sum + v
end
_:expect(sum).is(3)
end)
This will fail wanting pairs:
3: Loop over DT -- Tests:39: attempt to call a nil value (method 'pairs')
Driving:
function DeferredTable:pairs()
return pairs(self.contents)
end
I expect the test to pass.
3: Loop over DT -- OK
Now to extend the test with an added item which should be updated automatically before the loop runs:
_:test("Loop over DT", function()
local dt = DeferredTable{a=1,b=2}
local sum = 0
for k,v in dt:pairs() do
sum = sum + v
end
_:expect(sum).is(3)
dt:put("c",3)
for k,v in dt:pairs() do
sum = sum + v
end
_:expect(sum).is(6)
end)
Should fail getting 3 not 6.
Well, it would have if I had zeroed sum again:
_:test("Loop over DT", function()
local dt = DeferredTable{a=1,b=2}
local sum = 0
for k,v in dt:pairs() do
sum = sum + v
end
_:expect(sum).is(3)
dt:put("c",3)
sum = 0
for k,v in dt:pairs() do
sum = sum + v
end
_:expect(sum).is(6)
end)
3: Loop over DT -- Actual: 3, Expected: 6
That scared me, I can tell ya.
Now to make it work:
function DeferredTable:pairs()
self:update()
return pairs(self.contents)
end
Test runs.
Let’s talk about that magical behavior there. Is that a good idea, or a really bad one?
I argue that it is a good one. We want the programmer to make a single decision: this table needs to be deferred, because it may have things added to it while a loop is running. We’d prefer them not to have to think about when to update it. Presently we update the deferred contents in Tile … where? I don’t remember … maybe it’s in draw. Yes, probably draw. I’d look but I’m not editing Dung just now.
Point made, I think. The programmer can do an explicit update if she wants to but things will work in a sensible way even if she doesn’t. There might come a need for an un-updated loop, but we’ll put that in if and when we’re convinced we need it.
This is nearly done. There’s just one more thing I would like to try. I want to try to make the regular pairs function work on our DeferredTable.
Let me just change the loop test:
_:test("Loop over DT", function()
local dt = DeferredTable{a=1,b=2}
local sum = 0
for k,v in dt:pairs() do
sum = sum + v
end
_:expect(sum).is(3)
dt:put("c",3)
sum = 0
for k,v in pairs(dt) do -- built-in pairs
sum = sum + v
end
_:expect(sum).is(6)
end)
As written this will do something odd, certainly not pass.
3: Loop over DT -- Tests:46: attempt to perform arithmetic on a table value (local 'v')
The issue is that now we’re iterating over the instance’s table, fetching out the names of the member variables and methods and trying to add them up. Unlikely to work.
But there’s this metamethod named __pairs. If it works as I suspect, we can do this:
function DeferredTable:__pairs()
return self:pairs()
end
And in fact:
3: Loop over DT -- OK
So that is sweet. Now we can treat a DT almost like a regular table, except that we have to use put and get, which isn’t too much of a burden.
Let’s have a brief retro, comparing yesterday’s solution with today’s.
Retro
Yesterday’s implementation is quite tricksy. It reaches deep into the Codea bag of tricks, using some very obscure capabilities. (And, as written, it has that odd behavior that a delete doesn’t delete a value waiting for update. I’m not even sure how to fix that: we don’t get an event on access to an existing element.)
Even if it worked, it’s quite intricate, and too cool for school. I’m glad I did it, I’m glad to have a better appreciation of how those meta functions work, and a better appreciation of how easy it is to get them wrong.
In contrast, our DeferredTable class is pretty straightforward code. Its only cuteness is in the implementation of __pairs, and that’s a two-liner, so it should be readily understood by our newbie programmer when they show up.
I’m of a mind to adopt this class into Dungeon. The cost will be yet another tab or two. (Right now, I have the code in with the tests. I could just bring in the DT class. I could even reference it in, but I don’t often use that capability other than for referencing in CodeaUnit.
So OK, yes, let’s make a new tab in Dung, and then see about using the thing.
Adding DT to Dung
Simple enough to add a tab, and paste it in. I added this at the top:
-- DeferredTable
-- RJ 20210128
-- See D2ferred project for tests
Now let’s use it in Tile. This will be just a tad of retrofitting.
function Tile:init(x,y,kind, runner)
self.position = vec2(x,y)
self.kind = kind
self.runner = runner
self.contents = {}
self.newlyAdded = {}
self.seen = false
self:clearTint()
self.tile = nil
end
Here we’ll create our DT and remove the newlyAdded thingie:
function Tile:init(x,y,kind, runner)
self.position = vec2(x,y)
self.kind = kind
self.runner = runner
self.contents = DeferredTable()
self.seen = false
self:clearTint()
self.tile = nil
end
Now we can search for newlyAdded and change those:
function Tile:addDeferredContents(anEntity)
self.newlyAdded[anEntity] = anEntity
end
Ah. A problem. I forgot to test for a case. Let’s go back to d2 and deal with it.
Self-keyed objects
_:test("Self-keyed objects", function()
local dt = DeferredTable()
local tab = {}
dt:put(tab)
dt:update()
_:expect(dt).has(tab)
end)
We use what I’m here calling “self-keyed objects, objects that can serve as their own key in the hash table. This is my most common use of tables in the game, and I forgot even to test for it. This test will fail not finding tab:
4: Self-keyed objects -- Actual: table: 0x283974240, Expected: table: 0x2839767c0
That message surprises me a bit, but CodeaUnit has is weird, and I have no real right to expect has to work on these tables. Let’s change the test:
_:test("Self-keyed objects", function()
local dt = DeferredTable()
local tab = {}
dt:put(tab)
dt:update()
_:expect(dt:get(tab)).is(tab)
end)
4: Self-keyed objects -- Actual: nil, Expected: table: 0x283929580
That’s more like it. The fix is to change this:
function DeferredTable:put(k,v)
self.buffer[k] = v
end
To this:
function DeferredTable:put(k,v)
self.buffer[k] = v or k
end
If v is nil, use k. Test should run.
4: Self-keyed objects -- OK
Perfect. Move this to Dung and get back to work.
Now we can change this:
function Tile:addDeferredContents(anEntity)
self.newlyAdded[anEntity] = anEntity
end
To this:
function Tile:addDeferredContents(anEntity)
self.contents:put(anEntity)
end
We could rename that method to addContents also, but we’re in the middle of refactoring. Then …
function Tile:updateContents()
for k,c in pairs(self.newlyAdded) do
self.contents[k] = c
end
self.newlyAdded = {}
end
That becomes …
function Tile:updateContents()
self.contents:update()
end
However, we can probably do without that method entirely. We’ll see.
There are no more references to newlyAdded. Game should work.
Hm something has gone horribly wrong. Things are attacking the princess and shouldn’t be. I think I know what is wrong: we didn’t change the remove, because it didn’t refer to newlyAdded.
function Tile:removeContents(anEntity)
self.contents[anEntity] = nil
end
That becomes …
function Tile:removeContents(anEntity)
self.contents:remove(anEntity)
end
I think this’ll work, but I’d better look for other references to contents. Yes, that seems to be all there is. Nice find. Wonder how I could have noticed that other than by being smarter. That ship has sailed.
I notice something odd. When I run over a speed thing, the princess’s speed bar doesn’t increase. What is up with that?
Let’s get this other thing committed: Using DeferredTable in Tile contents.
Now what is up with the Speed power-up?
The Loot does this:
function Loot:actionWith(aPlayer)
self.tile:removeContents(self)
aPlayer:addPoints(self.kind, math.random(self.min, self.max))
end
The Player does this:
function Player:addPoints(kind, amount)
local attr = self:pointsTable(kind)
if attr then
local current = self[attr]
self[attr] = math.min(20,current + amount)
self:doCrawl(kind, amount)
end
end
And …
function Player:pointsTable(kind)
local t = {Strength="strengthPoints", Health="healthPoints", Speed="speedPoints"}
return t[kind]
end
So far so good. Are we not using it in the AttributeSheet thing?
...
self:drawText(m:name())
self:newLine(2)
self:drawText("Health")
self:drawBarFraction(self.healthIcon, m:health(), 20)
self:newLine()
self:drawText("Speed")
self:drawBarFraction(self.speedIcon, m:speed(), 20)
self:newLine()
self:drawText("Strength")
self:drawBarFraction(self.strengthIcon, m:strength(), 20)
self:drawPhoto(m:photo())
...
And:
function Player:speed()
return 5
end
I think I see the problem.
function Player:speed()
return self.speedPoints
end
This should also help in combat. I should tend to be faster more often.
Let’s try it. Seems better, but let’s check the encounter code.
function attack(attacker, defender, random)
yield(attacker:name().." attacks ".. defender:name().."!")
local attackerSpeed = rollRandom(attacker:speed(), random)
local defenderSpeed = rollRandom(defender:speed(), random)
if attackerSpeed >= defenderSpeed then
yield(attacker:name().." is faster!")
firstAttack(attacker,defender, random)
else
yield(defender:name().." is faster!")
firstAttack(defender,attacker, random)
end
end
function firstAttack(attacker,defender, random)
yield(attacker:name().." strikes!")
local attackerSpeed = rollRandom(attacker:speed(), random)
local defenderSpeed = rollRandom(defender:speed(), random)
if defenderSpeed > attackerSpeed then
attackMisses(attacker,defender, random)
if math.random() > 0.5 then
yield("Riposte!!")
firstAttack(defender,attacker, random)
end
else
attackStrikes(attacker,defender, random)
end
end
function rollRandom(aNumber, random)
return (random or math.random)(0,aNumber)
end
So we roll between 0 and our speed, and there are two speed rolls, one for first attack and then one to decide whether the defender manages to get out of the way.
If the speeds are equal, I’d naively expect to go first half the time and to miss half of the swings I make, so 3/4 of the time nothing good happens.
With speed twice that of the defender, I’d win the first roll 2/3 of the time and the second 2/3 of the time, for a total of 4/9, still less than half the time resulting in a hit.
Arguably the initial attacker has surprise, so we should have a more favorable roll for the chance of hitting, but for now, what I observe in the game matches a quick analysis.
I think we’re good. Commit: strength powerups now work.
One more thing. Let’s remove the update of our deferred table from Tile, since looping over it updates and we never search it in any other form.
Extract this:
function Tile:updateContents()
self.contents:update()
end
And the reference to it, in draw. Game should still work.
Commit: rely on auto-upate in Tile contents DeferredTable
I think that’s a wrap for the day. Let’s sum up.
Summary
We TDD’d a more conventional, object-oriented version of the deferred table idea. It turned out to be pretty straightforward, save only that it uses __pairs, which is a tiny bit down in the bag of tricks. Along the way, I realized that there was a defect in the thing, namely that if you set something up for addition and then try to remove it before updating, the thing will appear. That defect occurs in the fancy one as well and I don’t see a good way to fix it. We might even have to buffer deletes, which would be kind of nasty.
In any case, to my taste this OO version is so much more like our usual kind of code that I much prefer it, and I’ve chosen to import it into Dung and use it.
Along the way, because the new DeferredTable automatically updates before being iterated, I was able to remove a fairly obscure bit of code from Tile.
Our work has been a net increase in code, but a decrease of size and complexity in Tile, a complex class, provided by the addition of a handful of methods nicely encapsulated in a new class. We may even have use for it in the future.
Was it worth it? Well, as a learning exercise, I think it was quite valuable. I learned some things, and I hope you did as well. In terms of the investment of time in our product, possibly not. The design is better, so true technical debt has been reduced, but the impact will be small.
Arguably, building something like this should be done when the product needs it. In this case, I think the project needed it, as I had just stumbled over a problem with table updating.
As always, you get to decide how you’ll work. I’m just here to show you what I do, and tell you as best I can, what I’m thinking about.
Now I’m thinking about a banana.
See you next time!