Hello, loves!

I have two items in my queue and I don’t think I can address them in parallel. What shall I do? Contains brief LLM-AI slur. May contain footnotes.

I enjoy programming in the classic form, where you do it with your own brain and fingers, not using some assemblage of oligarchy-serving book-burning job-stealing planet-destroying computer programs that can’t even enjoy what they’re doing.

In those days we kept an opcode cue card in our pocket protector. It was the style at the time¹. Kids these days don’t get it.

And I enjoy writing about it as I do it, which has never been the style as far as I know. The style, if you were going to write at all, was more to work on the code until it was as perfect as it could be, and then write it up as if you were actually so smart that this perfect code just sprang all of a piece from your forehead, like Athena fully formed and fully armed springing from the skull of Zeus. Fortunately, such authors didn’t bother to split open their skull, which may have been the origin of the term “splitting headache”, but I digress.

I like to write as I go, sharing as much of my thought process as I can get down on in words. I do this mostly because it has become a habit, but I tell myself that I do it in case some reader somewhere might someday get a glimmer of an idea from what I think / say / do.

But there is a programming problem, an interesting one, that I want to address. But I can’t write about it as Ron Jeffries, for reasons². So working the new problem will interfere with my standard morning of sitting down with some random on-going program, currently the dungeon, and coding something up and writing about it, sharing it with the six people who are known to read these articles, g*d knows why³.

I’m not sure how to balance the dungeon and the other problem, how to manage the writing bit, even how to switch mental context between two very different problems, two different languages, and three different editors⁴.

I don’t think switching between AM and PM will do it for me, though I’ll try that first. Maybe a couple of days on one and a couple on the other? That should really foul up the keystroke memory and whether to type a dot or a colon⁵.

I’ll figure out something. My mornings entertain me, and I hope they somewhat entertain a few other folks.

Oh, and if in fact you do read these and you do enjoy them, please drop me an email or a toot. If the count drops to zero I might stop. Might⁶.

Hello, loves!

I propose to blather a bit this morning, starting with some ‘AI’-related ideas and then going I know not where.

Triggers

Jessica Kerr—jessitron—recently published an article that caught my attention: Who are we Now. Jessica is unquestionably a better programmer than I am, thinks very deeply about important matters, and has been digging deeply into the use of current ‘AI’ agents. You should read that article, before you read this and perhaps instead. Here is a quote that I want to call out:

Do I think AI is bad for us? Sure.
Does that mean I shouldn’t use it? No.

Another of my betters, Kent Beck, has also been working with the ‘AI’ systems and in a recent article closed with something that struck me:

The overall outcome for me is (finally) positive:
    Less useless code
   More confidence in the behavior of what remains

Kent has coined the term “genie” in referring to whatever ‘AI’ thing he’s using. I am not comfortable with that. If one has read enough about genies, it might be OK, because the genie in literature is capricious and when it grants you a wish, that wish almost always bites you where you’d prefer not to be bitten. I think a better term would have been “demon”, but I’m neither his boss nor his editor. I’m just some guy, y’know?

More Good than Harm

We live in a world that is incredibly complex, and that owes us nothing. Nature just is, and from the viewpoint of some random human, Jess or Kent or Ron, there is very little that we can really control. To some degree, we can control what we do, and sometimes we can influence what others do but generally not control them, and we might possibly have some influence on the overall system that makes up the world.

While someone might choose to sacrifice themselves, in some way, for something they value highly, I don’t think it’s my place to say that that person over there should sacrifice themselves for something that I care about. We all get to make those decisions for ourselves.

And it’s important to note that we can’t do much good for anyone if we’re dead, or if we live in a cave and do not engage with the world. And when we do engage with the world, we must inevitably do some harm. We eat something that would otherwise have been alive, we step on an ant that would otherwise have brought a delicious crumb home to the nest, we accidentally start a turf war between the Jets and the Sharks. You know how it goes: once you’re a Jet you’re a Jet all the way.

Thoughtful people, “good” people, I would say, try to do as little harm as possible, and try to do more good than harm, in some kind of balancing sense. And that has to start at home: we cannot do any good at all if we harm ourselves too much.

‘AI’ Harms

The current ‘AI’ thing does harms at many levels. Consumes vast resources, pumps money to the oligarchy, destroys rare books, costs ordinary people their jobs, and so on. In my personal view, the ‘AI’ subsystem is a net negative for humanity and for the world. Might it someday turn positive? Almost anything is possible but if it were to happen it would require a major change in the system of the world.

I think that Jessica and Kent think we’re stuck with it. And, as it happens, I, too, think we are stuck with it. We cannot put the genie demon back in the bottle box.

‘AI’ Responses

Whatever our overall thinking about ‘AI’, we have to choose how we, personally, will respond, how we will act within the world system as it is, containing among other evils, this ‘AI’ thing. There are innumerably many ways to respond:

Fight the System

There are those among us with the courage and energy to fight the system, to try to change it into a better system. Within that notion there are many ways to fight, ranging from writing letters to the editor to running for office to burning down the courthouse. Some of those actions work better than others, and it often seems that none of them work very well.

Withdraw

We can withdraw as much as possible from the system, ranging from stopping using Amazon to moving to a subsistence farm with no gas and electric, just a couple of pigs and a flock of raggedy chickens.

Live Our Best Life

I think everyone is really doing this one, making choices that seem to them to result in the best life for themselves, and for those they care about.

My best life, relative to ‘AI’, is to try to use it not at all, or failing that, as little as possible. I can make that choice because I am retired and do not need a job where I’d have to be adept with ‘AI’. The decision does cost me something in terms of what I want, which is somewhat like what I believe Jessica and Kent want.

Jessica and Kent, and many others I know, have made the decision to learn about the ‘AI’ systems, and to learn about them rather deeply, and to share what they have learned. I am absolutely confident that they do that to help others to live well in a world with ‘AI’ in it: helping others is on the mind of most of us who write articles, give talks, and so on. We enjoy being useful. (We may also enjoy the “fame”, and, yes, there is money in it, but with few exceptions, the people I know in my community are not just in it for the bread. They want to help people: they get off on helping people.)

Their best life includes learning about ‘AI’, and sharing what they know. I think that’s a very valid choice.

Other folx, some of whom I know and plenty whom I do not, are using ‘AI’ because their current job requires it, or because they see the handwriting on the wall and know that they’re going to need to be adept with it very soon. They’re doing it to stay employed, to feed themselves and their families, and to be able to do whatever other good they do in the world. Or, I suppose, evil, if any of them are evil henchmen or henchwomen. Henchpersons? Henchones? Never mind.

My own choice, not to participate in ‘AI’, means that I am giving up the possibility of helping people use it well. That is a choice that I make consciously and in the strong belief that it is right for me.

The point is, we all choose how to live our best life, in the midst of the whole glob that is the world, including the ‘AI’ sub-glob, and that is as it should be.

Acknowledge Both Sides

I pulled out those quotes up above. Those are the first time I’ve noticed either of those individuals acknowledging the darker side of ‘AI’ use, which includes not working very well and not being much fun. I’m not saying they haven’t expressed that before, only that this is the first time I’ve noticed.

I do wish that they, and everyone using ‘AI’ would more often acknowledge that side of things, because while we can almost certainly not put the demon back in the box, we might be able to mitigate some of its drawbacks if we are able to keep them in mind and work on them. We might be able to make it less harmful.

Live Your Best Life

I hope that you will pay continued attention to living your best life, minimizing harm and maximizing benefit to the world, as you see fit. Proceed thoughtfully!

Hello, loves!

While you were away, I did a little refactoring. Let’s see what it shows us. I think we’ve done some good here.

I’m still working on paths between sections, and still doing most of the work in the tests. Mostly. Yesterday’s afternoon session, unpublished, made a change to the Passages object.

Here’s what our active test looked like when last we met:

    layout.clear_subregions()
    path_cells = suite_1.find_extended_path(suite_2)
    # now path_cells[0] is in one suite, and
    # path_cells[-1] in the other, and
    # path_cells[1:-1] are the cells we need for
    # the new path room.

    room_2 = path_cells[0].room
    assert room_2 in suite_2.room_set
    room_1 = path_cells[-1].room
    assert room_1 in suite_1.room_set
    path_room_1_2 = Room(path_cells[1:-1])
    layout.add_room(path_room_1_2)
    layout.make_passages()
    assert path_cells[0].has_passage(path_cells[1])
    assert path_cells[-1].has_passage(path_cells[-2])
    yield

And now …

    layout.clear_subregions()
    path_cells = suite_1.find_extended_path(suite_2)
    # now path_cells[0] is in one suite, and
    # path_cells[-1] in the other, and
    # path_cells[1:-1] are the cells we need for
    # the new path room.

    room_2 = path_cells[0].room
    assert room_2 in suite_2.room_set
    room_1 = path_cells[-1].room
    assert room_1 in suite_1.room_set
    path_room_1_2 = Room(path_cells[1:-1])
    layout.add_room(path_room_1_2)
    # layout.make_passages()
    layout.passages.add_passage_unconditionally((path_cells[0], path_cells[1]))
    layout.passages.add_passage_unconditionally((path_cells[-1], path_cells[-2]))
    layout.passages.isolate(path_room_1_2)
    assert path_cells[0].has_passage(path_cells[1])
    assert path_cells[-1].has_passage(path_cells[-2])
    yield

The difference is there near the end. Instead of using make_passages, we use the extended path—the path between rooms that includes the starting and ending cells, in the rooms we’re connecting. We know we want the first two cells in that path to be a passage to the one room, and the last two to be a passage to the other room. So we add those two paths directly. The unconditionally method already existed.

THen we call layout.passages.isolate, which is a new method that makes the final path impermeable. That required some new code:

class Passages:
    def __init__(self):
        self.cell_connections: set[tuple[Cell, Cell]] = set()
        self.room_connections: set[tuple[Room, Room]] = set()
        self.isolated: set[Room] = set()
    def isolate(self, room: Room):
        self.isolated.add(room)

And we changed this:

    def create_passage(self, r1: Room, r2: Room, func):
        if r1 in self.isolated or r2 in self.isolated:
            return False
        elif self.rooms_are_connected(r1, r2):
            return False
        else:
            passage = func()
            self.add_passage_unconditionally(passage)
            return True

It’s just the first two lines that were added. If either of the two rooms to be connected is isolated, we return False without creating a passage. If not, then as before we don’t connect them if they are already connected, and if they aren’t connected, we call the provided function and connect them.

Your point, sir?

The point of writing the code yesterday, if I recall, was that calling make_passages is kind of a big deal, with a lot of magic going on, and in the case in hand, we know more precisely what needs to be done, so basically I wanted to work out just what we needed and do that.

So the operations we need to connect two sections are now in the test and, cleaned up, they go like this:

    layout.clear_subregions()
    path_cells = suite_1.find_extended_path(suite_2)
    path_room_1_2 = Room(path_cells[1:-1])
    layout.add_room(path_room_1_2)
    layout.passages.add_passage_unconditionally((path_cells[0], path_cells[1]))
    layout.passages.add_passage_unconditionally((path_cells[-1], path_cells[-2]))
    layout.passages.isolate(path_room_1_2)

That sure looks like code that belongs in DungeonLayout, doesn’t it?

I’d like to move it over there, and I’d like not to lose the value of this test, which is looking at some details that will be hidden in the method we extract. We can’t be sure that the code will find the same path every time. If there is more than one pair of “closest” cells, the starting and ending points of the path are random. Maybe if we seed the random number generator?

I think we’ll replicate this test except for calling our new method and see what we can see. And we will seed the RNG and use the same seed in both tests. Here’s the steps method in the new test:

        def steps():
            random.seed(14159)
            layout.subregion_LRBT(0, 20, 0, 20)
            new_rooms = make_diamond_section()
            suite_1 = Suite(layout, new_rooms)
            layout.subregion_LRBT(0, 20, 35, 55)
            new_rooms = make_diamond_section()
            suite_2 = Suite(layout, new_rooms)
            layout.clear_subregions()
            path_cells = suite_1.find_extended_path(suite_2)
            layout.add_isolated_path(suite_1, suite_2)
            assert path_cells[0].has_passage(path_cells[1])
            assert path_cells[-1].has_passage(path_cells[-2])
            # note: test relies on our path_cells being
            # same as the path found in `add_isolated_path`
            # currently Suite.find_closest_pair is
            # deterministic, so this works.
            yield

To allow for some checking, we privately call find_extended_path, which will return the same path that the add_isolated_path method uses, so we can check the ends to find anything we want to know.

The new method is as one would expect:

class DungeonLayout:
    def add_isolated_path(self, s1: Suite, s2: Suite):
        x_path = s1.find_extended_path(s2)
        path_room = Room(x_path[1:-1], 'isolated path')
        self.add_room(path_room)
        self.passages.add_passage_unconditionally((x_path[0], x_path[1]))
        self.passages.add_passage_unconditionally((x_path[-1], x_path[-2]))
        self.passages.isolate(path_room)

Let’s commit this: layout can create isolated path between suites.

Now I want to try this new scheme in an actual dungeon build. That’s going to be a bit messy, and it’ll probably give us some ideas for improvement. I’ll just edit the existing dungeon build.

Somewhat tedious but it goes like this:

def yield_table_steps(layout, dungeon):
    bottom_left(layout)
    make_a_diamond_room(layout)
    make_a_diamond_room(layout)
    make_a_cave_room(layout)
    make_a_diamond_room(layout)
    make_a_diamond_room(layout)
    finish(layout)
    suite_bl = Suite(layout, set(layout.rooms))
    yield
    top_left(layout)
    previous_rooms = set(layout.rooms)
    make_a_round_room(layout)
    make_a_round_room(layout)
    make_a_round_room(layout)
    make_a_round_room(layout)
    make_a_round_room(layout)
    finish(layout)
    suite_tl = Suite(layout, set(layout.rooms) - previous_rooms)
    yield
    clear_subregions(layout)
    layout.add_isolated_path(suite_bl, suite_tl)
    yield
    top_right(layout)
    previous_rooms = set(layout.rooms)
    make_a_cave_room(layout)
    make_a_diamond_room(layout)
    make_a_cave_room(layout)
    make_a_cave_room(layout)
    make_a_round_room(layout)
    finish(layout)
    suite_tr = Suite(layout, set(layout.rooms) - previous_rooms)
    yield
    clear_subregions(layout)
    layout.add_isolated_path(suite_tl, suite_tr)
    yield
    bottom_right(layout)
    previous_rooms = set(layout.rooms)
    make_a_cave_room(layout)
    make_a_diamond_room(layout)
    make_a_cave_room(layout)
    make_a_diamond_room(layout)
    make_a_cave_room(layout)
    finish(layout)
    suite_br = Suite(layout, set(layout.rooms) - previous_rooms)
    yield
    right_side(layout)
    layout.add_isolated_path(suite_tr, suite_br)
    yield
    clear_subregions(layout)
    previous_rooms = set(layout.rooms)
    make_diamond_in_round_room(layout)
    bottom_right_plus_center(layout)
    finish(layout)
    suite_boss = Suite(layout, set(layout.rooms) - previous_rooms)
    yield
    clear_subregions(layout)
    layout.add_isolated_path(suite_br, suite_boss)
    populate(layout, dungeon)
    yield
    add_content(layout, dungeon)
    yield

There’s a pattern there, where we save the previous rooms, create our new suite from all the rooms, and we’ll look at that next time to see how to make the job easier, but as written we get a map that looks good to me:

Between that and our tests, I think we’re happy enough. Commit the main. Let’s sum up.

Summary

We’ve seen a nice flow of events here. We started with some simple tests, built up to a pretty complex one, then expanded the internals of the test to see that a new method on DungeonLayout could do the job, then created a new test calling that method, wrote the method, bada bing bada boom.

Things went as smoothly as they ever do, and the result, the add_isolated_path facility seems to be just what we need.

The pattern in the main program, where we save all the existing rooms, build some stuff, make a suite of it, and use that suite later, seems worthy of packaging somewhere, probably as a function to use in building.

I suspect that we should begin to abstract building a bit, since (if this were a real game) there would be many dungeon layouts to be done and there should be some way of organizing them. Since we’re here to do things that interest me, we’ll do that if it seems interesting.

So. A productive afternoon and morning. Now to make a nice iced chai and find something to read. See you soon!

Hello, loves!

If things go as I expect, we’ll start working on the paths between sections of the dungeon. It could happen. It begins to happen.

My rough plan for the inter-section paths is to build suites for each end of the path, and then so call a layout method to draw the path. That method may not currently exist, but we know that the code to do the job is in there somewhere. Then, we’ll figure out a way to mark the path “impermeable”, that is, arrange things so that if a room tries to connect with it, it’ll be rejected. Frequent readers may recall that we have a structure that prevents rooms connecting more than once. I think we’ll see about extending that capability to prevent connecting even once.

For all these things, we’ll find out when we get there. And I plan to continue to work with tests to get this done. It always goes better and it went particularly nicely yesterday, so why not today?

Here’s our latest test on this topic:

    def test_two_sections_to_suites(self):
        def diamond(size):
            if origin := layout.unused_cell_or_none():
                room = RoomMaker(layout).diamond(number_of_cells=size, origin=origin)
                layout.add_room(room)
        def make_diamond_section() -> set[Any]:
            previous_rooms = set(layout.rooms)
            diamond(25)
            diamond(25)
            layout.finish_map()
            all_rooms = set(layout.rooms)
            new_rooms = all_rooms - previous_rooms
            assert len(new_rooms) == 2 or len(new_rooms) == 3
            return new_rooms
        def steps():
            layout.subregion_LRBT(0, 20, 0, 20)
            new_rooms = make_diamond_section()
            suite_1 = Suite(layout, new_rooms)
            layout.subregion_LRBT(0, 20, 35, 55)
            new_rooms = make_diamond_section()
            suite_2 = Suite(layout, new_rooms)
            yield
        layout = DungeonLayout(55, 55)
        stepper = YieldStepper(steps(), layout)
        stepper.run_to_completion()

A fanatic would have me write a new test to work on the path idea, but the test would look like this one and we’d have to move the helper functions around. We’re only going to work with steps for a while, so let’s sketch this in:

        def steps():
            layout.subregion_LRBT(0, 20, 0, 20)
            new_rooms = make_diamond_section()
            suite_1 = Suite(layout, new_rooms)
            layout.subregion_LRBT(0, 20, 35, 55)
            new_rooms = make_diamond_section()
            suite_2 = Suite(layout, new_rooms)
            path_room = layout.make_path(suite_1, suite_2)
            assert path_room is not None
            yield

There is no make_path, and I didn’t know what to assert about it. That’s OK, we’re just sketching. Now let’s see how paths are made. I think things start with finish_map in the layout:

    def finish_map(self):
        self.ensure_connected()
        self.make_passages()
    def ensure_connected(self):
        count = 0
        while not self.is_fully_connected() and count < 10:
            count += 1
            self._make_path_rooms()
    def _make_path_rooms(self):
        suites = self.define_suites()
        # self.check_intersection(suites)  # debug method
        # for s1, s2 in zip(suites, suites[1:] + suites[:1]):
        for s1, s2 in zip(suites, suites[1:]):
            cells = s1.find_path_cells(s2)
            if cells:
                room = Room(cells, 'path')
                self.add_room(room)
            # cells can be empty if no path is possible
            # because we only accept source room,
            # target room, or available cells.

I’m not surprised to find those last four lines, which do pretty much exactly what we need. Extract the method and return the room, I think. I try this:

    def _make_path_rooms(self):
        suites = self.define_suites()
        # self.check_intersection(suites)  # debug method
        # for s1, s2 in zip(suites, suites[1:] + suites[:1]):
        for s1, s2 in zip(suites, suites[1:]):
            self.make_path(s1, s2)
            # cells can be empty if no path is possible
            # because we only accept source room,
            # target room, or available cells.

    def make_path(self, s1: Suite, s2: Suite):
        cells = s1.find_path_cells(s2)
        if cells:
            room = Room(cells, 'path')
            self.add_room(room)
            return room
        else:
            return None

I should have moved those comment lines down. Do it now. A test is failing. I suppose it must be the current one. Ah. We need to clear the subregions before we look for the path. Subregions prohibit finding cells outside themselves.

        def steps():
            ...
            layout.clear_subregions()
            path_room = layout.make_path(suite_1, suite_2)
            assert path_room is not None
            yield

Test passes. Life is good. Commit: working on inter-section paths.

Well, life is semi-good. We need now to create the passages between this new room and the rooms in the sections to which it connects … and we do not know that. We may have made a misstep.

I think we’d have done well to look at find_path_cells in Suite. Let’s do so now:

class Suite:
    def find_path_cells(self, suite: Suite) -> list[Cell]:
        source, target = self.find_closest_pair(suite)
        rooms = [source.room, target.room]
        cells = self.layout.find_path(source, target, rooms)
        on_path = cells[1:-1]
        return on_path

Ah, yes. In this method, we find the closest two cells that might connect the suites. Then the find_path will return a path that includes those two cells. Those two cells are exactly the ones that our code wants to know about. I think we’re prefer to work with this information. We’ll change the test to call a new method in Suite, which we’ll extract first:

    def find_path_cells(self, suite: Suite) -> list[Cell]:
        cells = self.find_extended_path(suite)
        on_path = cells[1:-1]
        return on_path
    def find_extended_path(self, suite: Suite) -> Any:
        source, target = self.find_closest_pair(suite)
        rooms = [source.room, target.room]
        cells = self.layout.find_path(source, target, rooms)
        return cells

Now we’ll code inside our test for a while, which will make a bit of a mess, but enable us to see what we need and figure out where it goes.

    layout.clear_subregions()
    path_cells = suite_1.find_extended_path(suite_2)
    assert path_cells
    # now path_cells[0] is in one suite, and
    # path_cells[-1] in the other, and
    # path_cells[1:-1] are the cells we need for
    # the new path room.
    yield

The test is green. I wrote some notes as comments, since I needed to say those things here and want to remember them there. Excuse me a moment while I go over to Layout and remove the extra stuff from that method we extracted. I’ll leave it extracted, that was a righteous move in any case.

    def _make_path_rooms(self):
        suites = self.define_suites()
        # self.check_intersection(suites)  # debug method
        # for s1, s2 in zip(suites, suites[1:] + suites[:1]):
        for s1, s2 in zip(suites, suites[1:]):
            self.make_path(s1, s2)
    def make_path(self, s1: Suite, s2: Suite):
        cells = s1.find_path_cells(s2)
        if cells:
            room = Room(cells, 'path')
            self.add_room(room)
            # cells can be empty if no path is possible
            # because we only accept source room,
            # target room, or available cells.

Commit. We can always commit with green tests, and as we are kind of feeling our way, I’d prefer to commit often, as we inch forward, sort of like a ratchet jack stepping up one notch at a time.

Let’s Reflect

As things stand right now, we have a list of cells in our dungeon-building steps, not as high level an object as we might want to have. But it is an interesting one. It’s a list of cells. Its end cells are in rooms already, one from the first section, one from the second. The cells between those are in no room at present, but they are exactly the path cells we want in our new room, which we would like to have be immune to other rooms connecting.

From the viewpoint of the dungeon creator, this “object” is important. Somewhere along it we’ll want to place the “problem” that Dot has to solve to move to the next section. We could put that problem anywhere along the object, including in the two cells that are in the existing rooms.

At this moment, I don’t know what we need or how we’ll use it, but it seems clear that we will want to remember all this. It seems that we could remember it in the layout, or we could remember it right here in the build steps. I think it’s premature to put it into the layout. Let’s continue to build and see what happens.

Let’s do some asserts on what came back, and make a room of it. Then we’ll see if we can make it impermeable. I wonder how we could test that. Here we go for the room connecting part:

    layout.clear_subregions()
    path_cells = suite_1.find_extended_path(suite_2)
    # now path_cells[0] is in one suite, and
    # path_cells[-1] in the other, and
    # path_cells[1:-1] are the cells we need for
    # the new path room.

    room_2 = path_cells[0].room
    assert room_2 in suite_2.room_set
    room_1 = path_cells[-1].room
    assert room_1 in suite_1.room_set
    path_room_1_2 = Room(path_cells[1:-1])
    layout.add_room(path_room_1_2)
    layout.make_passages()
    assert path_cells[0].has_passage(path_cells[1])
    assert path_cells[-1].has_passage(path_cells[-2])
    yield

This test passes. I think it could fail if our two Suites turned out to be adjacent. Let me write a fake test for that.

Should we mark it to skip, or let it fail, or what? For now I’ll just leave this:

    def test_two_adjacent_sections(self):
        pass

We’ve done just about an hour of work and writing. Let’s have a break.

Summary So Far …

Our first step to connecting the section Suites turned out not to be in the right direction. We just belayed that one and took a different course, which seems to be working out so far. Given that the sections are not touching one another, we can draw a path between them, including the cells to be connected in the closest points between the suites. The inner cells of that path are the ones we want in our room connecting the cells. The outer two cells are the ones we connect to.

So we have all the facts we need to do our work, though we’re probably doing too much detailed work in the steps. When we have all the steps figured out, it should become more clear how to package them up for proper use.

It occurs to me that some decades back I’d have been terrified that we’re building up these things but don’t have any real idea how they’ll be used to allocate things in the dungeon, and so on and so on. I’ve come to be confident that while I will surely make a bit of a mess, I’m very unlikely to make a mess that can’t be readily cleaned up in small steps.

One might argue, yeah, but if you make 100 small steps out and then 100 small steps in, maybe you could have just made 100 small steps directly to what you needed, if you had just figured it out in advance. Or maybe only 50.

If I were more perfect than I am, maybe. But my experience with careful overall design is that to get it right takes a very long time, and then one has to build pretty much all of it before anything works, and then at the far end the design isn’t all that close to what you really needed anyway.

In short: my experience is that doing a lot of design before starting rarely, if ever, measures up to the dream of Design It Build The Design All Is Well.

But you do you, if you’re still allowed to do you. And if you’re not, well, it saddens me and I hope you’ll soon find a better situation.

We’ll just keep pushing the ball forward, bit by bit, in whatever direction feels right. We’ll see what happens, but it has worked well so far. I think that will continue.

And I’ll see you next time!

Hello, loves!

Today, unless I miss my guess, which frequently happens, we’ll see some of the strength of the YieldStepper notion. Tests FTW.

Our dungeons look much like we intend, with separate room sections connected by single paths between them:

However, we are working to add more capability for our Dungeon Design Team (I have the hat here somewhere). The purpose of those single paths is to allow us to bar entry to the next section until Dot has solved the problems of the current section, including whatever task opens the gateway to the next section. Furthermore, those paths need to have impermeable walls. Normally, in our dungeon building scheme, if two rooms share a wall, the builder code will place a passage in that wall, permitting, well, passage between them. This makes the dungeon much more interesting, with loops in the paths one can follow. But for these inter-section paths we cannot allow such incursions, because they would create shortcuts through which Dot might pass without solving all the puzzles we pose.

I have rough plans for all this. I think that if it were done “right”, it would go something like this:

1. Isolate the code that connects two Suites with a path;
2. During the build, identify each section as a Suite;
3. Use the Suite-connection code to produce a path room;
4. Mark the path room as impermeable.

As all this goes on, the build should record whatever information it needs for door-building, section-loading, whatever.

As things stand, our dungeon build is a function. If need be, it could be a method of a class, and quite likely it always should be. We’ll find out as we proceed.

If we were wise, which assumes facts not in evidence, we would do all this with tests. We’ll give ourselves the best possible chance by beginning with a test. Let’s try … write a build function that builds some rooms and makes a suite of them.

class TestAdvancedBuild:
    def test_hook(self):
        assert False

Fails. We have a new test class! Woot! So far so good. Now I’ll sketch a little build function, then see what it takes to make it work.

    def test_minimal_build(self):
        def steps():
            yield
        layout = DungeonLayout(55, 55)
        stepper = YieldStepper(steps(), layout)
        stepper.run_to_completion()

This runs green. With the yield replaced by pass, we get this error:

    def run_to_completion(self, view=None):
>       for _ in self.steps:
E       TypeError: 'NoneType' object is not iterable

Ah, right. There has to be at least one yield, or Python doesn’t recognize step as iterable. Not an issue in our code.

Here’s a more meaningful test:

    def test_section_to_suite(self):
        def diamond(size):
            if origin := layout.unused_cell_or_none():
                room = RoomMaker(layout).diamond(number_of_cells=size, origin=origin)
                layout.add_room(room)
        def steps():
            layout.subregion_LRBT(0, 20, 0, 20)
            diamond(25)
            diamond(25)
            layout.finish_map()
            rooms = layout.rooms
            assert len(rooms) == 2 or len(rooms) == 3
            suite = Suite(layout, rooms)
            yield
        layout = DungeonLayout(55, 55)
        stepper = YieldStepper(steps(), layout)
        stepper.run_to_completion()

We build two small diamond rooms in the lower left quadrant. We connect them with finish_map after which there will be two rooms if they happened to be adjacent, and three if a path was needed to connect them. We grab the rooms and make a Suite.

So far so good. Let’s think about this.

Reflection

There are at least two issues that strike me. First, this method step is already pretty complicated. But we have an example of a function diamond, that can move some complexity out. So we’ll keep an eye on complexity but it doesn’t seem unsolvable.

Second, while it was easy to get the rooms for the first Suite, since it needs all the rooms we have, getting the next Suite’s rooms seems a bit more tricky. But can’t we just grab all the rooms and subtract the ones we just grabbed? Let’s try extending this test. No. Let’s write a new one, and try to make it better.

    def test_two_sections_to_suites(self):
        def diamond(size):
            if origin := layout.unused_cell_or_none():
                room = RoomMaker(layout).diamond(number_of_cells=size, origin=origin)
                layout.add_room(room)
        def steps():
            layout.subregion_LRBT(0, 20, 0, 20)
            diamond(25)
            diamond(25)
            layout.finish_map()
            rooms_1 = set(layout.rooms)
            assert len(rooms_1) == 2 or len(rooms_1) == 3
            suite_1 = Suite(layout, rooms_1)
            layout.subregion_LRBT(0, 20, 35, 55)
            diamond(25)
            diamond(25)
            layout.finish_map()
            rooms_2 = set(layout.rooms) - rooms_1
            assert len(rooms_2) == 2 or len(rooms_2) == 3
            suite_2 = Suite(layout, rooms_2)
            yield
        layout = DungeonLayout(55, 55)
        stepper = YieldStepper(steps(), layout)
        stepper.run_to_completion()

We converted the rooms from layout to a set and subtracted. Nice enough. Won’t work quite like that for three, though.

I have an idea how to make those two patches of code alike enough to be used for all four of our corners. First this:

    def steps():
        previous_rooms = set()
        layout.subregion_LRBT(0, 20, 0, 20)
        diamond(25)
        diamond(25)
        layout.finish_map()
        all_rooms = set(layout.rooms)
        new_rooms = all_rooms - previous_rooms
        previous_rooms = all_rooms
        assert len(new_rooms) == 2 or len(new_rooms) == 3
        suite_1 = Suite(layout, new_rooms)
        layout.subregion_LRBT(0, 20, 35, 55)
        diamond(25)
        diamond(25)
        layout.finish_map()
        all_rooms = set(layout.rooms)
        new_rooms = all_rooms - previous_rooms
        previous_rooms = all_rooms
        assert len(new_rooms) == 2 or len(new_rooms) == 3
        suite_2 = Suite(layout, new_rooms)
        yield

That’s nearly good: we could just about extract those two bits as a method. But I think this will go better:

    def steps():
        layout.subregion_LRBT(0, 20, 0, 20)
        previous_rooms = set(layout.rooms)
        diamond(25)
        diamond(25)
        layout.finish_map()
        all_rooms = set(layout.rooms)
        new_rooms = all_rooms - previous_rooms
        assert len(new_rooms) == 2 or len(new_rooms) == 3
        suite_1 = Suite(layout, new_rooms)
        layout.subregion_LRBT(0, 20, 35, 55)
        previous_rooms = set(layout.rooms)
        diamond(25)
        diamond(25)
        layout.finish_map()
        all_rooms = set(layout.rooms)
        new_rooms = all_rooms - previous_rooms
        assert len(new_rooms) == 2 or len(new_rooms) == 3
        suite_2 = Suite(layout, new_rooms)
        yield

The previous version was accumulating previous_rooms, but we have no need to do that, since the layout does that as a matter of course.

Now we should be able to extract a function make_diamond_section. PyCharm does exactly what we had in mind:

    def test_two_sections_to_suites(self):
        def diamond(size):
            if origin := layout.unused_cell_or_none():
                room = RoomMaker(layout).diamond(number_of_cells=size, origin=origin)
                layout.add_room(room)
        def make_diamond_section() -> set[Any]:
            previous_rooms = set(layout.rooms)
            diamond(25)
            diamond(25)
            layout.finish_map()
            all_rooms = set(layout.rooms)
            new_rooms = all_rooms - previous_rooms
            assert len(new_rooms) == 2 or len(new_rooms) == 3
            return new_rooms
        def steps():
            layout.subregion_LRBT(0, 20, 0, 20)
            new_rooms = make_diamond_section()
            suite_1 = Suite(layout, new_rooms)
            layout.subregion_LRBT(0, 20, 35, 55)
            new_rooms = make_diamond_section()
            suite_2 = Suite(layout, new_rooms)
            yield
        layout = DungeonLayout(55, 55)
        stepper = YieldStepper(steps(), layout)
        stepper.run_to_completion()

The test continues to run green. Maybe we should move the assertion down into steps? No real need, this is just a test and example of how we might really do things.

I’ve only been at this for 80 minutes, but this is a super time to break and sum up. Commit: new tests demonstrating use of YieldStepper.

Summary

We see above that with YieldStepper we can put any code we want into the build steps. With the previous BuildStepper, we were much more constrained. So YieldStepper is showing its strengths already. Chuffed, I am, veritably happy as a dog with two tails.

I’m also pleased that I decided to do this in tests. I am often too lazy to do it, and when I do begin with tests, it usually goes better and never goes worse than trying to just jam the code in. It’s a shame that my habits aren’t better than they are. I do know better. And sometimes, like today, I actually do better.

It was much easier to build things up in small chunks, and easy to identify opportunities to remove duplication, which, as all here know, is my primary way of improving code. Doing something twice is a sign that there is a thing that can be isolated and just used instead of copied.

I want to call out the steps toward making the two sections more similar. I learned that trick around a quarter-century ago, when Kent Beck, working on the big screen, made two very different bits of code look alike, and then, when they looked alike, extracted them into a method. All looking on were amazed. Those were the days when we were all like gods, striding across the code, making it better. Or something.

But the point remains: the similarity was already evident, but by moving things around, and noticing that the previous values we needed were actually just the current set or rooms, we enabled the extraction that makes the code above more clear and rather nicely reusable. I’m sure we can extend this morning’s approach to making Suite instances for each section of our dungeon.

We’re still left with breaking out a Suite-to-Suite path maker, and making that path impermeable. But when we do that, and we will do that, assuming that I last that long, we’ll have the data we need in the build and we’ll just have a new method on layout that does the inter-section paths.

Unless I miss my guess, which is always a possibility. That’s what makes real programming fun.

And I’ll offer this bet: if you are currently condemned to using LLM-AI to build code, I’ll bet I’m having more fun than you are.

See you next time for more fun!

Hello, loves!

I think this morning we get to do the fun part with our new YieldStepper. Ants??

But first, we need to put in the view-handling part. When stepping the build, we need to update the view on every step. And in the BuildStepper, there’s some special code at the end of things, but maybe we’ll do that differently. To begin, however, I think we need to stick closely to the BuildStepper, which goes like this:

class BuildStepper:
    def __init__(self, table, layout, dungeon ):
        self.table = table
        self.index = 0
        self.layout = layout
        self.dungeon = dungeon
    def run_to_completion(self, view=None):
        while self._more_to_do():
            self._do_one_step(view)
        self._update_view(view)
    def maker_single_step(self, view=None):
        if self._more_to_do():
            self._do_one_step(view)
            self._update_view(view)
        return self._more_to_do()
    def _update_view(self, view):
        if view:
            sub = self.layout.subregion
            self.layout.clear_subregions()
            view.update_KSL()
            self.layout.subregion = sub
    def _more_to_do(self):
        return self.index < len(self.table)
    def _do_one_step(self, view):
        self.table[self.index](self.layout, self.dungeon)
        self.index += 1

The YieldStepper, so far, looks like this:

class YieldStepper:
    def __init__(self, steps):
        self.steps = steps
    def run_to_completion(self):
        for _ in self.steps:
            pass
    def maker_single_step(self):
        try:
            next(self.steps)
            return True
        except StopIteration:
            return False

I think that we may find that we can pass the view as an optional construction parameter, but for now let’s stick to the existing style. Since this code is driving a view, I’m not going to TDD it, as tests of views are difficult, slow, and anyway I don’t want to.

    def run_to_completion(self, view=None):
        for _ in self.steps:
            pass
        self._update_view(view)
    def maker_single_step(self, view=None):
        try:
            next(self.steps)
            self._update_view(view)
            return True
        except StopIteration:
            return False
    def _update_view(self, view):
        if view is not None:
            sub = self.layout.subregion
            self.layout.clear_subregions()
            view.update_KSL()
            self.layout.subregion = sub

PyCharm points out that we don’t have a layout to talk to. Let’s add that as an optional parameter as well, this time to the constructor.

class YieldStepper:
    def __init__(self, steps, layout=None):
        self.steps = steps
        self.layout = layout

PyCharm is happy, the tests are happy, and I want a save point for what’s next. Commit: adding layout and view to YieldStepper.

So far, all this code is in the test file. We’ll leave it there but now we have to change ‘main’ substantially, to use the new stepper.

To minimize work and change, I use the old methods to draw some rooms:

def yield_table_steps(layout):
    make_a_diamond_room(layout, None)
    yield
    make_a_diamond_room(layout, None)
    make_a_cave_room(layout, None)
    yield
    finish(layout, None)
    yield

And I bash main like this:

def main():
    layout = DungeonLayout(55, 55)
    dungeon = Dungeon(layout)
    # stepper = make_build_table(layout, dungeon)
    stepper = YieldStepper(yield_table_steps(), layout)
    screen_multiplier = 16
    screen_width = screen_multiplier*dungeon.max_x
    screen_height = screen_multiplier*dungeon.max_y
    arcade.Window(screen_width + 8*screen_multiplier, screen_height, 'Caveat Emptor')
    view = DungeonView(dungeon)
    # for single stepping comment out next TWO lines
    stepper.run_to_completion(view) #
    dungeon.run()
    # for single stepping comment out previous TWO lines
    view.run(stepper)

This is enough to find out why it doesn’t work. Forgot layout:

    stepper = YieldStepper(yield_table_steps(layout), layout)

There were squiggles, I just didn’t notice them. Now when I run it, the game comes up with an empty no rooms visible dungeon, but the starting message scrolls.

We’ll drop in some prints to see where we’re going.

My second run shows a dungeon that looks as I expect. What happened the first time, I’m rather sure, is that the few rooms I created just didn’t show up in the full view, and I never zoomed out.

Let me try the single stepping. When I comment out the two magic lines that turn on the stepper, I notice that one of them is dungeon(run), which I think was somehow left out of YieldStepper. But we should be able to see things happening step by step anyway.

Stepping works, and the dungeon does run. Ah. I forgot, we built that into the KeyPress code:

elif symbol == arcade.key.D:
    more = self.stepper.maker_single_step(self.view)
    if not more and not self.called_run:
        self.dungeon.run()
        self.called_run = True

I think it’s actually working! I expected at least some difficulty with the untested bits but we seem to have found an acorn. Excuse me for a moment while I set up the whole build process from BuildStepper, in the YieldStepper.

I’ll spare you the whole thing, but it goes like this:

def yield_table_steps(layout, dungeon):
    bottom_left(layout)
    make_a_diamond_room(layout)
    make_a_diamond_room(layout)
    make_a_cave_room(layout)
    make_a_diamond_room(layout)
    make_a_diamond_room(layout)
    finish(layout)
    yield
    top_left(layout)
    make_a_round_room(layout)
    ...
    populate(layout, dungeon)
    yield
    add_content(layout, dungeon)
    yield

We see already what is nice about this scheme: we can put in as many or as few yield statements as we want. So I elected to draw each section’s rooms in one go, then the connecting paths between them. Let’s have a movie of what happens now.

That is very much exactly what is supposed to happen. With the YieldStepper, we can build up a dungeon definition as above, with no yield statements, and then, if we want to observe some part of the build, we can insert just the ones we want. And, of course, it’s now easier to insert print statements where we may want them, and, for those who like to run the debugger, we could set a breakpoint right in the middle of the build.

The changes to ‘main’ are destructive to the BuildStepper style, and this is working, so I’ll clean out the BuildStepper bits and commit this: converted to YieldStepper.

Summary

So this has gone just as I had thought (and hoped) that it would. The yield approach makes defining a dungeon simpler and more flexible. We can use existing functions, as in the example above, or we could write out more detailed code right in line. (Of course we would soon refactor it back out into useful functions, right? Right?)

We can move yield statements around, adding and subtracting them so that we can run chunks that we don’t want to observe, then insert lots of yields around code that we do want to observe.

I think that practicing using yield with those tests probably made things go more smoothly, and I’m sure that TDDing the basics of YieldStepper was helpful, as it left the stepper in a condition very close to its final form, so that adding the view-related bits was unlikely to break anything.

It went smoothly, the stepping capability is far more flexible and convenient, and best of all let’s face it, doing it with yield is quite nifty. Kind of like real programming.

I commented to my fellow Slackers the other day:

One thing that I don’t always like about small steps is that one rarely gets the pleasure of solving a big complex problem. Of course the art lies in never solving big complex problems, instead turning them into small simple problems. But still, one remembers those few times when one solved the whole shebang in one massive mental effort. If that ever really happened. But seriously, when you’re in the flow of solving something with small steps, one just sees a bit, carves it off, solves it, carves another. Everything is like “oh wow that’s hard”, snip snip snip, “oh wow that was easy”. None of that god-like feeling. I kind of miss it, though my god-like days are probably long in the past anyway.

The amazing thing for me about the small step approach is that the whole solution never really comes into my mind as a “whole”. Instead, the code gets more and more capable, and often nicer and nicer, and it can end up in a very good place, and yet all I really did was make tiny things better. I’m moving in a useful direction, and that does require some decent intuition about what would be better.

You were promised ants.

But I feel like an ant. I go around carrying stuff from place to place and then one day I look up and we have this amazing system around us. But the ant could “know” that there are lots of ants making the system happen. If ants knew things. But here chez Ron, there’s just me. And still, things happen.

This particular step was a bit more than just carrying crumbs from place to place: I had a vision of what the YieldStepper could be, and actually built it with that in mind. Obviously no big deal, it only took a couple of hours.

But it has a different feeling than bit-by-bit code improvement. It’s like I called my shot, three rails, eight ball in the corner, and made it.

I don’t know. It feels good whenever I make progress, but today’s is just a bit more tasty than some days. Not a full on Callooh Callay, but at least a bit of a Yippee!

See you next time!

Hello, loves!

Better yet, it will involve actual programming! Let’s improve the Making App.

My brother GeePaw Hill refers to the Making App and the Shipping App as aspects of a software product. The Shipping App is, of course, the bits you ship to the users. The Making App, in contrast, refers to the things that we build into the product that help us to build the product. I have a very nice idea for the Making side of things, and it promises to be both useful and fun to build.

We’ve been working off and on on one of our Making requirements, around the BuildStepper, which lets a developer step through the creation steps that build up the dungeon, so as to see the effect of each step. It’s aimed at one of the teams in our large dungeon product development, the individuals who create layouts and populate them with denizens and curiosa. Of course, there’s only me here. We are using this imaginary dungeon product as a basis for setting up and solving programming issues that arise in real software development, as done by humans.

The BuildStepper is simple and somewhat useful. It is given a list of functions that called in order, and the functions do useful things as part of building up the dungeon. A stepper list looks like this:

steps =  [
        bottom_left,
        make_a_diamond_room,
        make_a_diamond_room,
        make_a_cave_room,
        make_a_diamond_room,
        make_a_diamond_room,
        finish,
        ...

And the functions are passed the layout and the dungeon, and look like this:

def bottom_left(layout, dungeon):
    layout.subregion_LRBT(0, 20, 0, 20)
def make_a_diamond_room(layout: DungeonLayout, dungeon):
    if origin := layout.unused_cell_or_none():
        room = RoomMaker(layout).diamond(number_of_cells=random.choice([25, 41]), origin=origin)
        layout.add_room(room)
def finish(layout, dungeon):
    layout.finish_map()

The Stepper has two modes. It can just call all the functions in sequence, all in one go, or it can call them one at a time under keyboard control. The all in one go is for playing the game, and the step by step is for observing the dungeon building process as we configure it.

For anything we we want to do in building up the dungeon, we “just” have to create a new function and put its name in the list. This works as we intended, and I’m sure we felt that it was just a first step toward what we’d ultimately need, but whether we did or not, the current stepper idea has Serious Drawbacks, including but not limited to:

1. Even for one-liners like bottom_left and finish, we have to write a function so that the stepper can call it;
2. There is no provision for printing any kind of ongoing info, other than creating yet another function to print a message;
3. There’s no good way to do conditional builds;
4. There’s no good way to access information and tuck it away for use in building the dungeon, unless we store info in globals and build yet more functions to save and access it.

So far, in our frequent use of the stepper while working on the path making / subregion idea, I’ve built up a general dislike for how it works. If some poor devils were condemned to working in the dungeon building up things with Stepper, I’m sure I could hear their moans and screams all the way up here in my programming aerie. Anyway, my own irritation with it must have hit some kind of subconscious peak, because

I Have an Idea

The idea has a delightful techie name: coroutines! And Python has a keyword that is just what we need: yield. Let me summarize the ideas for those who need a refresher. Of course we’ll be into the details shortly. Coroutines are used for “cooperative multitasking”. In our use here we’ll be writing a function that can pause in the middle to return a result, and then the next time it’s called, pick up where it left off to provide the next result. The yield keyword is like return, except that on the next call, the function will pick up after the yield instead of at the top.

Python provides yield as part of the notion of “generators”, functions that return a series of result without building the series into a list or other structure, instead just, well, generating values one at a time. And in our use, we’ll use this capability to provide single stepping for our dungeon builders.

For a conceptual example, let’s consider a sequence like the build steps we showed above. If we were not stepping, we might just write a long function that called various other functions:

def build_some_stuff(layout, dungeon):
    layout.subregion_LRBT(0, 20, 0, 20)
    for _ in range(2):
        if origin := layout.unused_cell_or_none():
            room = RoomMaker(layout).diamond(number_of_cells=random.choice([25, 41]), origin=origin)
            layout.add_room(room)
    layout.finish_map()

We would presumably refactor that long function for clarity and convenience, but the point is that it would be written and organized to the liking of the dungeon builder, and relatively unconstrained in what it could do.

But we can’t step a function like that, executing just one bit at a time. Unless we wrote it this way:

def build_some_stuff(layout, dungeon):
    layout.subregion_LRBT(0, 20, 0, 20)
    yield
    for _ in range(2):
        if origin := layout.unused_cell_or_none():
            room = RoomMaker(layout).diamond(number_of_cells=random.choice([25, 41]), origin=origin)
            layout.add_room(room)
            yield
    layout.finish_map()
    yield

Putting in those yield statements turns the function into a Python generator, which means we might call it like this instead of just calling it directly:

for _ in build_some_stuff(layout, dungeon):
    pass

That loop would just keep calling build_some_stuff until it stopped yielding things. Since it’s yielding None we have no reason to hold on to them.

But the cool thing is this: When we want to step the build, our stepping code can just call next on the generator, and it will execute from where it left off down to the next yield.

Here are a couple of tests showing the two ways of calling the same generator:

class TestGenerator:
    def three_things(self):
        yield 1
        yield 2
        yield 3
    def test_three_things(self):
        for i, thing in enumerate(self.three_things()):
            assert thing == i + 1
    def test_three_things_separately(self):
        things = self.three_things()
        assert next(things) == 1
        assert next(things) == 2
        assert next(things) == 3

I wrote a bunch of other little tests, putting yields into functions called by other yields and such, which may come in handy later. We’ll look at those if we need them. But this one is germane to the real idea:

    def make_room(self, layout):
        room = None
        if origin := layout.unused_cell_or_none():
            room = (RoomMaker(layout)
                    .diamond(
                        number_of_cells=random.choice([25, 41]),
                        origin=origin))
            layout.add_room(room)
        yield room
    def test_cannot_just_call_use_next(self):
        layout = DungeonLayout(20, 20)
        # must say next here (or loop)
        room = next(self.make_room(layout))
        assert len(layout.rooms) == 1
        assert room in layout.rooms

Here we have a very small bit of code that makes a very small dungeon, and actually yields the room, just so we can see what it does. That said, my present plan is that we won’t yield anything, unless we get some clever idea as we use the new stepper.

OK, more than enough chatter, let’s get to it. I think we should just TDD up a new kind of BuildStepper and plug it in. It’ll be tiny.

class TestYieldStepper:
    def test_hook(self):
        assert False

Perfect. One test fails. Replace it:

class TestYieldStepper:
    def test_create(self):
        stepper = YieldStepper()
        assert stepper is not None

And, of course …

class YieldStepper:
    def __init__(self):
        pass

You may be thinking “Why doesn’t he just test something real instead of doing these tiny stupid things”. And yes, I could, and yes, I often do. But I find that starting with these baby steps is easy, requires no particular thinking, and provides a basis where thinking can take place. So now we’ll write tests that are little users of YieldStepper, similar to the real ones that we’ll create Real Soon Now.

    def test_run_stepper(self):
        sum = 0
        def steps():
            nonlocal sum
            sum += 1
            yield
            sum += 2
            yield
            sum += 3
            yield
        stepper = YieldStepper(steps)
        stepper.run_to_completion()
        assert sum == 6

run_to_completion is the method on the existing BuildStepper that just executes whatever’s left in the stepper to be done. In the new class it looks like this:

class YieldStepper:
    def __init__(self, steps):
        self.steps = steps()
    def run_to_completion(self):
        for _ in self.steps:
            pass

Test passes. Let’s commit: working on YieldStepper. Now let’s drive out single-stepping:

    def test_maker_single_step(self):
        sum = 0
        def steps():
            nonlocal sum
            sum += 1
            yield
            sum += 2
            yield
            sum += 3
            yield
        stepper = YieldStepper(steps)
        stepper.maker_single_step()
        assert sum == 1

This is enough to fail, although I plan to extend it. maker_single_step is the method we use on the existing stepper, of course, so we’re replicating it here.

    def maker_single_step(self):
        next(self.steps)

Not terribly difficult. We do have an issue, however, which we’ll talk about in a moment. Let’s extend the test first:

    def test_maker_single_step(self):
        sum = 0
        def steps():
            nonlocal sum
            sum += 1
            yield
            sum += 2
            yield
            sum += 3
            yield
        stepper = YieldStepper(steps)
        stepper.maker_single_step()
        assert sum == 1
        stepper.maker_single_step()
        assert sum == 3
        stepper.maker_single_step()
        assert sum == 6

Green. Commit.

The issue is this: In the current stepper, we have this:

    def run_to_completion(self, view=None):
        while self._more_to_do():
            self._do_one_step(view)
        self._update_view(view)
    def maker_single_step(self, view=None):
        if self._more_to_do():
            self._do_one_step(view)
            self._update_view(view)
        return self._more_to_do()

I’m not concerned about passing the the view: that’s trivial. I am concerned about _more_to_do. If anyone is using that feature, we have a problem, which is that generators cannot answer whether there is more to do, they can only try. When a generator is called one more time than it has values, it raises a StopIteration exception. In our new run_to_completion we just iterate until done, and the for eats the StopIteration and stops. But in maker_single_step we can’t really replicate the return of True if there is more to do. We can, however, swallow the exception and return something at that point.

In our code for the ‘D’ operation (D for maker single step, obviously), we have this:

        elif symbol == arcade.key.D:
            more = self.stepper.maker_single_step(self.view)
            if not more and not self.called_run:
                self.dungeon.run()
                self.called_run = True

Basically that’s protection against calling run more than once, but calling it once.

Let’s try this. We’ll require one D past the last yield and do the run then. We’ll return True from maker_single_step if we did a command, and False if we’re off the end.

Write a test for that.

    def test_maker_single_step_return(self):
        sum = 0
        def steps():
            nonlocal sum
            sum += 1
            yield
            sum += 2
            yield
            sum += 3
            yield
        stepper = YieldStepper(steps)
        ok = stepper.maker_single_step()
        assert ok is True
        assert sum == 1
        ok = stepper.maker_single_step()
        assert ok is True
        assert sum == 3
        ok = stepper.maker_single_step()
        assert ok is True
        assert sum == 6
        ok = stepper.maker_single_step()
        assert ok is False
        assert sum == 6

And:

    def maker_single_step(self):
        try:
            next(self.steps)
            return True
        except StopIteration:
            return False

Green. Commit.

Reflection

This is an excellent opportunity to reflect, and, quite likely, to call a break in the proceedings, since the object is done (except for the View parameter, which is still trivial) needing only to be put into production and fed a sequence for its debut and final testing.

The existing BuildStepper is 33 lines long and is not as capable as this one is, while also being harder to use. The new one is presently 14 lines and will add fewer than ten when we add in the view updating which it does in fact need to do. The real proof of the idea, of course, will come when we create a sequence and actually use it to build a dungeon, testing it all in one go, and step by step (slowly I turned …).

I have one semi-large concern. It’s only one line, how large could it be, but it is this:

class YieldStepper:
    def __init__(self, steps):
        self.steps = steps()

YieldStepper calls the steps generator. What will happen if we need a parameterized generator? Let’s try to test that before the break.

    def test_parameterized(self):
        sum = 0
        def steps(count):
            nonlocal sum
            for c in range(count):
                sum += (c + 1)
                yield
        stepper = YieldStepper(steps(3))
        stepper.run_to_completion()
        assert sum == 6

This fails trying to call the already called generator:

E       TypeError: 'generator' object is not callable

OK, new rule, pass in a called generator. Or we could get fancy and call it if it needs calling? No let’s not.

class YieldStepper:
    def __init__(self, steps):
        self.steps = steps

Commit that. Let’s sum up and break.

Summary

This is going nicely. The yield works perfectly, and I’m sure we’ll be able to put as many yields in as we wish, keep track of things inside the build, and so on.

It’s tempting to push on and finish up, just need to put in the view and its update, which we can copy from BuildStepper, and then create an example. But I’ve been coding and writing for a couple of hours, so taking a break seems prudent. See you later!

Hello, loves!

If we make the path-maker sensitive to active regions, would that solve our issues? It might, but we solve them without that. Most peculiar.

Note

Not much code is written below, but then I remove it all and discover that I don’t need it.

The issue we need to fix is that once we have populated three corners of the dungeon, we will have correctly connected those three: the first two have no choice but to connect, and two and three will connect correctly because #2 is closer to #3 than #1 is. But when we go to connect up corner #4, it is a toss-up whether #1 is closer to #4, or #3 is. #2 is right out.

My first idea, as mentioned in the blurb, is to make the path making sensitive to region. If, before connecting #3 and #4, we set a right-side region, and only select rooms that fall in that region, it should do the right thing. Let’s see how we might do that:

class DungeonLayout:
    def finish_map(self):
        self.ensure_connected()
        self.make_passages()
    def ensure_connected(self):
        count = 0
        while not self.is_fully_connected() and count < 10:
            count += 1
            self._make_path_rooms()
    def _make_path_rooms(self):
        suites = self.define_suites()
        # self.check_intersection(suites)  # debug method
        # for s1, s2 in zip(suites, suites[1:] + suites[:1]):
        for s1, s2 in zip(suites, suites[1:]):
            cells = s1.find_path_cells(s2)
            if cells:
                room = Room(cells, 'path')
                self.add_room(room)

The commented-out for is the one that links back to front, always resulting in a loop among the rooms connected. We don’t want that all the time, so I’ve just turned it off for now at least.

In the code above we add rooms that connect separate “suites” together. We do not define the actual passages until later, in make_passages. I am hopeful that we can leave make_passages alone, at least for now.

If define_suites only found the right suites, I think we’d likely be OK. So …

class DungeonLayout:
    def define_suites(self):
        suites: list[Suite] = []
        unexplored = self.rooms.copy()
        while unexplored:
            suite: Suite = self.find_suite(unexplored[0])
            suites.append(suite)
            unexplored = [room for room in unexplored if room not in suite.room_set]
        return suites

If we only considered the current region as we set up unexplored, we should get the right result. Let’s try Wishful Thinking:

    def define_suites(self):
        suites: list[Suite] = []
        unexplored = self.rooms_in_subregion()
        while unexplored:
            suite: Suite = self.find_suite(unexplored[0])
            suites.append(suite)
            unexplored = [room for room in unexplored if room not in suite.room_set]
        return suites

Now we just have to write that method:

class DungeonLayout:
    def rooms_in_subregion(self):
        return [room for room in self.rooms if self.subregion.contains_room(room)]

So far so good. Now:

class SubRegion:
    def contains_room(self, room):
        for cell in room:
            if cell not in self:
                return False
        return True

That looks right. Ask me where my tests are, but not until we try this.

Lost Track

Somewhere between that last sentence and this morning, I was just fiddling, and finally realized that my subregions weren’t quite right, especially at the end. Even with all the Wishful Thinking code removed, and the right sequence of subregion setup, we’re getting maps that are what we intend, such as this one:

That’s really what I was trying to do, and the glitches I was seeing had to do with improperly setting up the subregions, making the code fail to discover space for the paths needed.

Running a few more random dungeons, I do get this map:

I’m not sure whether to consider that a defect or not. If we put the door to the central boss room at the entrance to the room, this would be perfectly valid. Even if we put the door at the beginning of the stub path from the vertical hallway, it would be valid, and I’m rather sure that’s where the stub path goes.

Fiddling with the subregions generated some maps that weren’t acceptable but also this very fine one:

I think we’re pleased with our results here. Setting the subregions back to the ones that created the first two maps above, we’ll commit the changes. As it turns out, after some reasoning errors on my part, all we seem to need is careful selection of subregions and stepper flow. The stepper looks like this:

def make_build_table(layout, dungeon):
    steps =  [
        bottom_left,
        make_a_diamond_room,
        make_a_diamond_room,
        make_a_cave_room,
        make_a_diamond_room,
        make_a_diamond_room,
        finish,
        top_left,
        make_a_round_room,
        make_a_round_room,
        make_a_round_room,
        make_a_round_room,
        make_a_round_room,
        finish,
        clear_subregions,
        finish,
        top_right,
        make_a_cave_room,
        make_a_diamond_room,
        make_a_cave_room,
        make_a_cave_room,
        make_a_round_room,
        finish,
        clear_subregions,
        finish,
        bottom_right,
        make_a_cave_room,
        make_a_diamond_room,
        make_a_cave_room,
        make_a_diamond_room,
        make_a_cave_room,
        finish,
        right_side,
        finish,
        clear_subregions,
        make_diamond_in_round_room,
        bottom_right_plus_center,
        finish,
        clear_subregions,
        populate,
        add_content,
    ]
    return BuildStepper(steps, layout, dungeon)

Let’s sum up.

Summary

The trick to getting two sections to connect is to make the subregion, draw the section, finish to make sure it is connected, then clear the subregions and finish again, which will connect everything that isn’t connected. You can exert sufficient control what connects by setting the subregions properly.

With my product manager hat on (a very jaunty fedora, I might add) this scheme seems to be doing what I asked for, so far, and I suspect that we’re going to want better control over the layout. Discussing this with the lead developer (he doesn’t wear a hat at all and his hair is a mess) we agree to wait and see, and I’m comfortable that we’ll be able to improve the situation should we need to.

In the actual world of ronjeffries.com, we’re just here to set ourselves interesting improvements to the program that tend to break the design, so as to discover that we can improve it without massive rewriting delays, which have, in my development lifetime, killed more than one project that didn’t deserve to die.

What is interesting about this problem, which is to create a dungeon in sections that could be separately populated with problems and solutions, is that while I did try a lot of ideas to get this to happen, so far most of the work has been in configuring subregions that will cause path creation to do what we want.

We do still have one or two things to do, and some new ideas have come to mind:

1. We need to provide some way to control “doors”, or whatever kind of obstacles the dungeon designer wants to place between sections. This will require more feedback between the build sequence and the designer, some kind of trace of where key locations are, perhaps.

2. We probably want to create our inter-section paths to be impermeable, so that they don’t get openings other than at the ends, which can lead to shortcuts that the designer did not contemplate.

3. We might want to provide a more robust stepper “language” that gives more control to the dungeon designer.

4. The Suite object came into view a couple of times during this exercise, and it is possible that we’d benefit by making it more explicit and capable. For example, if a subregion’s rooms automatically formed a Suite, we might have better control over how paths are created.

5. The current paths draw are quite straight. We have the ability to make them more wandering, and might want to use that, or even enhance it.

6. … We’ll run out of interest in this program long before we run out of ideas. I fully intend to be working on something different by 2027, assuming that I last that long.

Summary Summary

Small changes leading from where we are to where we need to be seem always to be possible. Of course there are exceptions. I might not be able to figure out a small step transition from Python on the Mac to COBOL on a mainframe. Oh, wait, we already figured out how to go from COBOL on the mainframe to Smalltalk on a SparcStation in small steps. Maybe something similar … but no one wants a dungeon in COBOL anyway …

Small steps, my friends, seem always to be possible. They seem to be easier to make, less prone to errors, and more flexible with respect to interweaving other features.

Fascinating. See you next time!

Hello, loves!

If we draw suite paths in the right order, it should give us the layout we want with no extra work. If this works, it’s a lovely idea. Herein we eschew pompous prolixity.

The new PassageCollection object, which I am inclined to rename to Passages, is now in place. It ensures that once a passage between two rooms has been formed, that passage is preserved even if we’re nominally computing new passages. We did that because when stepping the build for the developer, we want to see the passages as we go, and the passage-computing code would make new decisions about where things should go.

Arguably, we coulda shoulda changed the passage-making in some fundamental way so as to specify somehow just exactly what it should do. That seemed daunting, and once we created the Passages object (see what a nice name that is, gonna change it right now … done) as I was saying, once we created the Passages object, it was very simple to have it keep track of which room pairs already have passages and just ignore any attempts to add passages to them. We even did it with a callback, so that user code need not waste time preparing a path that is inevitably going to be rejected.

All very nice, and quite simple. The task before us now is that given groups of rooms that we have created in subregions of the map, we want to connect those groups with paths that will control the major flow through the dungeon: first this area, then this next area, and so on.

These “major” paths (just long thin rooms really) need to be different from most rooms. Our general rule is that if two rooms share a border, we’ll build a passage across that border. This rule tends to keep the dungeon connected, and seems to make interesting layouts with loops and different ways to get places. But for these inter-region paths, we want that rule changed. We do not want a surprise entry into the side of one of these paths if it happens to pass by some other room. If we were to allow that, there would be a “leak” in the dungeon, allowing the player to bypass some section without solving the problems in that section.

I believe, though I could be wrong, that this is going to turn out to be quite easy, and rather elegant. We’ll see, probably next time.¹

So we’ll need to do something special when we connect up our sections, to ensure that the path we build is not permeable with respect to surprise adjacencies².

Before we get there, however, we need to get the paths laid in, and I have an idea about that as well. There are two methods that work together to keep the dungeon connected:

class DungeonLayout:
    def finish_map(self):
        self.ensure_connected()
        self.make_passages()

ensure_connected looks at the existing rooms and while they are not all connected, adds path rooms between connected “suites”. It just makes paths that are adjacent to a pair of suites, and then make_passages connects things up. It’s in make_passages that the extra entries into the sides of a path are formed.

So. Imagine that we want to create a dungeon with four sections in the four corners, and a boss room, round with a diamond center, in the middle of the dungeon. (That’s what the current build does, you may recall.)

It seems to me that if we were to build the center bit last, we could build section 1, connect it up, build section 2, connect it up, then connect 1 and 2, then build 3, connect it up then connect (1,2) and 3, and so on, it would just work. Once 1 and 2 are connected, and 3 is internally connected, it looks like two suites to the finish_map and it should just hook them together. So we should be able to get the topology we want by just deciding where to place the sections and placing them in the order we want them connected.

So I propose that we begin by changing the build to do that. Here’s what it looks like now:

def make_build_table(layout, dungeon):
    steps =  [
        make_diamond_in_round_room,
        finish,
        bottom_left,
        make_a_diamond_room,
        make_a_diamond_room,
        make_a_diamond_room,
        finish,
        top_left,
        make_a_round_room,
        make_a_round_room,
        make_a_round_room,
        finish,
        bottom_right,
        make_a_cave_room,
        make_a_cave_room,
        make_a_cave_room,
        finish,
        top_right,
        make_a_cave_room,
        make_a_diamond_room,
        make_a_round_room,
        finish,
        clear_subregions,
        finish,
        populate,
        add_content,
    ]
    return BuildStepper(steps, layout, dungeon)

Let’s reorder things to go bottom left, top left, top right, bottom right, center, and add in some additional connections. (I think we’ll initially get too many, but we know how to fix that if we do.)

New stepper:

def make_build_table(layout, dungeon):
    steps =  [
        bottom_left,
        make_a_diamond_room,
        make_a_diamond_room,
        make_a_diamond_room,
        finish,
        top_left,
        make_a_round_room,
        make_a_round_room,
        make_a_round_room,
        finish,
        clear_subregions,
        finish,
        top_right,
        make_a_cave_room,
        make_a_diamond_room,
        make_a_round_room,
        finish,
        clear_subregions,
        finish,
        bottom_right,
        make_a_cave_room,
        make_a_cave_room,
        make_a_cave_room,
        finish,
        clear_subregions,
        finish,
        clear_subregions,
        make_diamond_in_round_room,
        finish,
        populate,
        add_content,
    ]
    return BuildStepper(steps, layout, dungeon)

Note the clear_subregions finish combinations. Those are supposedly going to directly connect around the square. Let’s see what it really does.

So that’s not bad. We have those double paths, the extra paths I was referring to. I think I almost like those. I was imagining that they’d probably be more separated but since we try to find the closest cells between suites to draw our paths, it makes sense that they’ll tend to be side by side.

The reason they are drawn at all is that we draw paths all the way around the suite set from 1 to 2, 2 to 3, n-1 to n, n back to 1. We can suppress that last drawing if we care to.

The other issue is that we wanted the path to the boss room to come from bottom right and it does not. We can fix that with a better subregion for the last bit.

In principle, what we really want, I suspect, is for subregions to be more than mere rectangles, but let’s defer that concern for now. If we do that final finish in a subregion containing just bottom right and the area for the boss room it should do just what we want.

As drawn, it appears to me that the boss room overlaps the top rectangle. Let’s examine the rectangles as defined and see what we need to do:

main.py
def bottom_left(layout, dungeon):
    layout.subregion_LRBT(0, 17, 0, 23)
def top_left(layout, dungeon):
    layout.subregion_LRBT(0, 17, 33, 55)
def bottom_right(layout, dungeon):
    layout.subregion_LRBT(37, 55, 0, 23)
def top_right(layout, dungeon):
    layout.subregion_LRBT(37, 55, 33, 55)

I’ll adjust those to make a little room, and add one more subregion, to include bottom right plus the central area:

main.py
def bottom_left(layout, dungeon):
    layout.subregion_LRBT(0, 20, 0, 20)
def top_left(layout, dungeon):
    layout.subregion_LRBT(0, 20, 35, 55)
def bottom_right(layout, dungeon):
    layout.subregion_LRBT(35, 55, 0, 20)
def top_right(layout, dungeon):
    layout.subregion_LRBT(35, 55, 35, 55)
def bottom_right_plus_center(layout, dungeon):
    layout.subregion_LRBT(21,55, 0, 34)

Also I made the change that doesn’t make a path that circles back:

    def _make_path_rooms(self):
        suites = self.define_suites()
        # self.check_intersection(suites)  # debug method
        # for s1, s2 in zip(suites, suites[1:] + suites[:1]):
        for s1, s2 in zip(suites, suites[1:]):
            cells = s1.find_path_cells(s2)
            if cells:
                room = Room(cells, 'path')
                self.add_room(room)

It’s that + suites[:1] that makes us circle back to the first suite. Those changes give us this picture:

That is almost as intended. Woot! Note the tiny loop where the boss room connects to the lower right. I think that is our adjacent paths inter-penetrating and it’s the thing that I think we can fix elegantly … next time.

I want to make it more interesting. Let’s add some rooms.

Hm. Now there is a connection lower left to lower right that I do not expect, and no connection from upper right to lower left. I think I know why that’s happening: the path-finding isn’t sensitive to the available regions, and the bottom two are closer together than the right side two.

This is a surprise to me and will have to be dealt with. My brilliant idea seems to be slightly flawed. Let’s hope it’s just slightly. If we can make passage-making sensitive to regions, the idea holds water. If not … I’ll need another idea, but I want this one to work. We’ll see.

So that’s fun. In addition to fixing this up, we still have one thing to do, having to do with the double paths. We may need one or two more sessions to get this done. Interesting …

Hello, loves!

I think PassageCollection should have the protection against duplication built in. We have a lambda, how nice.

As things stand, the user of the PassageCollection is responsible for ensuring that there’s only one passage per room. Since that’s the default behavior we want, let’s build it into the object. Let’s have add_passage return True if the passage was added, and False if it was not.

Creating a passage to add is costly, so let’s see if we can pass in a lambda that is only called if needed. Let’s enhance our test.

    def test_create_passage(self):
        layout = DungeonLayout(10, 10)
        room_1 = Room([Cell(0,0), Cell(0,1), Cell(0,2)])
        room_2 = Room([Cell(1,0), Cell(1,1), Cell(1,2)])
        layout.add_room(room_1)
        layout.add_room(room_2)
        passages = PassageCollection()
        cells = lambda: (Cell(0,0), Cell(1,0))
        done = passages.create_passage(room_1, room_2, cells)
        assert done == True
        assert len(passages.cell_connections) == 2
        assert len(passages.room_connections) == 2
        cells = lambda: (Cell(0,22), Cell(1,2))
        done = passages.create_passage(room_1, room_2, cells)
        assert done == False
        assert len(passages.cell_connections) == 2
        assert len(passages.room_connections) == 2

And this:

class PassageCollection:
    def create_passage(self, r1: Room, r2: Room, func):
        if self.rooms_are_connected(r1, r2):
            return False
        else:
            passage = func()
            self.add_passage(passage)
            return True

I expect that no one will look at the return except for the tests.

Now we can change the production code from this:

class DungeonLayout:
    def make_passages(self):
        for r1, r2 in itertools.combinations(self.rooms, 2):
            if not self.passages.rooms_are_connected(r1, r2):
                passage = self.random_passage(r1, r2)
                self.add_passage(passage)

To this:

    def make_passages(self):
        for r1, r2 in itertools.combinations(self.rooms, 2):
            self.passages.create_passage(r1, r2, lambda: self.random_passage(r1, r2))

Much nicer. Let’s rename the add method now.

    def create_passage(self, r1: Room, r2: Room, func):
        if self.rooms_are_connected(r1, r2):
            return False
        else:
            passage = func()
            self.add_passage_unconditionally(passage)
            return True
    def add_passage_unconditionally(self, passage:tuple[Cell, Cell]):
        if passage:
            c1, c2 = passage
            self.cell_connections.add((c1, c2))
            self.cell_connections.add((c2, c1))
            self.room_connections.add((c1.room, c2.room))
            self.room_connections.add((c2.room, c1.room))

That looks good. Commit: PassageCollection create_passage calls creation lambda only if a passage is not already present for the provided rooms.

Summary

I almost invariably find that after a break of a half day or day, some improvement ideas come up for something I’ve just done. I find it both useful and pleasant to pay attention to the ideas and usually find something worth doing.

Here, we have moved a bit of complexity inside the PassageCollection, and our use of it is a bit simpler, although we do now have a lambda to call, but as that is quite common in the dungeon, I think it’s fair to think of it as a well-understood technique.

And there is no one here to say me nay. See you next time!