The Robot World Repo on GitHub
The Forth Repo on GitHub

It turns out that a bit more study would have shown me that I already did a trivial Sys object and put it in play. Let’s normalize that and then move a bit forward. A small surprise, and a bit of improvement.

I was browsing the code and noticed this test:

    def test_c_stack_set_up(self):
        f = Forth()
        f.process_line(': TEST 3 CASE GET_C_STACK ENDCASE ;')
        assert f.c_stack.name == 'CASE'
        assert f.c_stack.cells == []

And that is supported by this code:

class Sys:
    def __init__(self, name):
        self.name = name
        self.cells = []

class Lexicon:
    def define_case_of_endof_endcase(self):
        def _get_c_stack(f):
            f.c_stack = f.compile_stack[-1]

        def _case(f):
            sys = Sys('CASE')
            f.compile_stack.push(sys)

        def _endcase(f):
            f.word_list.append(f.find_word('DROP'))
            f.compile_stack.pop()

And this morning we did this test:

    def test_pattern(self):
        f = Forth()
        test = (': TEST'
                '  2 CASE'
                '    1 OF 111 ENDOF'
                '    2 OF 222 ENDOF'
                '  ENDCASE '
                ';')
        expected_stack = [222]
        f.process_line(test)
        w = f.find_word('TEST')
        branch_location = w.index('0BR') + 1
        # branch_location = 4 + 1
        target_location = w.index('0BR', branch_location) - 3
        # target_location = 13 - 3
        assert w.words[branch_location] == target_location

Supported by this code:

class CompileInfo:
    def __init__(self, name):
        self.name = name
        self.locations = []

    def add_target(self, location: int):
        self.locations.append(location)

class Lexicon:
        def _of(f):
            f.word_list.append(f.find_word('OVER'))
            f.word_list.append(f.find_word('='))
            f.word_list.append(f.find_word('0BR'))
            info = CompileInfo('OF')
            info.add_target(len(f.word_list))
            f.compile_stack.push(info)
            f.word_list.append(f.find_word('BR_TARGET'))
            f.word_list.append(f.find_word('DROP'))

So we should pick a class, Sys or CompileInfo. And a word for the contents, is it cells or is it locations?

I think CompileInfo is a safer class and file name than “sys”, and we actually have a method in ClassInfo, so let’s convert over to using it.

        def _case(f):
            sys = CompileInfo('CASE')
            f.compile_stack.push(sys)

And in the test:

    def test_c_stack_set_up(self):
        f = Forth()
        f.process_line(': TEST 3 CASE GET_C_STACK ENDCASE ;')
        assert f.c_stack.name == 'CASE'
        assert f.c_stack.locations == []

In the course of this I also discover:

        self.pw('GET_C_STACK', _get_c_stack, immediate=True)

        def _get_c_stack(f):
            f.c_stack = f.compile_stack[-1]

Ah, yes, I remember now. That is a debug word that I can use to get (and test) the value of the top of the compile stack at a given time. Let’s rename that word and its associated method to give us a better chance of remembering what it is.

class Forth:
    def __init__(self):
        ...
        self.c_stack_top = None
        ...


class Lexicon:
        self.pw('GET_C_STACK_TOP', _get_c_stack_top, immediate=True)

        def _get_c_stack_top(f):
            f.c_stack_top = f.compile_stack[-1]


    def test_c_stack_set_up(self):
        f = Forth()
        f.process_line(': TEST 3 CASE GET_C_STACK_TOP ENDCASE ;')
        assert f.c_stack_top.name == 'CASE'
        assert f.c_stack_top.locations == []

In the test above, we see what we can do with GET_C_STACK_TOP. Right in the middle of the action, we grab the compile stack top and tuck it away where the test can inspect it. So we can see what is happening right in the middle of a compilation, if we need to. I would prefer not to need to. We’ll continue to try new testing ideas as we get them.

Anyway, we’re green. Remove Sys. Commit: Move from Sys to CompileInfo class. Adjust tests. Rename test word to GET_C_STACK_TOP.

Reflection

It’s interesting how much I can forget over the course of a week. I could read my articles, I suppose, but who wants to do that? And in Real Life I wouldn’t be documenting my every move like this anyway.

I “should” have reviewed the case tests more carefully. Had I done so, I’d have noticed Sys this morning. Also, I might not have made the progress that I made.

Let’s check that what we pop off the compile stack is what we expect, for CASE, changing this:

        def _case(f):
            f.compile_stack.push(CompileInfo('CASE'))

        def _endcase(f):
            f.word_list.append(f.find_word('DROP'))
            f.compile_stack.pop()

To this:

        def _endcase(f):
            f.word_list.append(f.find_word('DROP'))
            ci = f.compile_stack.pop()
            assert ci.name == 'CASE', f'expected CASE on compile stack, found {ci.name}'

This will turn into a diagnostic if it ever happens, in our unwinding of words. I’ll make a card to remind me to write a test for it.

Let’s now review how we did the CompileInfo for OF-ENDOF:

        def _of(f):
            f.word_list.append(f.find_word('OVER'))
            f.word_list.append(f.find_word('='))
            f.word_list.append(f.find_word('0BR'))
            info = CompileInfo('OF')
            info.add_target(len(f.word_list))
            f.compile_stack.push(info)
            f.word_list.append(f.find_word('BR_TARGET'))
            f.word_list.append(f.find_word('DROP'))

        def _endof(f):
            f.word_list.append(f.find_word('BR'))
            f.word_list.append(f.find_word('BR_TARGET'))
            info = f.compile_stack.pop()
            location = info.locations[0]
            f.word_list[location] = len(f.word_list)

I think it will be common to create a CompileInfo right when one needs it, as we do here in _of. And we should check that we have the right one and unwind all of its listed locations in closing words like _endof.

I think it might be useful if the CompileInfo included a pointer to the word list. Let’s give it a new construction __init__:

class CompileInfo:
    def __init__(self, name, word_list, location=None):
        self.name = name
        self.word_list = word_list
        self.locations = []
        if location:
            self.locations.append(location)

And we fix up our tests and code:

        def _of(f):
            f.word_list.append(f.find_word('OVER'))
            f.word_list.append(f.find_word('='))
            f.word_list.append(f.find_word('0BR'))
            info = CompileInfo('OF', f.word_list, len(f.word_list))
            f.compile_stack.push(info)
            f.word_list.append(f.find_word('BR_TARGET'))
            f.word_list.append(f.find_word('DROP'))

I’m not sure I love that, but I love it more. We might want a helper method somewhere. Commit this much: making CompileInfo more capable.

Now for _endof, which goes like this:

        def _endof(f):
            f.word_list.append(f.find_word('BR'))
            f.word_list.append(f.find_word('BR_TARGET'))
            info = f.compile_stack.pop()
            location = info.locations[0]
            f.word_list[location] = len(f.word_list)

We do this:

        def _endof(f):
            f.word_list.append(f.find_word('BR'))
            f.word_list.append(f.find_word('BR_TARGET'))
            info = f.compile_stack.pop()
            assert info.name == 'OF', f'expected OF, found {info.name}'
            for location in info.locations:
                f.word_list[location] = len(f.word_list)

Green. Commit: extending CompileInfo.

Then push that code to CompileInfo:

class CompileInfo:
    def patch(self, name):
        assert self.name == name, f'expected {name}, found {self.name}'
        for location in self.locations:
            self.word_list[location] = len(self.word_list)

        def _of(f):
            f.word_list.append(f.find_word('OVER'))
            f.word_list.append(f.find_word('='))
            f.word_list.append(f.find_word('0BR'))
            f.compile_stack.push(CompileInfo('OF', f.word_list, len(f.word_list)))
            f.word_list.append(f.find_word('BR_TARGET'))
            f.word_list.append(f.find_word('DROP'))

        def _endof(f):
            f.word_list.append(f.find_word('BR'))
            f.word_list.append(f.find_word('BR_TARGET'))
            f.compile_stack.pop().patch('OF')

Commit: refactoring, improved capability in CompileInfo.

That’s nearly good. I think we might benefit from a smarter compile stack, but we’ll hold off on that for a bit. And it might be reasonable for the constructor on CompileInfo to assume that it should start its list with len(word_list. We’ll see about that in due time. Oh! We probably will not need to provide a location at all, if we just tell the top of the stack to add location, it can add the length of the word list automatically, because it now has the word list.

Summary

Just a bit of code improvement. I had in mind just doing whatever refactoring seemed desirable in _of and _endof, but discovered those other glitches and bumps. Everything was working, and we’d have discovered the issues soon enough, but I’m glad I spotted them now.

I think there’s definitely some further consolidation and improvement to be had here. And I have a bit of concern about the word_list. It’s just a vanilla list and yet it carries a lot of weight during compilation. That said, in a classical Forth it would just be the end of a long list of cells, continually filled in as new words were created. Not exactly a smart structure there … but Forth really has no such concept. If you want something to be smarter, you build some new words.

Anyway, I feel like I’m coming back up to speed and that things are moving forward. See you next time!