The Repo on GitHub

I’ve come up with an interesting possibility for reducing method clutter. Let’s see what we think. (We think we like it!)

Our immediate words are defined like this:

    def define_immediates(self, lex):
        lex.append(PrimaryWord(':', lambda f: f.imm_colon(), immediate=True))
        lex.append(PrimaryWord(';', lambda f: f.imm_semi(), immediate=True))
        lex.append(PrimaryWord('IF', lambda f: f.imm_if(), immediate=True))
        ...

With associated methods off elsewhere, like these:

    def imm_colon(self):
        definition_name = self.next_token()
        self.compile_stack.push((':', definition_name))

    def imm_semi(self):
        key, definition_name = self.compile_stack.pop()
        word = SecondaryWord(definition_name, self.word_list[:])
        self.lexicon.append(word)
        self.word_list.clear()

    def imm_if(self):
        self.compile_conditional('*IF', self.word_list)

Experimenting, I tried this approach, with local definitions:

    def define_immediates(self, lex):
        def _colon(f):
            self.compile_stack.push((':', (self.next_token())))
        lex.append(PrimaryWord(':', _colon, immediate=True))

        def _semi(f):
            key, definition_name = self.compile_stack.pop()
            word = SecondaryWord(definition_name, self.word_list[:])
            self.lexicon.append(word)
            self.word_list.clear()
        lex.append(PrimaryWord(';', _semi, immediate=True))

        def _if(f):
            self.compile_conditional('*IF', self.word_list)
        lex.append(PrimaryWord('IF', _if, immediate=True))

It seemed to me that putting the code right with the lexicon entry had some merit. It would reduce clutter, making it unnecessary to search for the methods, and in fact it removes the methods from the class’s namespace entirely.

You’ll note that, as written above, the parameter f, the current Forth instance, is not used. So long as the lexicon definition code is Forth class methods, self refers to the current instance. If we were to move the Lexicon construction to another class, we might have to do something a bit different.

I think, though, that since self is just a name, we could write this:

        def _semi(self):
            key, definition_name = self.compile_stack.pop()
            word = SecondaryWord(definition_name, self.word_list[:])
            self.lexicon.append(word)
            self.word_list.clear()
        lex.append(PrimaryWord(';', _semi, immediate=True))

Now PyCharm objects that calling that self shadows the outer self. And I really think we’d do best to keep it named f or forth. Bad idea, belay that. Too clever by half.

    def define_immediates(self, lex):
        def _colon(forth):
            forth.compile_stack.push((':', (forth.next_token())))
        lex.append(PrimaryWord(':', _colon, immediate=True))

        def _semi(forth):
            key, definition_name = forth.compile_stack.pop()
            word = SecondaryWord(definition_name, forth.word_list[:])
            forth.lexicon.append(word)
            forth.word_list.clear()
        lex.append(PrimaryWord(';', _semi, immediate=True))

        def _if(forth):
            forth.compile_conditional('*IF', self.word_list)
        lex.append(PrimaryWord('IF', _if, immediate=True))

I think I rather like this notion. It will surely work for any word that can’t be done with a simple lambda. I could imagine doing them all this way, even the ones that can be lambdas.

I’ll convert the rest of the immediates. The result is this:

    def define_immediates(self, lex):
        def _colon(forth):
            forth.compile_stack.push((':', (forth.next_token())))
        lex.append(PrimaryWord(':', _colon, immediate=True))

        def _semi(forth):
            key, definition_name = forth.compile_stack.pop()
            word = SecondaryWord(definition_name, forth.word_list[:])
            forth.lexicon.append(word)
            forth.word_list.clear()
        lex.append(PrimaryWord(';', _semi, immediate=True))

        def _if(forth):
            forth.compile_conditional('*IF', self.word_list)
        lex.append(PrimaryWord('IF', _if, immediate=True))

        def _else(forth):
            forth.patch_the_skip(['*IF'], 1, forth.word_list)
            forth.compile_conditional('*ELSE', forth.word_list)
        lex.append(PrimaryWord('ELSE', _else, immediate=True))

        def _then(forth):
            forth.patch_the_skip(['*IF', '*ELSE'], -1, self.word_list)
        lex.append(PrimaryWord('THEN', _then, immediate=True))

        def _begin(forth):
            forth.compile_stack.push(('BEGIN', len(forth.word_list)))
        lex.append(PrimaryWord('BEGIN', _begin, immediate=True))

        def _until(forth):
            key, jump_loc = forth.compile_stack.pop()
            until = forth.find_word('*UNTIL')
            forth.word_list.append(until)
            forth.word_list.append(jump_loc - len(forth.word_list) - 1)
        lex.append(PrimaryWord('UNTIL', _until, immediate=True))

        def _do(forth):
            forth.compile_stack.push(('DO', len(forth.word_list)))
            forth.word_list.append(forth.find_word('*DO'))
        lex.append(PrimaryWord('DO', _do, immediate=True))

        def _loop(forth):
            key, jump_loc = forth.compile_stack.pop()
            loop = forth.find_word('*LOOP')
            forth.word_list.append(loop)
            forth.word_list.append(jump_loc - len(forth.word_list))
        lex.append(PrimaryWord('LOOP', _loop, immediate=True))

I’m not entirely certain whether I like this or not. It keeps everything together, and I do like that. And, as written, I think we can move the entire define method to another class, such as Lexicon, if we had a Lexicon class, and it would still just work. Alphabetizing might help. I’ll do that.

I have mixed feelings about that. The layout shown above keeps related items together, DO and LOOP, BEGIN and UNTIL, and so on. I think I’ll keep it as above.

Commit: immediates all converted to use local def functions.

We could extract these to single methods or small groups. Allowing PyCharm to reformat and make things static, doing a bit f rearrangement, and we get this:

    def define_immediates(self, lex):
        self._define_begin_until(lex)
        self._define_colon_semi(lex)
        self._define_do_loop(lex)
        self._define_if_else_then(lex)

    @staticmethod
    def _define_begin_until(lex):
        def _begin(forth):
            forth.compile_stack.push(('BEGIN', len(forth.word_list)))

        def _until(forth):
            key, jump_loc = forth.compile_stack.pop()
            until = forth.find_word('*UNTIL')
            forth.word_list.append(until)
            forth.word_list.append(jump_loc - len(forth.word_list) - 1)

        lex.append(PrimaryWord('BEGIN', _begin, immediate=True))
        lex.append(PrimaryWord('UNTIL', _until, immediate=True))

    @staticmethod
    def _define_colon_semi(lex):
        def _colon(forth):
            forth.compile_stack.push((':', (forth.next_token())))

        def _semi(forth):
            key, definition_name = forth.compile_stack.pop()
            word = SecondaryWord(definition_name, forth.word_list[:])
            forth.lexicon.append(word)
            forth.word_list.clear()

        lex.append(PrimaryWord(':', _colon, immediate=True))
        lex.append(PrimaryWord(';', _semi, immediate=True))

    @staticmethod
    def _define_do_loop(lex):
        def _do(forth):
            forth.compile_stack.push(('DO', len(forth.word_list)))
            forth.word_list.append(forth.find_word('*DO'))

        def _loop(forth):
            key, jump_loc = forth.compile_stack.pop()
            loop = forth.find_word('*LOOP')
            forth.word_list.append(loop)
            forth.word_list.append(jump_loc - len(forth.word_list))

        lex.append(PrimaryWord('DO', _do, immediate=True))
        lex.append(PrimaryWord('LOOP', _loop, immediate=True))

    @staticmethod
    def _define_if_else_then(lex):
        def _if(forth):
            forth.compile_conditional('*IF', forth.word_list)

        def _else(forth):
            forth.patch_the_skip(['*IF'], 1, forth.word_list)
            forth.compile_conditional('*ELSE', forth.word_list)

        def _then(forth):
            forth.patch_the_skip(['*IF', '*ELSE'], -1, forth.word_list)

        lex.append(PrimaryWord('IF', _if, immediate=True))
        lex.append(PrimaryWord('ELSE', _else, immediate=True))
        lex.append(PrimaryWord('THEN', _then, immediate=True))

I think I nearly like that. We’ll commit and see how we feel next time. Commit: make grouped define methods for immediates, on the way to similar design for all words.

Let’s reflect by way of summary. We need a word that means both. Perhaps tomorrow.

Reflection

This idea has mixed benefit, I think. On the plus side:

  • It keeps the definition of each word in a single place, rather than two.
  • It allows moving the lexicon creation outside the Forth class, which will give us two classes with narrow interfaces rather than the current one with a broad interface.
  • It has narrowed the Forth interface already, by removing the 9 immediate methods imm_whatever.

To its discredit, I would list:

  • Each word’s definition method is a bit more complicated, including both code and declaration

At this writing, I think it’s a favorable trade. When next we define some tricky words, we’ll see how it holds up.

An open question is whether, when the function is a one-liner, we’d prefer to put it into the lex.append as a lambda. It would reduce clutter, but might not really be all that easy to read. I think we should begin by expanding all the definitions, removing all the current helper methods, and then regenerate helpers from the mess. I love doing that because it seems so counter-intuitive, making the code worse to make it better.

We could probably even enclose the helper methods in with the current _this_and_that methods. We’ll see. I think that is the next step but one.

Next step, most likely, will be to move the lexicon-creation into its own class, supporting a smart object as the lexicon, rather than a simple list. Or some future step: we’ll decide tomorrow what to do tomorrow.

See you then!