The Robot World Repo on GitHub
The Forth Repo on GitHub

I have a little time. Let’s see what we can do. Many tiny changes produce improved code. Who knew? (We did!)

Let’s try a “BYE” word. How do you exit a Python program. Is it exit()? Yes.

        self.pw('BYE', lambda f: sys.exit())

And, I don’t think I want to write a test for that but can remove the special check for ‘bye’ from main.

if __name__ == '__main__':
    forth = Forth()
    prompt = 'Forth> '
    while True:
        result = forth.compile(input(prompt))
        prompt = 'Forth> '
        if result == '...':
            prompt = '...> '
        else:
            print(result)

And in Terminal:

(.venv) Fresh-Air:FORTH ron$ python3 main.py
Forth> 2 2 + .
4 ok
Forth> bye
(.venv) Fresh-Air:FORTH ron$

Perfect! Commit: ‘BYE’.

I think we could check compile mode instead of the return, for the prompt:

if __name__ == '__main__':
    forth = Forth()
    prompt = 'Forth> '
    while True:
        result = forth.compile(input(prompt))
        if forth.compilation_state:
            prompt = '...'
        else:
            print(result)
            prompt = 'Forth> '

Forth> : double dup + ;
ok
Forth> 3 double .
6 ok
Forth> : quad double
...double ;
ok
Forth> 3 quad .
12 ok
Forth>

So that’s fine. Commit. check compilation state in main for prompt changing.

I’m not entirely comfortable with the compile method, whose name needs to be changed, returning a string to be printed, but we aren’t supporting strings in our Forth (yet) so we’ll let that ride.

Shall we hunt for bigger game? Let’s look at these two methods:

    def unsafe_compile(self, text):
        new_text = re.sub(r'\(.*?\)', ' ', text)
        self.tokens = new_text.split()
        self.token_index = 0
        while self.token_index < len(self.tokens):
            token = self.next_token()
            word = self.compile_a_word(token)
            word(self)
        if self.compilation_state:
            raise ValueError('Unexpected end of input')

    def compile_a_word(self, token):
        found_word = self.find_word_or_literal(token)
        if not self.compilation_state or found_word.immediate:
            return found_word
        else:
            self.word_list.append(found_word)
            return Word('no-op', [])

If compile_a_word did the execution, wouldn’t that be more clean?

    def unsafe_compile(self, text):
        new_text = re.sub(r'\(.*?\)', ' ', text)
        self.tokens = new_text.split()
        self.token_index = 0
        while self.token_index < len(self.tokens):
            self.compile_a_word(self.next_token())
        if self.compilation_state:
            raise ValueError('Unexpected end of input')

    def compile_a_word(self, token):
        found_word = self.find_word_or_literal(token)
        if not self.compilation_state or found_word.immediate:
            found_word(self)
        else:
            self.word_list.append(found_word)

Let’s rename both of these:

    def process_line(self, text):
        new_text = re.sub(r'\(.*?\)', ' ', text)
        self.tokens = new_text.split()
        self.token_index = 0
        while self.token_index < len(self.tokens):
            self.process_token(self.next_token())
        if self.compilation_state:
            raise ValueError('Unexpected end of input')

    def process_token(self, token):
        found_word = self.find_word_or_literal(token)
        if not self.compilation_state or found_word.immediate:
            found_word(self)
        else:
            self.word_list.append(found_word)

Oops, forgot to commit. Do it now: refactoring.

Some more renaming:

    def process_token(self, token):
        definition = self.get_definition(token)
        if not self.compilation_state or definition.immediate:
            definition(self)
        else:
            self.word_list.append(definition)

    def get_definition(self, token):
        definition = self.find_word(token)
        if definition:
            return definition
        num = self.parse_number(token)
        if num is not None:  # might be zero
            literal = self.find_word('*#')
            return Word('', [literal, num])
        else:
            raise SyntaxError(f'Syntax error: "{token}" unrecognized')

Better. Commit: renaming methods.

I notice that compile_literal is no longer used:

    def compile_literal(self, num, word_list):
        word_list.append(self.find_word('*#'))
        word_list.append(num)

Remove it. Commit remove unused method.

I want to try the walrus operator here, see if we like it:

    def get_definition(self, token):
        if (definition := self.find_word(token)):
            return definition
        if (num := self.parse_number(token)) is not None:  # might be zero
            literal = self.find_word('*#')
            return Word('', [literal, num])
        else:
            raise SyntaxError(f'Syntax error: "{token}" unrecognized')

I think I like that. Commit: walrus. Let’s inline one more time:

    def get_definition(self, token):
        if definition := self.find_word(token):
            return definition
        if (num := self.parse_number(token)) is not None:  # might be zero
            return Word('', [(self.find_word('*#')), num])
        else:
            raise SyntaxError(f'Syntax error: "{token}" unrecognized')

Commit: inline.

Would it be more symmetric if parse_number returned the literal word? I think it would.

    def process_token(self, token):
        definition = self.get_definition(token)
        if not self.compilation_state or definition.immediate:
            definition(self)
        else:
            self.word_list.append(definition)

    def get_definition(self, token):
        if definition := self.find_word(token):
            return definition
        elif (num := self.get_literal(token)) is not None:  # might be zero
            return num
        else:
            raise SyntaxError(f'Syntax error: "{token}" unrecognized')

    def get_literal(self, token):
        try:
            return Word('', [self.find_word('*#'), int(token)])
        except ValueError:
            return None

Yes, I am fond of this. Commit: refactoring.

I am not entirely happy with the names of the members. I would prefer that find_word were named get_definition which we have already used. We’ll let that ride for now.

Reflection & Summary

We have smoothed things out substantially. Our main is smaller and better, with no special case for exiting and clearer code for prompting. Our top-level methods are better named and have a better distribution of responsibility. The individual methods are shorter, simpler, less convoluted.

There is still too much rigmarole going on with exceptions and returning results. We raise at one level and then unraise at another. I think we’ll find that we do need to raise internally. Classical Forth would just branch to an abort location, and we do not have that luxury. In upcoming sessions, we’ll look around and see what we like.

That said, we have a few items left on our cards, and we should begin to have more focus on providing capability that our Robot system needs. Like file input, for example, and headless running. All in good time. Things are improving!

See you next time!