Further Forth
The Robot World Repo on GitHub
The Forth Repo on GitHub
I have at least two more things to do before this thing is even good for its initial purpose. And I have what I think might be an interesting idea. Kind of a ragged morning.
Interesting Idea
I’ll write this down first, to get it on “paper” and out of my mind. The original plan for the robot world thing was that it would be a world server and various users would script their bots ad do client calls asking the world to do its few primitive operations on the bots. Then I had this Forth idea, where users could write Forth scripts for their bots and send them to the server.
But there never will be a server. The staffing for this project collapsed or never got off the ground, and it was just one of those ideas that people have that never come to anything.
So the new idea is … rip out the client-server stuff and create a robot world program that runs on “your machine”, and you program your bots in Forth. So it’s a game you can play on your local machine,
This, too, will never happen, but it is interesting enough that I’ll probably do it.
But First …
The current Forth needs more words in its base definition, but that is almost by the way. It also needs better error-handling, but only a bit better. There are rules in the standard for what you do when there’s an error, and it basically comes down to clearing some stacks so that you’re at a known point. And we need the ability to read definitions from a file, and probably to write them out, so that you don’t have to type your whole program in every time you start forth and every time you change the program.
A related fact comes to mind. In standard Forth, if I am not mistaken, when you redefine a word, words that use the old definition of that word continue to reference the old one. We’ll need to deal with that issue one way or another. If I’m not mistaken, and now that I’ve nested “not mistaken” the odds are not good, our existing code has that property.
But even more first … I’ve been trying to rig up my little main program so that when you’re at the Forth prompt, and you type an up arrow, the previous line you typed appears on the Forth> line, just like it does when you’re running the Terminal. All the incantations I’ve been able to find for this do not work. At best, the cursor just travels up the screen.
Belay that! What I have does not work in the PyCharm Run window but it does work in the Terminal window. Good enough. Forget that I mentioned it.
Current main is:
from source.forth import Forth
import readline
if __name__ == '__main__':
forth = Forth()
prompt = 'Forth> '
while True:
line = input(prompt)
if line == 'bye':
break
try:
forth.compile(line)
print('ok')
prompt = 'Forth> '
except Exception as e:
if str(e) == 'Unexpected end of input':
prompt = '...> '
else:
print(e, ' ?')
Apparently just importing ‘readline’ does the job.
OK, let’s move on to error handling after that odd interruption. The life of the programmer involves dealing with interruptions all the time, often of our own making. But now I have the thing I needed, the up arrow capability. We’re very likely to type in Forth lines incorrectly: being able to recall them and do better is a big help.
For error handling, the Forth standard includes an actual word, ‘ABORT’, and also ‘ABORT”’. The latter prints a message first. We’ll have a method on our Forth, and I hope everyone will forgive me if I call it ABEND, for ABnormal END.
Ah, hell. I started coding this. It’s so easy. But we know where that leads. Let’s at least test it a little bit.
And, as always even writing the test clarifies the situation:
def test_abend(self):
f = Forth()
with pytest.raises(ZeroDivisionError):
f. compile('1 2 3 0 /')
We do get this exception … but we haven’t called abend. I had been planning to call it in the main:
try:
forth.compile(line)
print('ok')
prompt = 'Forth> '
except Exception as e:
if str(e) == 'Unexpected end of input':
prompt = '...> '
else:
forth.abend()
print(e, ' ?')
I think this won’t quite do. It’ll work if the main is the only way we ever run Forth, but not otherwise. What would be better, I think, would be to have another method in Forth class, perhaps safe_compile, that catches exceptions and returns some kind of status information. Should that method do any printing? I think not: it should be able to work for a headless Forth app that might not tolerate random printing.
OK, having thought that thought, what we probably want is for our everyday compile method to behave along those lines … No! That is too much change in an already complicated method. We’ll posit a new safe_compile, do that, work from there. I’ll write some tests for that.
def test_safe_compile(self):
f = Forth()
result = f.safe_compile('1 2 3 2 /')
assert result == "ok"
class Forth:
def safe_compile(self, text):
self.compile(text)
return "ok"
So far so good. Now an error:
def test_safe_compile(self):
f = Forth()
result = f.safe_compile('1 2 3 2 /')
assert result == "ok"
result = f.safe_compile('1 2 3 0 /')
assert result == 'integer division or modulo by zero'
My plan is just to return the string of the error, with a special case we’ll discover shortly.
def safe_compile(self, text):
try:
self.compile(text)
except Exception as e:
return str(e)
return 'ok'
Green. Do I feel good enough about this to commit? Not yet, but I think we’re doing fine. I want to deal with the need for new input.
def test_safe_compile_needs_more_input(self):
f = Forth()
result = f.safe_compile(': FOO 42 ') # no semicolon
assert result == '...'
We’ll have two “expected” returns, “ok” and “…”. Any other is an actual error.
def safe_compile(self, text):
try:
self.compile(text)
except Exception as e:
msg = str(e)
if msg == 'Unexpected end of input':
return '...'
else:
return str(e)
return 'ok'
Now, before we do the abend, let’s change and test main.
if __name__ == '__main__':
forth = Forth()
prompt = 'Forth> '
while True:
line = input(prompt)
prompt = 'Forth> '
if line == 'bye':
break
result = forth.safe_compile(line)
if result == '...':
prompt = '...> '
else:
print(result)
I see one thing not handled, the question mark, but we’ll sort that next. Here’s my result in terminal. More issues than I realized:
Forth> 2 2 + .
4 ok
Forth> : foo 123
...> ;
ok
Forth> foo .
pop from empty list
Forth> 1 0 /
integer division or modulo by zero
Forth>
I expected the ‘foo’ word to be completed correctly, but apparently it was not.
And there is the issue of the expected question mark. I’ll set that aside. What went wrong with the definition of foo?
It had some definition but apparently did not include the literal.
Ah. That bug is deeper than I thought. Long story short, we are clearing the word list in compile_a_word on the assumption that we’ll always be able to get a token. But in the case of a definition that spans more than one line, we need to preserve that list.
The fix turns out to be easy:
def compile_a_word(self):
if self.compile_stack.is_empty():
self.word_list = []
while True:
token = self.next_token()
if token is None:
raise ValueError('Unexpected end of input')
if (definition := self.find_word(token)) is not None:
if definition.immediate:
definition(self)
else:
self.word_list.append(definition)
elif (num := self.parse_number(token)) is not None:
self.compile_literal(num, self.word_list)
else:
raise SyntaxError(f'Syntax error: "{token}" unrecognized')
if self.compile_stack.is_empty():
break
return Word('nameless', self.word_list)
If the compile stack is not empty, the only way we can re-enter the compile_a_word is if we have exited it, which can only happen on an exception. So the if at the beginning works for the simple case of a definition over more than one line. I think the rest will sort out on abend.
Whoa whoa whoa (also woe)!
We’re going too fast and loose here. That change to when we clear the word list is kind of a big deal, and while I am pretty sure it’s righteous for the multi-line colon definition (I tried one), I’m not so sure it’s right overall.
I think that compile_a_word method with all its if statements and such is rather questionable. It’s not at all easy to be sure it works, with two exceptions and a break statement, plus four separate if blocks. Very questionable.
That said, all our tests are running. Let me make one tiny change:
def safe_compile(self, text):
try:
self.compile(text)
except Exception as e:
msg = str(e)
if msg == 'Unexpected end of input':
return '...'
else:
return f'{e} ?'
return 'ok'
We’ll return the exception message followed by a space and a question mark, so that our Forth output looks more like what we want:
Forth> 2 2 + .
4 ok
Forth> 3 0 /
integer division or modulo by zero ?
Forth> .
pop from empty list ?
Forth> 1 2 3 0 /
integer division or modulo by zero ?
Forth> .
2 ok
OK, I want those question marks for historical reasons. Note from the last ‘.’’ that there is still stuff left on the stack. That’s the job of our abend, so let’s get back to the test.
def test_safe_compile(self):
f = Forth()
result = f.safe_compile('1 2 3 2 /')
assert result == "ok"
result = f.safe_compile('1 2 3 0 /')
assert result == 'integer division or modulo by zero ?'
assert f.stack.is_empty()
That passes because of this:
def safe_compile(self, text):
try:
self.compile(text)
except Exception as e:
msg = str(e)
if msg == 'Unexpected end of input':
return '...'
else:
self.abend()
return f'{e} ?'
return 'ok'
def abend(self):
self.stack.clear()
self.compile_stack.clear()
self.return_stack.clear()
self.active_words = []
I’m going to commit this. It’s passing all the tests and works in the terminal. Despite my uneasiness, it’s working as we have so far intended. Commit: added safe_compile, abend, and used safe_compile in main.
I’ve made a note that compile_a_word is ragged. Let’s do a couple of renames, renaming compile to unsafe_compile and safe_compile to compile.
Green: commit: rename.
Summary
The day is ragged. My mind is ragged, some small personal stuff going on. The code is ragged, dealing with all the various conditions that can arise. I think we’re closer to reasonable but I’m not entirely satisfied with the clarity. We’ll look at things repeatedly and see whether there is a better partitioning of behavior that will improve at least the clarity and, ideally, the actual design.
But it’s working like a real Forth. And that is a good thing.
See you next time! We’ll try to do better … as always.