Forth: 2 IF TODAY THEN
With any luck, we’ll do THEN today, which will make IF-THEN work. But first, some feedback led me to a very interesting site.
From the Mailbox
I got a kind email from Rob Kellock of New Zealand, who sent me an interesting and thoughtful essay on his own experience with a Python-based Forth. He pointed to the Python for Fun site, possibly by Chris Meyers, which includes a lovely little implementation of Forth in Python. The source code for this rather complete Forth is less than 180 lines of very compact Python.
My version is 157 lines right now, not counting tests, but it is not nearly as complete as the py4fun one. I haven’t been working toward compactness, but certainly the spirit of Forth includes that notion.
If you enjoy reading Python, you may find the Forth implementation listed above to be interesting. It is very straightforward. Fiddling with it on my iPad last night, I didn’t encounter anything mysterious or excessively clever. Always a good sign.
I always enjoy hearing from readers, via email ronjeffries at acm dot org or on mastodon, see link below. Thanks, Rob!
Our Mission
.., should we choose to accept it, is to make IF-THEN work in our little Forth. Let’s get to that. Afterwards, perhaps, we’ll reflect on where this thing is going.
Yesterday, at the last minute, I couldn’t resist putting in the handler for IF:
class Forth:
def compile_word_list(self, rest):
word_list = []
for word in rest:
if word == 'IF':
word_list.append(self.find_word('*IF'))
word_list.append(0)
elif (definition := self.find_word(word)) is not None:
word_list.append(definition)
elif (num := self.compile_number(word)) is not None:
definition = self.find_word('*#')
word_list.append(definition)
word_list.append(num)
else:
raise SyntaxError(f'Syntax error: "{word}" unrecognized')
return word_list
It’s those three lines beginning with if word == 'IF'. We append the secret word *IF, and a zero, which is intended to be replaced by the number of words to skip forward if the if is not true.
class Forth:
def define_primaries(self):
...
lex.append(PrimaryWord('*IF', lambda f: f.star_if()))
...
def star_if(self):
jump = self.next_word()
flag = self.stack.pop()
if not flag:
self.active_word.skip(jump)
We have tests showing that if the jump value is set correctly, the *IF works.
Our mission this morning will be to handle the THEN. (ELSE will come later.) THEN needs to know what word to replace, and what value to replace it with.
I’ll sketch in some code, but need to think about how to test this directly, if we can.
Wishful Thinking:
def compile_word_list(self, rest):
word_list = []
for word in rest:
if word == 'IF':
word_list.append(self.find_word('*IF'))
word_list.append(0)
elif word == 'THEN':
loc = self.my_position()
patch = self.saved_if_position()
word_list[patch] = loc - patch
...
We are building up the word list in this code. So my_position is the next available position in word_list, that is, the current len. With some renaming as my thoughts begin to come together:
elif word == 'THEN':
my_loc = len(word_list)
patch_loc = self.saved_if_position()
We know that IF can be nested. so we need to stack the position. Forth uses the actual official stack for this. We’ll try that, but I don’t think we’ll want to leave it that way.
def compile_word_list(self, rest):
word_list = []
for word in rest:
if word == 'IF':
word_list.append(self.find_word('*IF'))
self.stack.push(len(word_list))
word_list.append(0)
elif word == 'THEN':
my_loc = len(word_list)
patch_loc = self.stack.pop()
word_list[patch_loc] = my_loc - patch_loc
Is len(word_list) really right? Yes, I think it is. But we need a test for this.
def test_if_true(self):
f = Forth()
s = ': TEST 5 1 IF DUP + THEN 100 + ;'
test_word = f.compile(s)
test_word.do(f)
assert f.stack.pop() == 110
def test_if_false(self):
f = Forth()
s = ': TEST 5 0 IF DUP + THEN 100 + ;'
test_word = f.compile(s)
test_word.do(f)
assert f.stack.pop() == 105
The false case fails:
while self.pc < len(self.words):
w = self.next_word()
> w.do(forth)
E AttributeError: 'int' object has no attribute 'do'
A look at the compiled code tells me that it skipped one too many words. Changing the code to this passes the test:
elif word == 'THEN':
my_loc = len(word_list)
patch_loc = self.stack.pop()
word_list[patch_loc] = my_loc - patch_loc - 1
I’d like to be clear in my mind about why that’s the case.
I’ll draw a little diagram:
LIT 5 LIT 0 *IF _ DUP + THEN LIT 100 +
0 1 2 3 4 5 6 7 8 8 9 10
(The 8 is duplicated because THEN compiles no code.) Since the THEN is not in the list, we clearly want to skip 2 words, DUP and +. so that’s 8 - 5 - 1. I think we’re good.
Let’s recast the code like this:
def compile_word_list(self, rest):
word_list = []
for word in rest:
if word == 'IF':
word_list.append(self.find_word('*IF'))
self.stack.push(len(word_list))
word_list.append(0)
elif word == 'THEN':
last_loc = len(word_list) - 1
patch_loc = self.stack.pop()
word_list[patch_loc] = last_loc - patch_loc
elif (definition := self.find_word(word)) is not None:
word_list.append(definition)
elif (num := self.compile_number(word)) is not None:
definition = self.find_word('*#')
word_list.append(definition)
word_list.append(num)
else:
raise SyntaxError(f'Syntax error: "{word}" unrecognized')
return word_list
Better? I don’t know. But Green. Commit: IF-THEN working. Only one level tested. Should work nested.
Let’s test a nested one just to be sure. This will be tricky to code in Forth.
def test_if_nested(self):
f = Forth()
s = ': TEST 200 100 1 1 IF 5 SWAP IF DUP THEN THEN + ;'
test_word = f.compile(s)
test_word.do(f)
assert f.stack.pop() == 10
assert f.stack == [200, 100]
s = ': TEST 200 100 0 1 IF 5 SWAP IF DUP THEN THEN + ;'
test_word = f.compile(s)
test_word.do(f)
assert f.stack.pop() == 105
s = ': TEST 200 100 0 IF 5 SWAP IF DUP THEN THEN + ;'
test_word = f.compile(s)
test_word.do(f)
assert f.stack.pop() == 300
Those run. I am satisfied that IF-THEN works as intended. I was satisfied before this test, just less frustrated. It was in fact tricky to write.
Commit: added test.
Let’s sum up:
Summary
It’s a tribute to the simplicity of Forth that in just a few lines we can implement nested IF-THEN statements:
if word == 'IF':
word_list.append(self.find_word('*IF'))
self.stack.push(len(word_list))
word_list.append(0)
elif word == 'THEN':
last_loc = len(word_list) - 1
patch_loc = self.stack.pop()
word_list[patch_loc] = last_loc - patch_loc
The IF puts in the *IF word, which does or does not skip the number of words found in the cell after it, and saves that location on the stack. The THEN pops that location, subtracts where we are now, and patches the *IF’s location cell to the right skip value. It doesn’t even have to compile any new code.
Sweet. No credit to YT, all credit to Chuck Moore and the Threaded Code Team, whoever they were.
I noticed a trick in the py4fun version. They saved a sequence on the stack, containing the location, but also a string, probably ‘IF’. Then the THEN can check the pair to be sure that the stack isn’t corrupted. Shall we do that, just for safety?
if word == 'IF':
word_list.append(self.find_word('*IF'))
self.stack.push( ('IF', len(word_list)))
word_list.append(0)
elif word == 'THEN':
last_loc = len(word_list) - 1
key, patch_loc = self.stack.pop()
if key != 'IF':
raise SyntaxError(f'THEN did not find matching if, found: "{key}"')
word_list[patch_loc] = last_loc - patch_loc
I’ll change the saved value to see the error.
elif word == 'THEN':
last_loc = len(word_list) - 1
key, patch_loc = self.stack.pop()
if key != 'IF':
> raise SyntaxError(f'THEN did not find matching if, found: "{key}"')
E SyntaxError: THEN did not find matching if, found: "IFF"
So that’s nice. Commit: then looks for matching IF key
It seems clear that we can use this same approach for ELSE and whatever other looping kinds of statements we choose to adopt from Forth.
I am pleased. Let’s look into the future a bit.
Future
The original purpose of this Forth thing was to provide a little language for controlling robots in the robot world game. The idea is to send a package of Forth code over and let the World run it. I suppose we might still do that, just to see what happens.
I do think that debugging Forth inside a remote server will be nearly impossible. I can do it at home, but if we imagine—and I do mean imagine—real users, they’d submit their code, get some arcane syntax error, and little more information. If this were a real project, we’d surely want to provide better information, and, what might be nice would be some way to test the code in a local application. Maybe something that operates a fake bot locally or something.
If you look at the py4fun Forth, you’ll see that it executes the code as it consumes it, which works just fine. Our version currently compiles everything to our word lists. There’s no real provision for a naked Forth statement such as you’d type into a REPL. Maybe we should build a REPL.
I find that I’m enjoying this Forth diversion a lot. I’m tempted to take it forward kind of on its own, to see if it might be useful in some way. Rob Kellock spoke of using his Forth to control lighting, for example. I have an app for that, but maybe there’s something fun I could do.
Finally, it might be fun to see how compact I can make this version, and it might be fun to try to write a truly compact one, with less concern for objects and use of globals and such.
I look forward to finding out what I do. I hope you’ll join me!