The Robot World Repo on GitHub
The Forth Repo on GitHub

I added quite a few words, and found some issues. But we also need some big features. How shall we proceed? Also: Love one another.

Yesterday, while you weren’t watching, I added a number of new words that I picked from the Forth standard CORE definition page. Many of them I defined as secondary words. Our current list of those is this:

    def define_secondaries(self, forth):
        forth.compile(': CONSTANT CREATE , DOES> @ ;')
        forth.compile(': VARIABLE CREATE ;')
        forth.compile(': *DO SWAP >R >R ;')
        forth.compile(': I R@ ;')
        forth.compile(': TUCK SWAP OVER ;')
        forth.compile(': 2OVER 3 PICK 3 PICK ;')
        forth.compile(': 2DROP DROP DROP ;')
        forth.compile(': 2SWAP ROT >R ROT R> ;')
        forth.compile(': /MOD 2DUP MOD ROT ROT / ;')
        forth.compile(': 0= 0 = ;')
        forth.compile(': 0< 0 < ;')
        forth.compile(': 0> 0 > ;')
        forth.compile(': 0<> 0 <> ;')

Along the way, I wrote tests for some of those, and, I admit, I tested some manually. And as I did that, the 2OVER word kept coming up as undefined. It took me a while to realize that I had not yet defined PICK, and that the messages from all those calls to forth.compile get swallowed. They’re returned as strings, because that’s what we want at run time, but here, we cannot be quite so sanguine about errors. I propose to do something about that right now, since swallowed errors here lead to very strange errors later.

Apparently in a rush, I code this:

    def compile_ok(self, text):
        result = self.compile(text)
        assert result == 'ok', result

Let’s have a test for that before we do anything with it.

    def test_compile_ok(self):
        f = Forth()
        with pytest.raises(AssertionError):
            f.compile_ok('1 2 3 3OVER')

Green. Commit: add compile_ok for internal use.

And now I’ll change all those secondaries to use compile_ok. And I change one of the lines to cause it to have an error, and 72 tests fail, so that’s nice. Commit: convert define_secondaries to use compile_ok.

I look for other uses of compile that need fixing up. There are none in Lexicon, plenty in the tests. I think that’ll do for now.

Priorities

Immediate Bug Fix

Let’s talk for a moment about priorities. I went after that defect first thing, essentially without considering features and ideas that might be before us. Why? Because it was a defect in the code, and because it had caused substantial delays yesterday, perhaps as much as ten minutes while I tried to figure out what was wrong with 2OVER. Had I been truly on my game yesterday, I’d have written the test above and left it for this morning to be fixed. We do not have a test for the define_secondaries itself, so the test would have served as a reminder but I’d still have had to remember to change that method. We could write a test for that method, but since it depends on having all the primaries defined, it’s not easy to test on its own.

But the key point is, with a known defect that causes confusion, I automatically treated that as top priority and did the fix. I think that’s a Pretty Good Practice, and if I gave advice, or even sold advice, that would be on the list. But I do not provide advice, I provide experience. Still, maybe think about it.

Comments

In the course of the day yesterday and this morning while trying not to wake up, I became aware of some useful ideas. A user of this Forth, if we ever have one, will want to know what words are defined, and seeing words like PICK and ROLL, will perhaps wonder what they do. If they have the source code, they might be able to figure it out, and in that particular case it would be fairly easy:

    def pick(self):
        index = -1 - self.pop()
        self.push(self.stack[index])

    def roll(self):
        index = -1 - self.pop()
        self.push(self.stack.pop(index))

    # pick ( xu...x1 x0 u -- xu...x1 x0 xu )
    # roll ( xu xu-1 ... x0 u -- xu-1 ... x0 xu )

I put those comments in because I had difficulty seeing the difference myself, and I wrote the comments to guide the code. I predict that even given the comments, you’ll not find it easy to see that PICK copies the u-th stack item to the top and ROLL moves it to the top.

Forth does allow comments in a definition, and we support them:

    def test_comment(self):
        f = Forth()
        f.compile(': TEST ( a b -- b a ) SWAP ;')
        f.compile(': TEST (a b -- b a) SWAP ;')

So you can comment your code in your INCLUDE file, if you care to. But there are no comments in our built-in definitions, and there’s no way to get the comments if there were.

Idea
There would be long-term value to saving a word’s comment and being able to display it on request. There would be value to giving all our words a comment.

We could add a comment field to the Word, default it to empty, fill it in manually in primary words and automatically from the initial comment in secondary words.

Word List

Yesterday I implemented a couple of debugging words, .CWD, which just prints the current working directory, useful for INCLUDE, and .WORDS, which prints a sorted list of all the words:

! % * *# *DO *LOOP + , - . .CWD .WORDS / /MOD 0< 0<> 0= 0> 0BR 1+ 1- 2DROP 2DUP 2OVER 2SWAP : ; < <= <> = > >= >R @ ALLOT AND BEGIN BR BR_TARGET BYE CASE CONSTANT CR CREATE DEPTH DO DOES> DROP DUMP DUP ELSE ENDCASE ENDOF FALSE GET_C_STACK_TOP I IF INCLUDE INVERT LOOP MOD NIP OF OR OVER PICK R> R@ ROLL ROT SQRT SWAP THEN TRUE TUCK UNTIL VARIABLE

In the Terminal version of our Forth, that prints as one big long line. In the game version, the line runs off the end of the output line and there’s no way to get it back. I tried printing each word on a separate line, and that scrolls off the top, with no way to get it back.

Ideas
We should work up some decent way of dealing with long lines of output. A simple hack would be pretty easy.

It would be nice if we could have a real scrolling pane, such as is probably available in tkinter. This may be possible but it would take a few sessions to figure out whether and how. Might be quite difficult.

Integrate Game Graphics

Our program isn’t usable at all until we get some kind of graphical display of the world, presumably the one that now runs in the Python Robot World linked at the top of these articles. There are at least two major possibilities.

  1. We have been thinking that rather than use the Robot World linked at the top of these articles, we would bring over some of that program’s source code and rework it into this program. I’m confident that this can readily be done.

  2. Another possibility has come to mind. We could complete the client-server mode and then drive the resulting server from a Forth app. That would allow us to run Forth in Terminal, ind of its native mode, and to talk to the game, which would run in its own separate window. This is surely also possible, although I do not know some technical material that I’d have to learn.

The second approach has the merit of being consistent with the original objectives of the Robot World project, and results in a server that could be driven by code written in any language. In principle it could be turned into an app or a web app, and might even be extended to allow bots from multiple clients to operate in a single world, interacting with the world and each other. It’s a much more open-ended approach.

The first approach has the advantage of not requiring any substantial learning and experimentation with network communication, and if all we want is a Forth-driven single user Robot World, it is probably faster to do it that way. But not certainly: Robot World is configured to accept JSON now, so if it could be split in a few days, the rest of the work would be about the same, with the resulting product being at least as good for the Forth side, and the open-ended advantages of the network-driven version.

Dammit

I have just about convinced myself that the right thing to do is to push Robot World to the client-server form, and then connect Forth to it via the network and JSON. I’ll have to do some research and beseech my friends for ideas and help but if it can reasonably be done that way, we’ll have a more flexible product, probably without much more effort. If I knew right now how to connect two programs together via messaging, I’d be more confident of that but how hard could it be?

Therefore

To make a final decision about which way to go, I need to consult some experts, and do some experiments with programs talking to programs, to get a sense of how hard it is. I’ll need to estimate how difficult it will be to split the existing Robot World into two parts. We’ve tried to make the two sides independent, and they are probably joined only by JSON over a direct connection, but there might still be a few issues.

I am pretty sure that we’ll choose the client-server approach, #2. It seems like a better design, leaving more options open for the future, and it seems likely that it will not delay our forth-robot connection by much. And quite possibly, future changes will actually be easier. I see no reason why they’d be more difficult.

Meanwhile

We’ll consider small improvements, such as the comment idea, as we go forward. It depends on what I learn over the next day or so of research.

Summary

Sometimes we learn, well into a product’s life, that we want to go a different way. If our code is sufficiently modular, we have options and are not likely to have much rework to do. If it isn’t … not so much. I think that in this effort, we’re in good shape whichever path we choose, and that therefore we’ll choose the client-server path. It will be interesting to see what I do … at least interesting to me!

See you next time! Meanwhile, love one another.