Python Wordle on GitHub

I’m having trouble coming up with a real plan here. And I think I’ve squeezed everything I can out of score. I borrow an idea from GeePaw Hill.

After we parted yesterday, I spent Even More Time trying, without much success, to further optimize score calculation, on the grounds that we’ll be doing a lot of it when we generate whatever we generate whenever we finally get around to working on a solver for Wordle. I feel confident in that prediction, because I think we’ll have to score our 12,972 total words against the 2,315 solution words, for a total of over 30 million score calculations.

Well, except that I already know that I can do about a million scores per second, so we’re talking 30 seconds to do them all. I got excited about the number when, for a reason I cannot recall, I was thinking that I would need to calculate another 2315 combinations, which would take me a day, but I don’t see that I will ever have to do that.

For some reason, my thinking about this problem is fuzzy. I don’t have a clear picture of where I’m going, nor a clear picture of useful next steps to figure out where I’m going. If you’ve troubled yourself to read the previous three articles, you’ll surely have noticed that. Maybe you had ideas that you wanted to shout at me, and I rather wish you had pinged me on mastodon to let me know.

My friend and nemesis GeePawHill has been playing with Wordle, since Ken Pugh contaminated him as well as me. Hill has built an interesting structure that I’ll try to describe here.

For each of the 12,000 words we know, Hill calculates the score against each of the 2000 solutions. There are only 243 possible scores (3^5), so for each word he stores a dictionary from score to a list of the solutions that get that score.

Now he is using that information to play the game, which is of course unfair, but it was already unfair for any of us to do anything at all with the solutions list. And as Hill put it, a game that thinks some of these words are valid has no right calling fair or foul.:

Ron Jeffries
Hardly fair tho.
GeePaw Hill
Fair? From a guy saying REAST is a valid word?

I think that I’ll wind up going a different way with this, following Grant Sanderson’s information theory approach, but today, I want to replicate Hill’s structure, just for the exercise.

I’m pretty sure that Hill has some specialized objects for doing this job. He scores the guesses with a string of E, N, and Y, and calls that score an ENY and has an ENY map and such.

Hill is better than I am in many ways (and worse in others, I fondly hope) and one of his betters is that he is much quicker to move to a specialized collection than I am. What I’m going to do this morning is spike a solution in a test, and then see what it suggests.

Here’s my test:

    def test_simple(self):
        all_guesses = WordCollection.from_file("valid_guesses.txt")
        all_solutions = WordCollection.from_file("valid_solutions.txt")
        guesses = all_guesses.words[0:10000:500]
        assert len(guesses) <= 20
        solutions = all_solutions.words[0:2000:100]
        assert len(solutions) <= 20

        solutions_dict = {}  # guess -> dict (score -> [solutions])
        for guess in guesses:
            guess_dict = {}  # score -> [solutions]
            solutions_dict[guess] = guess_dict
            for solution in solutions:
                score = guess.score(solution)
                if not score in guess_dict:
                    guess_dict[score] = []
                guess_dict[score].append(solution)

        print()
        for solution in solutions_dict:
            print(solution)
            for score in solutions_dict[solution]:
                print(f"    {score:05d}")
                answers = solutions_dict[solution][score]
                for answer in answers:
                    print("        ", answer)
        assert False

I create two lists of Words with 20ish entries each, one of guesses and one of solutions. (Now that I think of it, we should use the combined list, not the guesses list.) I sliced the lists down with big steps so as to get an interesting mix of words.

The middle block of the test just creates the desired dictionary, and the final block prints it is a somewhat pretty format.

The result looks, in part, like this:

...
Word(bocca)
    10021
         Word(aback)
    00001
         Word(aptly)
         Word(false)
         Word(frail)
         Word(grasp)
         Word(human)
         Word(rival)
         Word(shady)
         Word(stair)
    20020
         Word(birch)
    20000
         Word(bulky)
    00101
         Word(clash)
    10110
         Word(cubic)
    01000
         Word(drone)
         Word(pesto)
    01001
         Word(lasso)
    00021
         Word(march)
         Word(smack)
    00200
         Word(niche)
    00000
         Word(purse)
Word(certs)
    10000
         Word(aback)
    00010
         Word(aptly)
...

And it goes on for almost 650 lines.

What does this structure tell us? Well, for example, if you guessed the word “bocca” and the score comes back as 10021, or YNNEY, the solution is “aback”. (Remember, we only have 20 words in our solution list: aback, aptly, birch, bulky, clash, cubic, drone, false, frail, grasp, human, lasso, march, niche, pesto, purse, rival, shady, smack, and stair.)

It’s easy to see how we could use this information to play the game, if we could remember these words lists or had a computer by our side, which, if we are playing Wordle, we certainly have.

We’d guess a word, get the score, look in the dictionary for that guess and that score, and voila, there are the only possible answers. If we guess another word, we’ll get another list and the answer has to be in both lists. It’s likely that a few passes of this would give us a good result.

I think maybe we should get the full dictionary and simulate a few games, just to see how well it does. But it would be “cheating” to use the known solutions, wouldn’t it? Yes, but we’re just programming here, not playing Wordle for money. (And if we were, I would definitely not cheat. You can trust me on that. Really.)

I think we should create some kind of object here, that, given two lists of words, will create an instance of itself with a few useful methods, which we will devise as we write our next test.

    def test_solution_dictionary(self):
        all_guesses = WordCollection.from_file("valid_combined.txt")
        all_solutions = WordCollection.from_file("valid_solutions.txt")
        guesses = all_guesses.words[0:10000:500]
        assert len(guesses) <= 20
        solutions = all_solutions.words[0:2000:100]
        assert len(solutions) <= 20
        solution_dict = SolutionDictionary(guesses, solutions)

I go a bit beyond my remit and code:

class SolutionDictionary:
    def __init__(self, guesses, solutions):
        self.dict = self.create_dict(guesses, solutions)

    def create_dict(self, guesses, solutions):
        return {}

What I am communicating here is that the object is going to create its dictionary right from the get-go.

If we were going to use this to help us in the game, we’d choose a word to guess, get a score, and ask the SolutionDictionary what words it has for that guess and score. So let’s test:

    def test_solution_dictionary(self):
        all_guesses = WordCollection.from_file("valid_combined.txt")
        all_solutions = WordCollection.from_file("valid_solutions.txt")
        guesses = all_guesses.words[0:10000:500]
        assert len(guesses) <= 20
        assert guesses[10].word == "herns"
        solutions = all_solutions.words[0:2000:100]
        assert len(solutions) <= 20
        assert solutions[10].word == "human"
        solution_dict = SolutionDictionary(guesses, solutions)
        guess = Word("herns")
        solution = Word("human")
        score = guess.score(solution)
        solutions = solution_dict.solutions_for(guess, score)

I propose a new method solutions_for, which will return the list of possible solutions given whatever score we get for herns against human, which I think will be 20010. Shall we assert that? We shall.

We need solutions_for. We can code it readily:

    def solutions_for(self, word, score):
        return self.dict[word][score]

It occurs to me that we will need to bulletproof this object against words or scores that do not work. Let’s do that now:

    def solutions_for(self, word, score):
        try:
            return self.dict[word][score]
        except KeyError:
            return []

Our test is passing. We need to make it harder. Let’s just do this for now:

    def test_solution_dictionary(self):
        all_guesses = WordCollection.from_file("valid_combined.txt")
        all_solutions = WordCollection.from_file("valid_solutions.txt")
        guesses = all_guesses.words[0:10000:500]
        assert len(guesses) <= 20
        assert guesses[10].word == "herns"
        solutions = all_solutions.words[0:2000:100]
        assert len(solutions) <= 20
        assert solutions[10].word == "human"
        solution_dict = SolutionDictionary(guesses, solutions)
        guess = Word("herns")
        solution = Word("human")
        score = guess.score(solution)
        assert score == 20010
        solutions = solution_dict.solutions_for(guess, score)
        assert solutions

That fails with solutions empty, as we expected. Now to do the actual work.

    def create_dict(self, guesses, solutions):
        solutions_dict = {}  # guess -> dict (score -> [solutions])
        for guess in guesses:
            guess_dict = {}  # score -> [solutions]
            solutions_dict[guess] = guess_dict
            for solution in solutions:
                score = guess.score(solution)
                if not score in guess_dict:
                    guess_dict[score] = []
                guess_dict[score].append(solution)
        return solutions_dict

I just extracted the working code from the spike test into the method. Now let’s enhance the test based on what actually came back.

Disappointingly, the only word that comes back for that score is “human”. Let me dump the dictionary and pick a more interesting result. Here’s the new passing test:

    def test_solution_dictionary(self):
        all_guesses = WordCollection.from_file("valid_combined.txt")
        all_solutions = WordCollection.from_file("valid_solutions.txt")
        guesses = all_guesses.words[0:10000:500]
        assert len(guesses) <= 20
        assert Word("berth") in guesses
        solutions = all_solutions.words[0:2000:100]
        assert len(solutions) <= 20
        assert Word("frail") in solutions
        solution_dict = SolutionDictionary(guesses, solutions)
        guess = Word("berth")
        solution = Word("frail")
        score = guess.score(solution)
        assert score == 100
        solutions = solution_dict.solutions_for(guess, score)
        assert solutions == [Word("frail"), Word("grasp"), Word("rival")]

And the class that passes:

class SolutionDictionary:
    def __init__(self, guesses, solutions):
        self.dict = self.create_dict(guesses, solutions)

    def create_dict(self, guesses, solutions):
        solutions_dict = {}  # guess -> dict (score -> [solutions])
        for guess in guesses:
            guess_dict = {}  # score -> [solutions]
            solutions_dict[guess] = guess_dict
            for solution in solutions:
                score = guess.score(solution)
                if not score in guess_dict:
                    guess_dict[score] = []
                guess_dict[score].append(solution)
        return solutions_dict

    def solutions_for(self, word, score):
        try:
            return self.dict[word][score]
        except KeyError:
            return []

Move the class to its own file: I was developing it in the test file, which PyCharm supports nicely. And commit: Test and implement SolutionDictionary {solution: {score: [solutions]}}.

Now there’s just one more thing to do: create the full dictionary, see how long it takes and whether Python explodes.

I am guessing it’ll be something similar to 30 seconds, at least of that order.

    def test_full_timing(self):
        all_guesses = WordCollection.from_file("valid_combined.txt")
        all_solutions = WordCollection.from_file("valid_solutions.txt")
        t0 = time.time()
        dict = SolutionDictionary(all_guesses, all_solutions)
        delta_time = time.time() - t0
        assert delta_time == 0

Expected :0
Actual   :29.312423944473267

30 seconds. Life is good. I’ll patch a skip onto that test, 30 seconds is too long for rapid testing.

Reflection / Summary

OK, I feel slightly less ignorant and dull, because I at least understand Hill’s idea and was able to replicate it. That’s not as good as having an idea of my own, but I’m not sure that I’ve ever had a really good idea of my own: I seem always to get those from my betters. Bad ideas, on the other hand … I have in my time been excellent at that.

A somewhat interesting question might be whether this nested dictionary structure should have more of my own objects in it. At this moment, my answer is that I don’t see the need. Perhaps I’ll ask my betters in next Tuesday’s Zoom. And you can toot me with comments if you like.

What’s next? Well, it might be interesting to gather some statistics from the solutions dictionary, such as the max, min and average number of words in a guess->score->list. I’m not sure whether it would be useful. Hill has started creating statistics on the actual words, how many start with vowels, how many end with “s” and the like. Might be amusing.

I think what I would like to do is to understand Grant Sanderson’s video and create his information statistics. I’ll try to find the time to study that further and we might do that.

So. A decent Sunday morning start before breakfast. I still feel a bit dull, but got something done fairly smoothly.

I hope you’ll join me next time. Until then, what the heck is “herns”? A hern is a bitron, of course. You and I might know it as a heron, or a bittern. Where do they get these words?