For reasons, I decided to write a little web app with Google App Engine. The details don’t really matter: the thing will store some records with fields, and be able to retrieve a subset based on a selection, in a couple of different ways. Very standard thing, nothing to see here, move along.

Months and months ago, I installed the App Engine SDK and Launcher and such, and had written a few little sketches. It turns out that for some reason, the Launcher will no longer update on OSX 10.whatever, so I uninstalled the App Engine stuff and reinstalled it. On my iMac, that went OK and more or less as I expected. On the MacBook Air, well, thereby hangs this tale.

##Wednesday, Bloody Wednesday

For some reason, when I went to download the App Engine stuff on the Air, Google took me to its new “Cloud Engine”, for which I guess I must have signed up at some point. That path took me to a download that wasn’t based on the Launcher, but instead on command line tools. I had already noticed that I needed to see the Python error messages, so that was more or less OK with me, and that’s the way I went. Wednesday morning went that way.

Then I built the clock example from Dan Sanderson’s book, and made the first bit of it work. I didn’t want to push that example very far: it just looked like a good place to stand to build what I actually want, since I had other examples using the Model lass and such. But what I did need, I knew, was testing. So the rest of Wednesday’s session was taken up with trying to figure out how to run a testing framework against App Engine, locally. Turns out there’s a page for that here. So I copied that test down from GitHub and the nightmare began. It didn’t run!

It didn’t run because it couldn’t find some library or other. Something led me to believe I needed to set my PYTHONPATH to point to the new App Engine libraries. In the Launcher-based installation, these are all hidden and one need not know about them. In the command-line installation, one needs to know more. One of the things one needs to know is that the SDK libraries weren’t even downloaded with the initial stuff, or at least appeared not to be. So I brought those down again. I fiddled with the path and stuff but no joy: I could not get that sample test to run. Well, tomorrow would be another day (Thursday, relative to that day which was day before yesterday as I write this, and Wednesday in a more general notation.) And tomorrow (now yesterday) Tozier would be at the coffee shop and I knew he’d love to help.

##Fast forward to yesterday …

Thursday morning, at home, I browsed the GitHub files where that datastore_test.py file was and found a very simple hello world example, which had a test that looks like this:

import main
import webtest


def test_get():
    app = webtest.TestApp(main.app)

    response = app.get('/')

    assert response.status_int == 200
    assert response.body == 'Hello, World!'

My plan was to start with this much simpler test, because simplicity is the only place where I can live comfortably.1

So that’s where Tozier and I started, trying to run that little test. The ads seemed to say that you run it with python main_test.py, and when we did we got messages relating to not finding webtest.

A lot of Googling and file searching ensued, and we found webtest on my computer at the obvious location: /Users/ron/google-cloud-sdk/platform/google_appengine/lib/cherrypy/cherrypy/test/webtest.py, right where you’d think to look. So with a bit of munching about, we wound up with an updated PYTHONPATH:

:/Users/ron/google-cloud-sdk/platform/google_appengine/lib/webob-1.2.3:/Users/ron/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.2

By a bit of munching around, I mean the following, more or less accurate list:

  • re-download App Engine SDK just to be sure;
  • Try to use PIP to install some library that showed up missing;
  • Discover PIP was not present;
  • Decide to use HomeBrew to install it;
  • Decide to update Brew;
  • Discover Brew was broken;
  • Try to update Brew;
  • Discover it wouldn’t update;
  • Try to install Brew;
  • Brew declined to install because it could clearly see it was already installed;
  • Uninstall Brew more or less without incident;
  • Install Brew;
  • Install PIP;
  • Find interesting caveat in the main PIP install page saying that it may not work with your existing OS-installed Python;
  • Work around the caveat as described elsewhere on the internet;
  • Use PIP to fetch whatever it was we wanted;
  • Test still would not run;
  • After messing around confused for ages, try to run the App Engine example which used to run;
  • Previously working tiny app died screaming with pages of Python output.

At this point we basically undid everything, with only a little difficulty, using arcane techniques that Tozier understood (or at least knew the incantations for) and after not too long we were back where we started, with the app running and the test not running.

Some further very simple tweak, which I no longer remember, and the test ran … and did nothing. That’s not too surprising, I guess, because there’s no

if __name__ == '__main__':

which, I believe, means that no one calls the testrunner.

You’re supposed to know that!

When you’re following a recipe, not cooking out of your own knowledge, if the recipe doesn’t say to add salt, you’re not going to add salt, even if you “should” know that salt should be added. We were following a recipe, and the twisty little paths all different that it led to, and there’s nowhere in the little example test that we copied right off of GitHub that tells us how to run the tests. There is a line in the datastore_test, but it was already off our radar when we went for the simpler test. So, yeah, I was supposed to know that, and I didn’t.

And then …

The little sample test didn’t run, and we couldn’t find an internet clue on what the incantation is that goes under the check for __main__. My guess today is that you can use the unittest runner, but yesterday that idea was not in our heads.

So by now Tozier was late picking up his wife, and took a break before hitting the road. I wrote this program:

import urllib2
response = urllib2.urlopen('https://localhost:8080/')
html = response.read()
print html

I started the main app running as usual with the usual dev_appserver.py command, then ran the test above with python. It prints the output from the clock app:

<p>The time is presently: 2016-06-24 14:45:34.988095</p>

Voila! In two minutes (ignoring the four hours before) I had written a program that calls the App Engine code and reads out what it says. Clearly, from here, I can send various inputs to the App Engine, and check them out. If I can get a unit testing framework to load, I can use it. If not, I can write my own.

I can test, at last. Three days in, with a four-line test base finally working. It’s all good!

It’s all good … NOT!

So this morning,2 I thought I’d write this article about my experience, to try to draw some learning from it. I created the folder and the file, typed in the YAML and the first paragraph, and started Jekyll to build the html as usual.

AND IT BROKE

Air:rj-com ron$ jekyll serve
/Users/ron/.rvm/rubies/ruby-2.2.0/lib/ruby/2.2.0/yaml.rb:4:in `<top (required)>':
It seems your ruby installation is missing psych (for YAML output).
To eliminate this warning, please install libyaml and reinstall your ruby.
...

Arrgh, I cried, or words to that effect, into Tozier’s DM. I decided we could probably fix it when next he was with me, next week, and went ahead writing. However, Tozier heard me. He got in touch, saying that he had felt a great disturbance in the force, as if millions of voices had cried out in terror and were suddenly silenced. From wherever he is today, he connected to my Mac and we

  • Checked RVM to be sure Ruby was OK. He ignored this message[^msg].
  • Ran gem list, which gave the same message about psych.
  • Ran gem install psych, which gave the same message.
  • Ran gem install libyaml ditto.
  • Ran brew install libyaml’\, which worked.

At that point we tried running Jekyll again and it ran. We backed slowly away from the computer. I’m not dead certain, but I think we’re back to where I was sometime Tuesday.

##Is there a lesson in this?

Well, if I were in the business of programming Google App Engine, a few days’ effort up front to get it going might seem not to be a big deal. However, when I visit a team who have just added a new person, I invariably find that person struggling to set up their workstation. There’s usually someone in the room who, like Tozier, kind of knows how to fumble through the various rvm, brew, pip items. A little reading of stackoverflow and you’re good to go. Until something has to be updated or it breaks. With luck, it’s only person-days lost out of person-months. With luck, no one breaks a blood vessel.

Me, I just want to write programs, mostly small, that do things that may be worth writing about. And I want to write articles like this one, convert them to HTML, add them to a few indexes, and put them on my site.

The project I have in mind for Google App Engine will only take a few days. A few days of yak shaving is more obviously a few days wasted than if I were slaving away in a sweatshop somewhere.

But Wednesday I shaved a few yaks. Thursday Tozier and I shaved a few more and then had to stick all their hair back on when it made things worse. Then today, we shaved a couple more.

It seems to be yaks all the way down. My computer is full of yaks.

I guess all that unproductive hackery is the price we pay for productivity. But it sure doesn’t feel like productivity this week.

Still, I do have that four-line test running now … and probably I have a friend somewhere who’ll message me and tell me what to try next.

Stand back, yaks! I’m after you!

  1. OK, I admit, my BMW might not be considered the pinnacle of the simple life. But anyway I do like simple software. ↩

  2. Friday. Are you not paying attention at all? ↩