Triggering the iPad Build

I thought today we’d work on the triggering mechanism for the iPad build. Instead something important happened!

When Bill gets here, we’ll review where we are. My vague recollection from before Agile2017 is that we think the core functionality is solid: copying in files, building the site, and FTPing up the necessary results. I still have a bit of fear about turning the buzz-saw loose on my actual site, lest we copy all kinds of weird files to the wrong places and be unable to clean it up readily.

We’ve done nothing on watching the Dropbox trigger file, running the buzz-saw, and then not triggering it again until the trigger file is updated again. I’m inclined, prior to his consultation, to work on that. For now, I’m working on an article about the state of Agile, called Business Agile: Built Upon Sand.

Triggering

How might we trigger the build? We don’t want to trigger it every time a new article shows up in Dropbox: the author might not be ready. We’re thinking there is a “special” file that says to go, and somehow, after we go, we make it so that that file has to be updated again before we go again.

Bill asks whether we could add an entry to our passwords file, saying whether to render or not. Author sets it “true”, building sets it “false”. We think not, because the passwords file isn’t in Dropbox. But the same idea works with a file in the Dropbox.

I had been thinking about using some dated files or lines. Maybe a file that says build-as-of-time-t1 and built-as-of-time-t2 and if t2 >= t1 don’t build. But maybe just toggling the contents of a single file is better.

Bill suggests that maybe I should get over myself and let the thing just unconditionally run at some time of day and put everything up. He admits he couldn’t get over himself so I am not strongly moved by this argument. The idea would, however, be an interesting smaller story.

We discuss this further. We realize that if our current program were running as a cron job, it would every now and again generate my site and push everything referenced in Dropbox up to my site. If, therefore, I only ever wrote articles into Dropbox, this one little program could do all my publication.

One issue is that I manually commit my site to git and GitHub by way of an archive. We could “easily” add that ability to our little program, namely just committing every time the cron ran. That might be a bit much but we could. Hm it only commits deltas but they’d be small or empty. There would be the issue of what git does with an empty delta. We’ll try it … we find, as you probably already knew, that a git commit with nothing to change just does nothing (we didn’t check the return code), and similarly for git push. So for current purposes, if our cron job did git commits and pushes it would be harmless. The log doesn’t even show the empty commits.

This suggests that we could redefine our product a bit, and change the author process a bit (save files to Dropbox, use the private category for drafts), and use our little program for both iPad and Mac development, without the need for the file watching feature at all.

Important (to me) Digression

This is what should happen in a “real” Agile product development. We encounter a slightly sticky problem, we talk about it, and we suddenly see a different course of action, a real product redefinition, that would let us ship earlier, and would also let us serve a broader “market” of users with a single solution.

This idea wasn’t in the real plan at all.

On the contrary¹, Tozier, channeling straw person objectors, raises the spectre that we can’t just change our minds, we’ll lose all that important work and thinking that has gone before. I answer that¹ we kind of had to think anyway, and that in fact there’s nothing in our program that has to change. Yes, I did learn a bit about fswatch a month or so ago, but we didn’t build anything on it, not even much in the way of testing.

The product is both more broadly useful, and easier to do, than we saw at the beginning … or even an hour ago.

This is important. Think about it. We’ll talk about this again soon.

End Digression back to the plan

So now what? What if we were to take this path: set our current programming running on cron, and have it be the only process I use to run Jekyll. If I then wrote my Mac-based articles into Dropbox, they’d get published sooner or later.² Articles put into Dropbox from the iPad would get published. We’d want to add git actions to the current program as another story. We might have other stories come to mind.

But we’d be done and could ship this as soon as we’re confident that the buzz-saw won’t destroy the site.

Unfortunately we’re still thinking. Tozier channels some other irritating person who asks why we don’t do this whole thing with Jekyll’s post-processing hooks, which is probably possible. Ron finds another whining voice who points out that if we’d just upgrade our Jekyll we’d have incremental build. We remember Chet, who we are careful to say is not with us, telling us we could do it with Jenkins.

Yeah, we could. Sunk cost stands in our way, though it shouldn’t. Ignorance of how to do these things and the new cost of taking a new approach, however, should stand in the way, and it does. We can ship this version pretty quickly now, at very low cost, and even if there is “technical debt”³ or fake technical debt⁴ that can be resolved with a whole ‘nother solution, that would come with additional investment. When our next story comes along we can look at whether an entirely different approach is a good investment.

I think we’ll go ahead with the cron idea. However, our allotted working period of a couple of hours has almost elapsed. We do have a little time left for exploration. We could experiment with building a little cron job to run our script, which, we imagine, will do what we need. Maybe we can build one that runs in my local folders by tweaking what’s in the password files or something.

After we talk about it a bit, we decide to create a new “jekyll-cron” main program and put it in cron. We can build that program incrementally, first to do nothing but say hello, then to build a test version of JekyllRunner and run it, then finally building the real one.

Here goes:

# jekyll-cron.rb

puts "JekyllCron running at #{Time.now}"

This works in Sublime. Now we try it in Terminal.

Air:test-ftp ron$ ruby jekyll-cron.rb 
JekyllCron running at 2017-08-15 11:03:39 -0400
Air:test-ftp ron$ 

Now our next step is to set up a cron job and Tozier wisely suggests we learn how to turn them off as well as on. I think one just edits a file. Time for Google.

Quite a bit of yak-shaving later, trying to get Sublime to be my editor of choice for crontab, we use nano, with this line:

*/1 * * * * cd ~/programming/test-ftp/ && ruby jekyll-cron.rb >> crondone.txt

And after a minute or so, sure enough the crondone.txt file has stuff in it:

JekyllCron running at 2017-08-15 11:26:04 -0400
JekyllCron running at 2017-08-15 11:27:01 -0400
JekyllCron running at 2017-08-15 11:28:00 -0400

Voila, we know how to run a cron job. And we think we can turn it off by deleting the edited line.

The evil of the day, etc. We’ll call it a morning. We’ve learned a lot and simplified our product.

Today’s main lessons are:

• Don’t try to make Sublime be your default Terminal editor;
• Give up and use nano for simple stuff;
• Thinking about the product can produce much simpler solutions, even late on in the process.

It’s that last one that I propose to hammer on a bit more right here.

It’s Hammer Time

Requirements bad.
Collaboration good.

All too often, “Agile” teams are given what amounts to a List of Non-negotiable Requirements in story form, and are expected to plow through those at high velocity until the project is done. This is an inferior idea. It’s a bad idea. It’s not consistent with “Agile”, where we said:

Business people and developers must work together daily throughout the project.

Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

When business and development truly collaborate, good things can happen. In Scrum, much of this is in the Backlog Refinement activity, which is not a scheduled meeting but an ongoing allocation of working together. In backlog refinement, the Customer / Product Owner and [some of] the developers talk about upcoming needs and priorities, and they draw from the prioritized “Product Backlog” to create “Sprint Backlog Items”. These items should at worst be down-sliced stories from the Product Backlog. More ideally, they’d be new solutions, created in the moment, to address the needs reflected by the backlog.

Also in Scrum, there’s the Sprint Review, where we show the running tested product to all our stakeholders, and receive and consider their ideas, wishes, and priorities, in the light of seeing the real product. This feeds into the backlog and into refinement.

In the Scrum canon somewhere is the notion that the Product Owner says what is needed and the Dev Team says how it’ll be done. At a bare minimum this means that the Dev Team gets to say how it’s coded. What works better is when the Dev Team, guided by input from the Product Owner (and everyone), comes up with a new and elegant way of serving the need. Newer and elegant doesn’t mean that they come up with some amazing new way to do this if only we buy a thousand new servers, change to a serverless cloud architecture, and start coding in Elm. It means the creative discovery that “Hey, if we just did these two things here, we could get this out there helping people, faster and better than we realized yesterday”.

Tozier and I, playing both the Product Owner and Dev Team roles, came up with a tiny example of that today. Of course it’s easier when it’s the same people, you’re not on deadline, and your product is tiny. But the principle is the same and it’s critically important if you want to do “Agile” well.

Think about this, please. And see you next time!