I’m alone at the BAR1 today, so I’m probably not supposed to write any production code, but I feel like programming, so I’ll work on building some kind of object to help with our issues with Dir, FTP, and such. I’ll start with a few moments to think about what’s needed.

As it works right now, we scarf up all the files and folders under the Dropbox folder where new iPad articles go. We copy them down into the corresponding folders inside my rj-com folder on my Macs.2 After we run Jekyll, everything under rj-com has been generated over to the Jekyll _site folder, which lives inside rj-com. Jekyll converts Markdown files to HTML, moves other files, applies templates and all that stuff to build up the web site. After Jekyll runs, we use the original list of what we found in Dropbox to decide what we should push to my web site, using FTP.

Doing all that involves keeping an array of strings that represent folder and file names, driving various copy commands and FTP commands from the strings. This is, of course, a code smell. Using raw strings to represent structures and semantic ideas is a clear call for a smarter object to help with what’s going on. Here’s one example from our code:

  def move_the_articles
    move_batch("#{@jekyll_folder}/_site/articles", 
      'articles', files_to_ftp(@ipad) )
  end

  def files_to_ftp(folder)
    perform_in_folder(folder) do
      Dir.glob('**/*').collect { |f| rename_md_to_html(f) }
    end
  end

  def move_batch(local_site_folder, ftp_target_folder, folders_and_files )
    perform_in_folder(local_site_folder) do 
      ftp_connection = ftp_connected_to_target
      ftp_connection.passive = true
      ftp_connection.chdir(ftp_target_folder)
      folders_and_files.each do |file|
        ftp_one_file(ftp_connection, file)
      end
      ftp_connection.close
    end
  end

  def perform_in_folder(new_dir, &block)
      pwd = Dir.pwd
      Dir.chdir(@dir_pwd)
      Dir.chdir(new_dir)
    begin
      result = block.call
    ensure
      Dir.chdir(pwd)
    end
    return result
  end

  def ftp_one_file(ftp, file_name)
    if File::directory? file_name
      ftp_mkdir_safely(ftp, file_name)
    else
      ftp.putbinaryfile(file_name,file_name)
    end
  end

The main thing to notice here is all the string manipulation going on. We’re passing file names around as strings, and we’re dipping into the File class to find out whether our input string is a folder. A better design would be to have an object there where file_name is, and to ask it whether it’s a folder, or better yet, tell it to do the right thing, namely either make a folder on the ftp site or copy a file into it.

Similarly, the code around perform_in_folder is bizarre. Its only real purpose is to make the current folder that Dir knows about be the one we want, and then to restore Dir to whatever it used to be, in case someone else is relying on it. Here, there should be an object that’s unique to us, that we can interrogate as we wish, without worrying about some other guy holding on to some other similar object.

So I want to begin to work up an object or objects to help with this kind of thing. A candidate to use, that we’ve been talking about for a while, is the Ruby Pathname class. This thing holds on to a string that represents a file or folder, and supports many of the obvious methods from Dir, FileUtils', FileTest, and File`. It looks like it might be a nice supporting class for what we need.

The next question is, what do we need? Here are some ideas, in no particular order:

  • Get a collection of all the files in the iPad Dropbox source;
  • Safely create necessary folders in the rj-com source;
  • Copy all those files down into corresponding folders in rj-com;`
  • Safely create folders on my web site, corresponding to the Dropbox source folders;
  • Move files, using FTP, from the corresponding rj-com/_site folders to my website;

But wait, there’s more. Our basic product idea has evolved to something like this:

Watch for changes in the Dropbox iPad source folder, or for changes in the rj-com folders themselves, copy any new files from the iPad source into rj-com, run Jekyll, and push up to the web site all the newly-generated files and the various overhead files (categories, index page, feed, etc).

So we’d like to support some kind of date-checking capability, to determine which files are “new”. This would also allow us to efficiently use the JekyllRunner to handle all the site generation, not just the iPad input support.

Enough talk. It has taken me 50 minutes to write this up, far more time than if I had just made some notes on a card or something. Let’s do some work. Here’s my first cut at a “story” for this thing:

Check whether one of two files is newer. Given a partial file path and file name, build an object that checks under two different base folders and answers whether one of the files is newer than the other. That is, given a/b/c.txt, check whether X/a/b/c.txt is newer than Y/a/b/c.txt.

My rough idea for how to do this will be to have a Pathname, and a couple of base folders, and to root around until I can answer the question. In short, I’ve never done this before, so I don’t know quite what I’ll do, but I plan to start with Pathname. And I’ll try to TDD this from the beginning. I think I’ll put this stuff in the test-ftp folder where we developed the other stuff. Here goes.

I begin by making folder and file paths test-X/a/b/c.txt and test-Y/a/b/c.txt with the former being newer. Now for the test. I begin, as always:

require "minitest/autorun"
require "pathname"

class Test_File_Thing  < Minitest::Test
  def test_hookup
    assert_equal(4, 2+3)
  end
end

This fails nicely, as intended. Now a real test. Arrange, Act, Assert, but we write the assertion first:

  def test_newer
    assert(thing.newer?, "should have been newer")
  end

Yeah, well, if you were here you’d have a better idea. I’m thinking I’ve made a thing (I don’t have a good name for this object yet, and that it’ll be thinking of two parts rooted in path-X and path-Y and we’re asking whether its first is newer than the second.) Maybe we’ll want to have methods like first_newer and second_newer or something. Don’t know, don’t care. We’re spiking and we don’t expect to keep any of this code unless a miracle happens.

My next sketch is:

class Test_File_Thing  < Minitest::Test
  def test_newer
    thing = FilePair.new('a/b/c.txt', 'path-X', 'path-Y')
    assert(thing.newer?, "should have been newer")
  end
end

I’ve made up a name now, FilePair, and a creation sequence that names the common part of the name and the two proposed roots. I’m feeling no big commitment to any of these. This won’t run of course, until I build FilePair.

class FilePair
  def initialize(common_path, first_path, second_path)
    @common_path = common_path
    @first_path = first_path
    @second_path = second_path
  end

  def newer?
    false
  end
end

OK, a bit of faking there, as planned. Let’s see what Pathname has to offer here, since my tentative plan is to use it. I could, I suppose, just do some File.join stuff but that’s not the plan. Here goes:

class FilePair
  def initialize(common_path, first_path, second_path)
    @pn1 = Pathname.new(first_path)+common_path
    @pn2 = Pathname.new(second_path)+common_path
  end

  def newer?
    @pn1.ctime > @pn2.ctime
  end
end

This actually works, and I freely admit that I checked manually whether ctime returns the actual change time of the file, using a puts. I did so because of this cryptic line from the Pathname document:

ctime → time
Returns the last change time, using directory information, not the file itself.

I wasn’t sure just what they meant by that, so I looked.

OK, so far so good, this is a bit of progress. I probably need a few other methods, like perhaps second_exists and then some folder and file makers and movers. Here’s second_exists?

class Test_File_Thing  < Minitest::Test
  def test_newer
    pair = FilePair.new('a/b/c.txt', 'path-X', 'path-Y')
    assert(pair.newer?, "should have been newer")
  end

  def test_second_exists
    ok = FilePair.new('a/b/c.txt', 'path-X', 'path-Y')
    assert(ok.second_exists?, "didn't find c.txt")
    missing = FilePair.new('a/b/nothere.txt', 'path-X', 'path-Y')
    assert(! missing.second_exists?, "eek found nothere.txt")
  end
end

class FilePair
  def initialize(common_path, first_path, second_path)
    @pn1 = Pathname.new(first_path)+common_path
    @pn2 = Pathname.new(second_path)+common_path
  end

  def newer?
    @pn1.ctime > @pn2.ctime
  end

  def second_exists?
    @pn2.exist?
  end
end

So this is going nicely, isn’t it? Now I think I’ll put explicit setup and teardown in place, creating and destroying the test folders and files that I need. I’ll use Pathname to do it, where possible, for the practice.

class Test_File_Thing  < Minitest::Test
  def setup
    pnY = Pathname.new('path-Y/a/b/c.txt')
    pnY.mkpath
    sleep(1) #gotta make it newer
    pnX = Pathname.new('path-X/a/b/c.txt')
    pnX.mkpath
  end

  def teardown
    pn = Pathname.new('path-X')
    pn.rmtree
    pn = Pathname.new('path-Y')
    pn.rmtree
  end
  ...

So that worked. I had to put a sleep in there to ensure that the X path, done second, was newer. As an experiment, I’m going to remove the teardown, to see what mkpath does if the path exists.

And in so doing I learned two things. First, the mkpath happily returns if the path already exists. Second – and probably this should have been obvious – mkpath created a folder named c.txt, not a file. So we’ll have to do the file creation separately. I hope that’s not a problem:

OK, that was a bit weird. Here’s what I wound up with:

  def setup
    pnY = Pathname.new('path-Y/a/b/')
    pnY.mkpath
    pnYC = pnY+'c.txt'
    pnYC.write("hello")
    sleep(1) #gotta make it newer
    pnX = Pathname.new('path-X/a/b/')
    pnX.mkpath
    pnXC = pnX+'c.txt'
    pnXC.write("hello")
  end

That creates things as intended. It is indifferent to whether the paths exist or not, that is, it creates safely as far as I can tell. (I’m assuming it didn’t somehow delete the whole path and then recreate it.) My teardown is still turned off and the tests all run without it, since the creation doesn’t fail if things exist. That’s good as far as I’m concerned. Still, I’ll turn the teardown back on and check visually to be sure the test-... folders go away. And they do.

OK, that’s a good hour’s work, some writing, some learning, some sample code. I’ll call that a success. Maybe I’ll play some more over the weekend, or maybe wait until Tozier and I get together next Tuesday.

Bye for now!

  1. Brighton Agile Roundtable, the round table we sit at in the Brighton Barnes & Noble coffee shop. ↩

  2. There are two, my home machine, and my laptop. They are kept in sync via git and GitHub: they’re not in Dropbox. ↩