We keep re-reading about the FTP object, and re-learning how it works. Then a week later, we’ve forgotten. Yesterday, I started doing something about it.

I was feeling a bit off yesterday, so cancelled the work at the BAR. Later in the day, I perked up and decided to learn – yet again – about how the Net::FTP object works. I was particularly interested in how to be sure that a nest of folders exists, so that we can publish a file like this:

articles/017-08ff/ipad-q-test2doc/index.html

The way my publication scheme works, every article and its associated files is in a separate folder. The top level, articles is pretty much guaranteed to exist, but everything the rest of the way down may or may not be there. Any folder might be new, for a new article, and any file might be new. Or, we might be doing a revision and the folders and files might exist all the way down.

Our Publisher just wants to know the Pathnames of the files to move, and to move them to the corresponding places on my web site – that is, on the other end of our FTP connection. So Publisher needs to safely create and traverse the remote structure, creating any necessary folders and files along the way, but not removing anything that already exists.

The Ruby Dir and FTP classes don’t help with this as much as we’d like. They both have what we call a “little yellow man”. They are always positioned on a particular folder, much as you are in Terminal as you cd around, or in your file browser as you click on this or that folder. So in the various incarnations of this little program, we’ve done various things to be sure we always know where the little yellow man is standing, and we’ve gotten in trouble many times when he wasn’t where we thought he was.

Along with this, FTP in particular is rather odd. We’re never quite certain whether we can in one go create a file path like articles/017-08ff/ipad-q-test2doc or not. If that one exists, can we create articles/017-08ff/some-new-place by reference from the top? Or must we trek down the tree until the little yellow man is standing in articles/017-08ff and then say ftp.mkdir('some-new-place)?

Now, you probably actually know or remember the answers to those concerns. We, in our dotage, and only working every few days or once a week on this, with many other things on our minds, always forget. No matter how great your memory is, in your work you probably encounter functions, methods, classes, or objects that you once knew well but right now don’t recall quite exactly.

If you do remember everything you need to know perfectly, well, that’s great. You might still enjoy using something like what follows in order to learn something the first time. For me, what I’m about to show you here is helpful and I’m wondering why we haven’t done it quite this way before.

So, as I was saying, yesterday I decided to write a real test for FTP. It’s not done yet and if we go to BAR today we’ll probably extend it. But here’s what I have so far:

require "minitest/autorun"
require "net/ftp"
require 'pathname'

# The code here is pretty close to the simplest thing that will work;
# Or, in the case of some tests ... not work.

class Test_FTP  < Minitest::Test
  def test_start
    path = 'Dropbox/ftp-experiments'
    ftp = Net::FTP.new('localhost')
    ftp.login(user="ron", passwd="PASSWORD")
    assert_equal('/Users/ron', ftp.pwd)
    ftp.chdir(path)
    top = ftp.pwd
    assert_equal('/Users/ron/Dropbox/ftp-experiments', ftp.pwd)
    assert(folder_exists?(ftp, 'first'))
    assert(! folder_exists?(ftp,'nowhere'))
    begin
      ftp.mkdir('first')
    rescue Net::FTPPermError => e
      assert(e.message.match('550'), "didn't match 550")
    end
    makepath(ftp, 'first/child')
    ftp.chdir('first')
    assert(folder_exists?(ftp, 'child'), "should have existed")
    ftp.chdir(top)
    ftp.chdir('first')
    ftp.rmdir('child')
    ftp.chdir(top)
    assert(! folder_exists?(ftp, 'first/child'), "should have been deleted")
    ftp.close
  end

  def makepath(ftp, path)
    save = ftp.pwd
    Pathname.new(path).each_filename { |e| make_dir_safely(ftp, e) }
    ftp.chdir(save)
  end

  def make_dir_safely(ftp, name)
    name = name.to_s
    # puts "at #{ftp.pwd} making #{name}"
    ftp.mkdir(name) unless folder_exists?(ftp,name)
    ftp.chdir(name)
  end

  def folder_exists?(ftp, f)
    return ftp.list.any? { |item| item.match('^d.* ' + f)}
  end
end

Now, I’m not going to take you through how this grew up in detail but here’s a bit of the story. I’d been chatting with Tozier via Slack, wondering what the best way would be to make a folder if it didn’t exist. It turns out that with the FTP object, if your little yellow man is standing somewhere, and the folder foo doesn’t exist, and you did this:

   ftp.mkdir('foo')
   ftp.mkdir('foo')

The second mkdir will throw an exception (550 file exists). I think it would have been more polite to return silently, smugly thinking to itself “well, that was easy”. Tozier and I saw two ways to deal with this concern. One is to test whether the folder exists and if not, create it. The other is to try to create it, and swallow the exception if it doesn’t. I don’t like exceptions, so my normal inclination would be to check and then create if needed, but I thought I should give the other way a fair chance. So I started with just this bit in my test:

    begin
      ftp.mkdir('first')
    rescue Net::FTPPermError => e
      assert(e.message.match('550'), "didn't match 550")
    end

Then I played around with what was in the rescue until I had what I wanted. What we have here now is that by my setup, first exists, so the exception will throw, the rescue will execute, and if the message is not a 550, the assertion will tell me I didn’t get the error I was trying to swallow.

No doubt there is a feature somewhere in minitest to express whether or not this code throws, but I was here to learn about FTP, not minitest, and anyway I like simple assertions, not fancy framework statements. Your mileage, etc etc.

Once I was through playing with that, I extended the test more and more, trying various things. Let’s talk about a couple of them. Check this out:

    path = 'Dropbox/ftp-experiments'
    ftp = Net::FTP.new('localhost')
    ftp.login(user="ron", passwd="PASSWORD")
    assert_equal('/Users/ron', ftp.pwd)
    ftp.chdir(path)
    top = ftp.pwd
    assert_equal('/Users/ron/Dropbox/ftp-experiments', ftp.pwd)

This little patch documents a couple of things. The first assert documents that when you open an FTP on localhost, your little yellow man is standing at the root of the folders belonging to the user who logged in. (We happen to know that on my web site, you start at /.) Anyway, you start somewhere and this first assert documents where it is.

Then we chdir to the path where the test program and folders are, Dropbox/ftp-experiments. Then we save the path (where the little yellow man is now standing), and assert that that’s where we are.

So when I read this patch of code, it tells me where we start, and checks that we wind up where we expect when we move the little yellow man.

Now, all this could be made more clear than it is. For one thing, this “test” is just one big method doing all kinds of little things. It just grew that way, and I’ve not refactored it. It would be better with lots of little tests with meaningful names like

  • test_ftp_root
  • test_rescue_existing_folder
  • test_two_level_create_does_not_work

and so on. When I was creating this thing, I was fiddling, almost as casually as one would in IRB, building up bits of understanding and trying to document them with asserts. I didn’t want to distract myself by creating a suitable setup method. I didn’t even do the obvious thing and use a Dir object to do some of my checking. I was focused on this one task. Maybe with a pair I’d have done differently – we’ll never know. The cat isn’t much use in pair programming.

So I don’t think this example is particularly excellent. And if things go as I hope today, we’ll improve this a lot. But my point here and now is that even this weak test provides some clarity about how FTP works, in a form we can review the next time we’re confused.

And that, I’m sure, will be soon.

So, there you are. When I’m learning a new object – or re-learning an old one – I find it useful to document my learning in a test. Often, I’m too “busy” to do that, and you’ve seen the result of that in this series of articles: I have to learn it over and over again, and I always learn just enough to get in trouble.

This is a technique that I don’t use often enough, because I have decades of experience of not being smart enough to do this, but being nearly smart enough to get away with remembering and re-learning. What should you do? That’s up to you. You might want to try documenting how things work with little tests. If so, I hope you find it useful.

See you soon!