Alex Bunardzic has been part of a rambling Twitter thread about TDD and people who object to it. One of his concerns is that TDD proponents agree that TDD isn’t always appropriate but don’t say when. Let’s explore that.

Alex’s concern seems to be that he’s trying to get people in his organization to use TDD, and a number of them raise objections. One of those objections is that even the experts say that they don’t always use TDD, so why should we use it here.

Now if you take sales training1, one of the things you’ll be taught is how to deal with objections. The first and strongest technique is to ignore them. In sales, you may then move on to countering objections, which is one reason why so many of us hate so much to be sold to.

I am not here to sell anything, but It seems that Alex is somewhat responsible for selling TDD in his company, so we may have different concerns. My concern in writing about and teaching ideas is just to be understood. I am happy to leave the decision about what to do in the hands of the listener. If I were on a quota for selling so many TDD installations each month, I might feel differently.

Nonetheless, I do know of some situations where I typically do not use TDD, and I’m more or less happy to talk about them. More or less because the TDD topic seems to drag on and on, furrowing into my soul. Anyway, here goes:

When do I not TDD?

Let me count some ways: I sometimes don’t TDD …

  1. when a decent TDD tool isn’t right at hand;
  2. when I can’t think of how to test something;
  3. when the output is visual in nature;
  4. when the program is a simple throwaway.

Let’s explore each of those a bit more, with examples.

I often do not use TDD when no decent testing tool falls readily to hand.

One example was Codea Lua on my iPad. It doesn’t ship with a TDD tool, and for a long while there wasn’t one available. I toyed with writing one, since figuring out reflection in Lua was kind of interesting, but never really came up with anything that I liked.

Then along came CodeaUnit, which works very nicely and which I used recently in an example TDD session. But see the following sections.

Another example is Second Life’s scripting language, LSL. The language is rather rudimentary, and does not include a reflection ability, which is important when you’re trying to produce a decent TDD framework. Without a tool available, I TDD less often than would be ideal.

Note, however: that lack of a decent tool is not a very good reason not to use TDD. Often my development in Lua or LSL takes longer than it would if I were following the TDD discipline. How do I know? I’m very experienced with what happens when I use TDD and when I don’t, so I readily recognize situations where the lack of tests is hurting me.

The best sign is debugging sessions that take more than a moment or two. This is always a sign that I’d have done better to have more tests.

Should you consider not using TDD if a tool doesn’t fall readily to hand?

I don’t think that’s a very good reason. There almost certainly is a good TDD tool for whatever language you’re working in. For me, who only writes small throwaway programs, the investment in finding and learning the tool may not be worth it. You’re working professionally. For you, the investment probably is worth it.

Very likely, it would even be worth it to me. Working without TDD puts me in debugging mode more often than I’d like. The value of TDD, even for my small projects, may well be there. I chalk my not doing it up to laziness, to tell the truth, not to wisdom.

* I often do not test things when the output is visual in nature.*

In Codea/Lua, one is often writing some code to draw moving objects on the screen. In Second Life, I often write scripts to move things about in the virtual world. While there are pieces of these scripts that might benefit from TDD – there always are – the main functionality is such that I don’t know how to use TDD effectively.

Let’s take two examples.

Imagine that I want to draw a steam engine on the iPad screen. Part of that picture will be an engine drive wheel that rotates as the train moves. Attached to that wheel, somewhere near the middle of its radius, is a push-rod. The wheel end of that push-rod goes around and around2. The other end of that rod moves back and forth horizontally, pushed by another horizontal rod, attached to the steam piston, that just moves back and forth.

Our mission is to calculate the position of that push-rod for every angle of the wheel as it rotates. To figure out how to do that, I draw some sketches and do a little geometry, often with right triangles, until I can figure out what position the far end is in given every angle of the wheel.

Clearly, when the wheel is at zero, the push-rod is fully away from the wheel, and its end point will be l + r, where r is the radius of the push-rod circle, and l is the length of the rod, given that the wheel is considered to be at position zero.

When the wheel is at 180 degrees, the end point will be at l - r. When the wheel is at 90 or 270, the end point will be … sqrt(l^2 - r^2). Meanwhile, the start point if the angle is theta is <1,0> rotBy(theta).

Probably. A little more geometry convinces me that the end point is at sqrt(l*l - y*y) - x, where x and y are the relative coordinates of the starting point.

Probably, there’s a better way to calculate the end point. It’s got to be some kind of sin or cos function or something like that. But this way works fine, because we have the ability to draw a line between two points.

So I just code that, and it looks like this, without a lot of cleanup:

-- Loco

function setup()
    theta = 0 -- degrees
    deltaTheta = 1
    origin = vec2(600,600)
    radius = 200
    rodRadius = radius/2
    rodLength = 500
end

function draw()
    background(40, 40, 50)
    strokeWidth(5)
    drawWheel()
    drawRod()
    theta = theta + deltaTheta
    
end

function radians(angleInDegrees)
    return angleInDegrees*math.pi/180
end

function drawRod()
    pushMatrix()
    local rodOrigin = vec2(0,rodRadius):rotate(radians(theta))
    local x = rodOrigin.x
    local y = rodOrigin.y
    translate(origin.x, origin.y)
    ellipse(rodOrigin.x, rodOrigin.y, 15)
    local rodEnd = math.sqrt(rodLength*rodLength - y*y) + x
    line(x,y, rodEnd,0)
    -- local dist = vec2(x,y):dist(vec2(rodEnd,0))
    -- print(dist)
    popMatrix()
end

function drawWheel()
    pushMatrix()
    noFill()
    translate(origin.x, origin.y)
    ellipse(0,0,2*radius)
    rotate(theta)
    line(-radius,0,radius,0)
    line(0,radius, 0, -radius)
    popMatrix()
end

Here’s a little movie of it running.

You may notice the two commented-out lines in drawRod. Those print the length of the push-rod line. It’s always 500. That’s the only test I could think of to be sure I was getting the right point. That it’s at zero relative to the wheel is obvious in the code and on the screen.

In this little program, I don’t see anything that needs TDD-style testing, and I don’t see much that I know how to test at all.

But there were some defects. First, a couple of times I didn’t like the positioning or size of the drawn items. I just looked at the picture and changed the numbers. Second, in optimizing by creating the local x and y, I somehow messed up the call to translate(), which made the push-rod draw itself down by the origin. So I fixed that.

The program came into being nicely and I honestly don’t see where TDD would have helped me. However, there were points in time where I didn’t feel confident, and at least one point where there was an actual mistake in the code. So if I were better at TDD, I might have done a little better using it.

The same kind of thing happens in Second Life. In SL, my scripts often move things around, which is basically done by setting their x,y,z coordinates. To make what looks like smooth movement, you adjust the coordinates a little bit, very frequently. The calculations are much like those in the Lua example, some sines and cosines and rotations and roots. I “test” by running the program and watching how it looks.

There is a particularly interesting thing in SL that adds to the fun. If you give your object a weird incorrect set of coordinates, it is prone to fly there, leaving your view and leaving you with no idea where it has gone or why.

The usual thing is to stop the program before moving to <0,0,0>, or when a move is longer than some sensible distance, or outside some bounding box. I find that I always put in those checks after losing the object a time or two.

Would TDD help me avoid that trouble? I’m not at all sure. As in the Lua case, I don’t know numerically what I want: I have to see it to decide if I like the effect.

I often code defects and often wish I had better tests, or something better constraining the code. But I rarely use TDD, both because there is no decent tool, and because I am not sure how I’d use it if I had it.

Does this mean you shouldn’t use TDD because your output is visual?

I’m not sure. Just because I sometimes don’t know how to do a thing doesn’t mean the thing isn’t useful. It doesn’t even mean that it wouldn’t be useful in cases where I don’t know how to do it. I do definitely feel that I have more bugs, and spend more time debugging, in these non-TDD situations.

And that’s my central case:

I often do not use TDD when I can’t think how to test something.

Without a testing framework, I can’t easily think how to test things, especially things without clear numerical or other clear answers. I do see how to test, say, a couple of FIFO list functions in Second Life, and not too long ago, I wrote those for a friend, and included a “main program” that tested the functions.

I was glad that I did, because list manipulation in SL is a bit clunky, and I did in fact make a few mistakes, which my tests duly found.

But much of the time, I don’t see a way to write an explicit test, and I fall back on printing and similar debugging tricks.

Could I therefore give the advice: don’t use TDD if you can’t see how to write a test? I don’t think so. I’m sure you won’t use TDD if you can’t see how to use it, but the main thing with TDD is to use it better and better. So I try to see new ways to use it, and I want my listeners and readers to do the same.

I “should” see more ways to test, and this year I see more than I did last year. I forgive myself if I can’t see how to test something, but that’s not the same as recommending not working to get some tests even if I can’t see just now how to do it.

What does that mean for someone who doesn’t have the two decades of experience with TDD that I have?

Well, I’m still learning. When I don’t – as yet – have a good handle on how to TDD something, it’s easy to fall back to my many other useful techniques, like debugging or code-and-pray. But ofter the years, I’ve continued to try to learn things, and continued to try things that get me closer to TDDing everything.

I’m glad I’ve gone that, and I commend the notion to your attention.

I often don’t TDD a simple throwaway program.

Sometimes you just want to regex the hell out of some file, or extract some records from a bunch of web pages or some other simple scripty kind of thing. Often, when something like that is before me, I just go ahead and write it, because I’m going to throw it away as soon as I have the pages scraped or the lines converted.

It doesn’t seem worth it to TDD the thing if I’m just going to throw it away.

Sometimes I even do throw it away … after the two or three or five times I edit and run it again trying to get the answer I really want.

Does this mean I think you should skip TDD for programs you’re going to run once and then throw them away?

Well, I’m closer to OK with that than I am to the other cases here, but I often spend enough time editing and rerunning to make me think there was probably a central bit of the program that would have benefited from some TDD.

Usually that bit is the regex or whatever scraping / transform part is at the center of it. Often it takes several tries to get that right. Regexes spanning multiple lines, for example, always bite me somewhere along the way.

So even here, I suggest that my laziness isn’t a great excuse for you to skip TDD. It might be an OK excuse … if excuses are what you want.

Summing Up

After over two decades of learning, practicing, and experimenting with TDD, there are unquestionably cases where I do not use it. What’s interesting is that even in the cases where I can rationalize not using TDD, I often encounter problems that would be very likely to have been caught had I been TDDing.

It’s a bit like when you don’t wear your eye protection and you splash soap in your eyes, or you quickly degrease something without gloves and your hands get dry and chapped and smell terrible, or you make chili and then rub your eyes.

You’re like “Dayum, I should have …”.

More often than not, when I don’t use TDD, I get a bit of that feeling.

My best advice is to learn it and keep learning, use it and keep pushing … and in those cases where you just plain can’t, or just plain don’t want to … get better at noticing the places where it would have helped.

Then make up your own mind. Don’t come to me for permission to do TDD, and certainly not for permission not to.

Sin your own sins, that’s my advice.

  1. I did take sales training. I’m not a decent salesman. The training called for behavior that was repugnant to me. ↩

  2. The wheels on the train go round and round … ↩