How do we learn?

Sources

There are a few things bouncing around the Internet and around my mind, that induce me to write this:

• In this old article, I expressed my view that starting with some estimates and using that start to drive to slicing and dicing features, and ultimately to a viable plan doesn’t prove much of anything about estimates.
• Mike Hill wrote about “Preasons”, admonishing us to be skeptical about “rules & slogans accompanied by appealing. intuitively correct, theoretical reasoning using simple logic applied to pre-existing abstractions.”

It seems like most of my time is spent either dispensing what may seem like advice, or else reading what seems like advice in someone else’s tweet or article. It’s rare to see anything that is much like even statistical proof that the advice is actually good advice. And Hill is right in his article, and I’m right in mine: these things that work, these things we may actually advise people to try, none of them are guaranteed to work.

Let’s be clear about this

None of our advice always works. It doesn’t even always work for us, much less for other people who try it. Here are some examples:

TDD

TDD is my favorite way to program, and I’ve tried many. It gives me smooth, steady progress. It helps me build orderly designs. It helps ensure that I don’t break anything.

TDD doesn’t always work, even for me. I just flat do not know how to TDD in some languages, and on some problems. I know no useful way to TDD a GUI, for example. All I know how to do is sketch the GUI, write code that might produce it, run it, look at it, tweak until close enough.

Scrum

Scrum is a pretty good way to organize a one-team project. It provides for short feedback cycles, insists on an empowered connection to the business side, and has a very few lightweight rituals to keep you on the road to success.

Scrum doesn’t always work, even for me. I wasn’t able to figure out how to use Scrum in the conversion of my web site, because almost every single thing we did was discovering how to do it, and only after discovering did we actually do something. There was almost no moment when we could predict two items in advance what we should do next.

Refactoring

As we build a program, inevitably the design we have deviates from the design we’d like to have. This comes about partly because our work isn’t perfect. It comes about partly because as time goes on, we learn that some of the decisions we made were not ideal, and we now see a better way to do things. Refactoring is an incremental process where we can evolve our program to bring its design more in line with what we think it should be.

Refactoring always works for me in the sense that I can always improve the program. But it doesn’t always lead to a design I’m satisfied with. Sometimes that’s just because the better design is too far away, and since I refactor incrementally, and do not work often on the program, I don’t refactor much. Sometimes, I don’t have decent tests, and may not even know how to get decent tests. In this situation, attempts to improve the design become more risky and it’s not worth it.

Refactoring can work even more poorly for some teams. I know a team who put two of their very best people on refactoring some design aspect that needed improvement. It was only going to take a week or so. After 44 days, they gave up, when they realized that their new design couldn’t be fitted into the system, because it had evolved away from them.

We could go on …

It’s common to look at these situations and draw two very different conclusions:

1. Well, see, it works for me and I’m sure you just weren’t doing it right.
2. Well, see, that proves that TDD/Scrum/Refactoring doesn’t work and therefore it’s a bad idea.

The first position, when one takes it, seems to irritate those who hold the second position. In this situation, the done thing is for the people who found that TDD/Scrum/Refactoring didn’t work for them will immediately pull “No True Scotsman” out of some orifice and beat you over the head with it.

Unfortunately, the people who take the first position are commonly quite right. Very likely, in exactly the situation in hand, they’d have done better. I really can TDD/Scrum/Refactoring better than someone who’s just trying it for the first time, and odds are I can do it in their situation. Similarly, I stumble over a carpet while Karl Wallenda walks across my lake on a wire, in a hurricane, blindfolded, carrying a bag of angry cats.¹ For me, at least, walking on a wire across the lake is a bad idea, even without the cats.

Similarly, no true Scotsman was there when these guys had trouble with TDD/Scrum/Refactoring and surely they were doing their best when they stumbled on the carpet. It didn’t work — for them. For the Scotsman, however, it quite likely would have worked.

Science!

Yes, well. If my claim is that TDD always works, and you try TDD and it doesn’t work, then you’ve proven that my claim is false, if in fact you really tried TDD. Even in real science, you do have to duplicate the experiment. You can’t just toss a couple of leaves in a glass and claim that iced tea doesn’t work.

Still, there’s important material here. Even if TDD works for me, even if estimates work well for Clarke, even if whatever you do works for you, it’s not science.

Software development with teams is inherently at least complex and often chaotic. I mean complex and chaotic in David Snowden’s sense in his work with Cynefin. Our work is almost always on the wrong side of cause and effect. It’s not completely disordered, but things are so loosely coupled that no rote approach will always work. We need to try things, observe the results, adjust what we try.

Software development is so entangled with people and their interactions that nothing works all the time. We must inspect and adapt. Hmm, that sounds familiar.

–