You can't build a decent house incrementally. Therefore XP's incremental software development cannot work. Houses aren't software. Software can be flexible and modular in a way that houses are not.

People often raise analogies, like house-building, to “prove” that XP’s incremental evolutionary approach to building software cannot work. Like most analogies, these are questionable. Nonetheless, I wanted to get some thoughts in place from the viewpoint of the inflexibility of physical objects compared to that of software.

OK, let’s explore the home-building analogy, from this basis: On every project I’ve ever worked on in more years than most readers have been alive, the requirements always changed in some very substantial way before the end of the project. Ways analogous to new rooms (we’ve decided we need a graphs module), deciding to have a basement after all (we’re going with Oracle), and replumbing the entire house (instead of buying a bigger server, we want you to just make it multi-server).

In housing, if we tell our architect that we want to be able to inexpensively change our minds about these things, we will wind up with one of a very few unsatisfactory solutions:

  1. Some kind of modular ugly snap-together manufactured housing.
  2. Some amazing over-engineered home with expensive conduits, no load-bearing walls, ugly as sin and still not able to be what we want.

Software isn’t like that. Those aren’t the only options.

Modularity works. When all the database access goes through one module (high cohesion of the code) with a simple interface (low coupling of the code), you can go over to Oracle without rewriting the entire system.

One way to get good modularity is to design the whole system, based on as much information as you can get, to be modular. All key use cases, holistic design, call it what you will. That scheme works just fine, with just one drawback: it has a long delay before delivering actual valuable software to the customer.

We want a better way, one that attains the same design quality, but that also lets us deliver value as we go.

We claim, and there are lots of people doing it, that you can grow a system in such a way that it is always modular and always delivers value. It goes like this:

Write a tiny system that does something of value. This takes essentially no design time, because it’s tiny and you’re smart. Take the time, which won’t be much, to make the system modular, or as we call it these days, well factored.

Now you have a tiny system which is well factored and delivers value. By the induction hypothesis, this system is flexible.

Take another requirement, and add it to the system. Because the system is well factored, the requirement will be easy enough to add. Now make the system well factored AGAIN, making sure that it expresses all the abstractions that need to be there. This still doesn’t take much time, because either you extended an already-existing abstraction, which is easy if everyone uses the same ones, or you added one, which is easy because your abstractions are loosely coupled by the definition of well factored.

Repeat. Add capabilities, refactor. Keep, always in hand, a system that is well designed for what it does. Such a system is modular and therefore flexible.

There are three main ways this could break:

  1. Modularity may not lead to flexible systems. We think this is a given. So do you, or you would have stopped where I said "stop here".
  2. The team might stop refactoring, letting the system get less and less modular. Yes, this will kill you. If you set out to build a system incrementally, you must keep it well factored or you will get wedged. Danger, Will Robinson, warning warning.
  3. The refactoring might be somehow more expensive because of "excessive rework". That turns out, in our experience, not to be the case. I think it's because modularity works, and, on the average, the system is only of complexity 1/2. I'm not sure why it works, but in our experience it works.

So what do you do in a world demanding flexibility? You can design your software for flexibility, building it to whatever standards of flexibility you believe in. Or you can use those same standards and do it incrementally.

If you muse, think, imagine, and design, your customers go a long time without value. If you build the framework first, they go an even longer time.

If you go incrementally, you can give your customers value as you go, while preserving everything of software quality that you believe in. We prefer that alternative, and recommend it.