This really is going to take more time than I wish to spend on it. However… Here it goes. I’ll put it in installments, r, p, t, f, for requirements, programming, test, and field. I’ll also try to establish very safe points of agreement early, hopefully we’ll all see it coming.

The first thing is to establish that it really does cost x10 or similar to detect and handle mistakes as we proceed across r, p, t, f, (i.e. exponential) and that XP really is well suited to handle that exponential growth in cost. It is not the case that the x10 doesn’t exist, or that XP nullifies it. XP deals with its existence, is “well aware of it” to use my preferred anthropomorphizing locution.

The other thing I want to do is be cautious about the phrase “cost of change”, because it is not clear with that phrase whether we mean or or more of (1) cost of detection / notification / preliminary isolation, (2) cost of implementing a change request, or (3) deploying the revision.

Reexamining the Cost of Change - Requirements

Finding a mistake during requirements (cost=r) is much cheaper than finding the same mistake during programming (cost=p).

p > 10 * r according to the following account.

Customer is talking to Programmer, says, “We want to blah blah blah…. etc.”

Programmer has critical ears on alert and says, “But blah blah blah…”

Customer says, “Oh my gosh, that’s right. No we don’t want blah blah, we want blah …”

Elapsed time: a few seconds to a few minutes.

Call it 10-100 seconds.

Compare that with Programmer not listening with critical ears. Spends however long making blah blah run. Shows to Customer.

Customer says, “Look at that, Oh my gosh, that’s not right. No we don’t want blah blah, we want blah …”

Programmer curses or shrugs, depending on phlegma level in blood, calls on partner, revises tests, revises design, revises code.

Elapsed time: 5 minutes to 2 hours for 2 people (2 days, perhaps, but we’ll leave that out for now).

Call it 10- 100 minutes.

Finding the same mistake was 60 times more expensive; p =~ 60 * r. I only need it to be x10, so x60 is fine for my line or reasoning.

Ergo, I maintain I am safe in saying that the exponential cost of change is well established and still in force in the r-to-p portion of the story, even on XP projects.

IMPLICATIONS

Mind you, I don’t say any of this is bad. The exponential thing is merely true. XP handles it neatly: put the customer next to the programmer, have the programmer listen with alert ears, and the likelihood increases that they will find a mistake during requirements generation (wish exposition, if you wish an alternate phrase). Even without having watched any of you work, I am quite confident that Kent, Ron, Rob, and all the other XPers I have met really do practice critical listening and try to catch mistakes inside the requirements / wishing conversation where they can. Why? I’ll freely assert that it is because they and we and you all know that we stand to save much energy if we can make a correction during the speaking of requirements / wishes.

How does this compare to Bureaucratic Specification (BS) methodologies? BS methodologies are also aware of the increased cost of late discovery, so they spend more time and money early on trying to catch the mistakes. The idea is that we can afford to spend x60 on every requirements assertion, so we can afford to write so much more. The cost of a requirements change isn’t “r”, it is a bigger number “R = 60 * r”, ideally. Personally, I think this backfires, because the actual time spent on requirements is much more than x60, and the cost of programming changes is also higher. But the idea of shrinking the number R down toward r lies behind all of the low-bureaucracry methodologies, and the techniques are to reduce paperwork, increase face-to-face and shrink the number of people between customer and programmer. As usual, XP goes farther in this direction than any other methodology.

Reexamining the Cost of Change - Programming

Finding a mistake during programming (cost=p) is much cheaper than finding the same mistake during external test or systems integration or pre-deployment (cost=t).

t > 10 * p according to the following account.

Programmer makes a silly mistake, it could be an off-by-one or it could be something more substantial.

Programmer happens to have just written a unit test for that, unit test shows up the mistake.

Programmer fixes it, reruns test, checks in the software.

Elapsed time: 2 people, anywhere from 2 - 10 minutes for this category of mistake.

Call it 4-20 minutes.

Compare that with Programmer misses a test case. Program leaves programmer’s hands stamped OK.

Test department or whoever does final check on this beast has spent untold hours inventing weird and wonderful tests. One of them fails (bound to happen sooner or later). Tester fills in bug log with test that fails and information available.

If we are really lucky, they pass on the bug reports to the original programmer team. If not, multiply the following by some arbitrary multiplicative factor to deal with the fact that people not familiar with the original program are faced with fixing it.

Programmer picks up bug list, looks at bug report, finds a partner, they discuss, they work to isolate the problem, they evaluate the unit tests to discover why they didn’t catch said error. They investigate the design to see why it produced that error. They write new unit test, refactor, add code, etc until it passes, and mark the bug report corrected.

Tester picks up fixed bug list, finds the test to run, reruns the test, it passes.

Notice that there are at least three people in this story, not just 2.

Elapsed time: 3-4 people, each spending 10 - 30 minutes one this.

Call it 30 - 120 minutes

Finding the same mistake was on the order of 10 times more expensive in this scenario. While the ratio could easily get larger, it won’t get much smaller. So t > 10 * p as needed for my line or reasoning.

Ergo, I maintain I am safe in saying that the exponential cost of change is well established and still in force, even on XP projects, in the r-to-p-to-t portion of the story.

IMPLICATIONS

How does XP handle this portion of the exponentially growing cost of repairing mistakes? Fully automated regression test suites that must pass at 100% at all times for any checked in code. Man, is that ever a heavy methodological requirement! But it sure does reduce the number of bugs getting to the external test department, thank goodness.

Writing all of those unit tests sure does cost time and energy. But it easily pays for itself, and that is because of the exponentially growing cost curve, among other reasons. It pays for itself in other ways, but those aren’t relevant to this discussion.

Please note, I am not attacking any of the XP practices. My sole goal is to establish that the exponentiially growing cost curve is still in force; and more than that, to show that the XP practices “know” about that growing cost, and deal with it.

If XP really were based on the assumption that finding and fixing a mistake late were as anywhere near as cheap as finding it early, then it would use a more optimistic bug detection policy, playing the statistical game of early-market-gain vs. lost-opportunity-cost, calling for as few tests as possible and allowing a larger portion of them to be caught after the programmers were “done”.

Reexamining the Cost of Change - Testing

Finding a mistake during testing (cost=t) is much cheaper than finding the same mistake in the field (cost=f).

f > 10 * t according to the following account.

In Installment P I described finding and repairing a mistake found in external test, and repaired in 30-120 minutes by the original programming team.

Compare that with this:

Software gets deployed, goes out to 100 to 10,000 to 10,000,000 customers.

Some percentage of the Customer’s machines run into the bug. Each one loses anywhere from 15 minutes to several days in work. Some percentage of those call or write to the field service staff of the people who wrote the software.

One or more Field Service Staffers write up a bug report, pass it back to whomever.

Whomever starts an inquiry into the cause of the problem…

I don’t think I need to go on. It is clear that we have passed the x10 number required for the argument.

Important rule: don’t let bugs go out into the field unless the market opportunity is much greater than the large field repair bill.

IMPLICATIONS

At this stage, I claim I have adequately made the point that the exponential cost curve is still safely in place.

Quod erat demonstrandum.

Reexamining the Cost of Change - Conclusions

The exponential cost curve is mostly in detecting and communicating the Mistake and naming the change that is to be made. XP cannot change that curve, and indeed, XP takes that increasing cost curve neatly into account. So the first lesson I get is that one should not base a defense of XP on the ABSENCE of the curve, but rather on the PRESENCE of the exponential cost curve.

This leaves us with two topics - why is XP better than Bureaucratic Specification (BS) methodologies in dealing with that curve, and what about Kent’s flattened ( O(log n) for the initiated) cost curve.

I really don’t want to go into the math for BS methodologies. They also try to handle the exponential growth, but basically their cost for everything is a Very Large constant greater than XP’s. So much greater that XP is running the cost-to-fix in “p” (programming time) while the BS methodologies are still running in “r” (requirements time). So XP has the program written and fixed by the time the BS methodology has just the spec written and studied. Keep the exponential curve, but drop everything by very large constants, and XP is likely to have shipped the product while the other one is still being designed.

The flatter curve in the XPE book looks like an O(log n) curve. I have no idea where that shape comes from. I suspect it is supposed to look attractive and to make a point. The point it is supposed to make is that with Simple, Well-factored-code, with Full-unit-tests and a Programming-partner, we can make a change for less cost than we could without any of those four.

Actually, that indicates to me where the shape of the curve comes from. Let the line y=x on the graph mean “Same cost to implement a change as with your average methodology”. Then having a faster-than-linear curve would indicate “As time goes by and the program gets bigger, it costs MORE to implement a change than with your average methodology.” And so having a slower-than-linear curve indicates “As time goes by and the program gets bigger, it costs LESS to implement a change with XP than with your average methodology.” And I think this curve is correct.

OK, someone who is good with phase space curves can maybe save the day here. I think this is a phase-space curve. The vertical axis is Cxp, “Cost of change using XP”. The horizontal axis is Cfav, “Cost of change using your favorite methodology”. The solid line tracing “y = log(x)” is a trace over time or trace over program size of the cost of implementing a change request ( Cxp / Cfav ). At the origin is t=0 or psize=0, and as t or psize grows, it traces out this curve. And I think this curve is correct.

Alistair