A report from the C3 trenches, including a war story and an introduction to some of the key practices used on that landmark XP project.

For the past three years, as a member of the group behind the Extreme Programming movement, I’ve been the coach on a famous Extreme payroll project. We’re taking the project enterprise-wide, and it’s interesting and helpful to see how XP played a part.

Let’s begin with a true story.

Our new payroll engine has been in production for some time, and we’re working to add a new group of biweekly-paid employees. Late last fall, the payroll department head promised to take over a small special payroll of Extended Layoff Benefits. We call them Elbonians.

Elbonians couldn’t go through our regular process. We needed to accept a spreadsheet of benefits due, run it through our engine, and print the checks and reports directly.

In a couple of weeks, we had done it all. We picked up the Elbonians without a hitch. To us it seemed like an amusing and simple exercise. We just transformed some new kind of input into our standard form, ran them through our engine, wrote out a formatted report that looked like a check. But to the customers, it was another complete payroll, although a tiny one.

Naturally, after this minor success, we fouled up our main objective. Our sample biweekly guys were paying fine, our functional tests were at 99.something percent, and the deviations were understood and accepted by the customers. We were ready to launch all 15,000 biweekly payees.

We planned to launch over Christmas. Production support would carry the brunt of the effort, with other people coming in as needed.

It didn’t go smoothly. Things went wrong, as they will. Production support tried harder, worked longer. More things went wrong. Because of the holidays, they were reluctant to ask for help, and sometimes when they asked for help, people were not available.

Finally, the day came and things weren’t good enough. Everything worked, but there had been too many problems. We scrapped the launch.

Of course, when we came back after the holidays we felt like fools. We second-guessed ourselves, our process, our heritage, the notion of using computers, the capitalist system. Then we had a retrospective meeting to review what had happened.

We looked at what we had done wrong, and what we had done right. We figured out where we had made mistakes; we renewed our commitment to our practices; we added some new practices. We bandaged our wounds and got back on track. Then our top customer dropped the bombshell:

“I’ve been looking at what we have done with the biweekly launch, and with the Elbonians, and I don’t think we should push to launch the 15,000.”

Oh, no! We’ve failed, we’ll be thrown out on the street, our children will starve! But he went on:

“We have what we need to do payroll right. I want to go ahead and reengineer the entire process. I’m setting up Business Process Reengineering meetings ASAP. We’ll start replacing the legacy process with the team’s software right away.”

Imagine that. The worst foulup of your entire project, and your customer hardly even notices! They’re so confident that they cheerfully broaden the scope of your effort.

At the time, it was a surprise. But we understand how it happened. It happened because the Extreme Programming values, Feedback, Communication, and Simplicity lead to Confidence.

Extreme Feedback

To move rapidly, you need feedback, and an Extreme project is no exception. Ensure rapid progress and team confidence through feedback, especially testing.

As we’ll see, Extreme projects get feedback on progress from the repeated Commitment Schedule, and from the Iteration Plan and Tracking. You get feedback on the hard problems by doing Worst Things First, and get feedback on the design from the continuous use of CRC design.

But the most important form of feedback is testing. Testing gives you rapid development, confidence, and often most important, it builds support from your constituency. XP breaks testing into Unit Tests and Functional Tests.

Unit Tests

Extreme programmers write extensive Unit Tests for every object in the system. Check every aspect of the object that could possibly fail. Before you release your code, the Unit Tests for your objects must run at 100 percent. But that’s not all: the Unit Tests for the entire system must run at 100 percent!

Think about that for a second. When Ann releases her new code, all the tests run at 100 percent. Now Ralph comes up and installs his code. He runs the tests. All of his tests work: after all, he tested before he began to release. But some other tests, somewhere in the system, fail. What has happened? You know for sure that Ralph has made a change to something someone else relied upon. You know instantly when it happens, and it gets resolved on the spot. The effect is that you can go very fast very safely.

Imagine the confidence the tests give you in your own, and your teammates’ code. Your tests ensure that neither you nor anyone else can break your objects. And your teammates’ tests give you confidence that you haven’t broken anything of theirs.

When you’re confident in your tests, you’re not afraid to make changes to make things better. Your tests are a key factor in your ability to improve the code as needs arise: you know you can add new features without breaking the old.

Naturally, when defects arise, and they will, you add a test to the suite that shows the defect. Then fix the defect and run all the tests. The tests get stronger, so does your ability to test, and so does your confidence.

Functional Tests

Where Unit Tests are owned by the developers, Functional Tests are owned by the customer. If something needs to work, the customer must specify at least one Functional Test for it. Functional Tests show when stories are correctly implemented. Developers use them to check their work; customers to tell whether things are going well. Run Functional Tests continuously, every day. You’ll always know just where things stand.

Functional testing builds huge confidence for the team. If you’re afraid that this or that may not work, it’s a sign that you’re not testing enough. Everything worth doing is worth testing in a visible, repeatable fashion.

Extreme Communication

Extreme Programming defines explicit power sharing between the key players on the project, the customers and the developers.

The customers identify what is needed, write User Stories about those needs, provide the resources to do the project, set the priorities, and determine the acceptability of the final result. The developers provide time estimates for all stories, do the design and coding, and ensure software quality.

The customers need to know how long it will take, what’s done so far, and how good it is; the developers need to know what to do, when to do it, and how to know when it’s done.

It all starts with communication. Here are some of the key practices we use:

User Stories

Get requirements by having the customers tell stories about how the system will work. Customers write the stories on cards. Organize stories into topic areas, and classify them by priority and risk. Plan to test every story with one or more Functional Tests.

User Stories on cards become a tangible unit of discussion, scheduling, estimation, functional testing - and completion!

Commitment Schedule

The entire team, customers and developers, meets for one or two days. Developers estimate each story in terms of Engineering Weeks to implement it. Estimate resources, and compute how many Engineering Weeks you will get per real week. Arrange the stories into three-week development “Iterations”. Count the iterations - that’s the schedule.

The Commitment Schedule will be good, but not perfect. Over time, customers remove some stories and add other ones. And you won’t always get done just what you thought you could. Neither does any other project - what’s different in an Extreme project is that you schedule again, the same way, every three iterations. The team, customers, developers, management all get the same picture of how you’re doing, every couple of months.

Iteration Planning

Take about a half-day to plan each three-week development iteration. Bring the customers and developers together. Customers present the stories to be done in the iteration. (These are usually about the same as the ones planned in Commitment Schedule, but they don’t have to be.)

Ask questions to be sure that you understand the stories. Then team-brainstorm all the Engineering Tasks that have to be done to implement each story. Sign up for tasks, and estimate how long each will take. If some developers have too much to do and some too little, balance the load. Then get to work, implementing the tasks that make up the stories.

Iteration Tracking

Even three weeks is too long to let something be off track. So a couple of times a week, ask each developer, for each task, how much work she has in on it so far, and how much there is to go. Note whether the estimates are going up, indicating that the solution may be too complex or that help is needed. Note whether not enough time is getting applied, indicating that the developer may be distracted or have too much overhead. Also feed the results back into your estimates for future iterations.

Project Reporting

Communicate status consistently, using the four Extreme Project Variables: resources, time, scope, and quality.

Resources: report whether you have the number of people and other key resources on which you based our estimates.

Time: report on the results of the most recent Commitment Schedule, and whether people have been needing to work significant overtime. Overtime is a serious indicator of future trouble.

Scope: report on how many user stories exist, and how many are done. Show whether the number done is proportional to how far along you are supposed to be.

Quality: using your quality graphs, show the increase in the number of functional tests, the increase in the number correct. Readers can see for themselves whether the curve indicates that quality will be high enough by the release date.

Report the same information you use daily. Don’t sugar-coat it. Honesty early will help when you get in trouble later. Don’t do dog and pony shows: let everyone see what is really going on.

CRC Card Design

Extreme projects use CRC cards to communicate how the system works, and to do design. CRC sessions go quickly, and everyone can understand and contribute, even customers and managers. It’s easy to get everyone actively involved in making the design better.

Pair Programming

Extreme projects write all production code with two people sitting side by side at the computer. You get better communication, cross training, better adherence to standards, and faster development.

Yes, faster development. We have tried it both ways: pairing produces better code sooner. Several factors add up to this result. In a pair, one watches strategy, one watches tactics, so you’re less likely to go off track. You can keep going longer, by switching who is driving and who is helping. You’re less susceptible to interruptions, and you discover problems much more quickly. Pair programming is an instant, ongoing code review!

Extreme Simplicity

Just as our communication practices are all simple and direct, so is our approach to development. Our fundamental rule is:

Do the simplest thing that could possibly work!

When adding new features, consider a few alternatives and pick the simplest one that could possibly work. Build tests to prove whether it works, then implement it.

When it works, refactor the code so that the simple solution fits well with the rest of the system. The objective is to wind up with well-crafted objects, but the simplest objects that support the new requirement.

But doesn’t this focus on simplicity lead to cut-and-paste programming, redundant or unused methods, obsolete classes? No! Because you refactor mercilessly. Do two methods look the same? Refactor and combine them. Do two objects have similar function? Refactor until there’s just one, or until one is a subclass of the other.

And replace all references to use the new methods or objects. That way the system stays clean, well-organized, ready for the next change. The effect of combining simple implementation with extremely clean code is that you can develop very rapidly. In short, you go faster by keeping things simple!

You're not gonna need it!

One of Extreme Programming’s most controversial rules is “you’re not gonna need it”. This rule means that when you’re adding a new bit of function, avoid thinking about all the uses for the capability that you’re going to need in the future. That way lies too much thinking, too much generality, too much slowing down.

Now “everyone” knows that you have to design for the future, you have to plan ahead. Well, in today’s object-oriented systems, everyone is wrong.

Let’s think. What would have to be true for it to be better to add unused but foreseen capability now, rather than to wait until it is actually needed?

Well, first of all it would have to actually be needed. Our guess might be wrong about what we will need, or how soon we need will it. This favors waiting.

It would have to cost a lot more to do later than it would cost now. This is the programmer’s trap: I’m here now, I’ll just add this feature. Well, cut it out. If you add the feature, you have to add a test. And you have to deal with the feature every time you change the object. In a well-factored object system, it is just as easy to add a capability later. This favors waiting.

Finally, and most important: to make it pay off to add a future feature now, there would have to be nothing more urgent to do. But there is something more urgent: the feature you are supposed to be working on.

Rely on your planning process, and your story prioritization to get things done in the proper order. And discipline yourselves to trust that part of the process to work. This lets you deliver important business value faster.

We could debate this one a long time. In my XP training session, I have at least a half-dozen slides about why it works to wait until a feature is really needed. Suffice it to say that we have been doing this for years and have never gotten in trouble by waiting. The Extreme process ensures that important risks are resolved early, and the small things will fit in when they’re needed.

Worst Things First

Remember that we classify stories as high, medium, or low risk. In your Commitment Schedule, schedule the high risk stories toward the beginning of the project. That way you learn the most, and if there is a real show-stopper, you’ll find out as early as possible.

By addressing the worst things first, you learn the key information that will help you make good design and architectural decisions as you go along. Worst Things First is one reason why doing the simplest thing actually works.

There’s an interesting side effect to doing the hard stuff first: the easy stuff is left until last. That means that as the project goes on, the work gets easier. That helps you keep going fast, even if you’re tired, even if you slow down for other reasons. It helps you keep your promises on the schedule.

Extreme Confidence

This has been a whirlwind tour of Extreme Programming. To fully describe XP is well beyond the scope of this article. Separate articles from this one have barely scratched the surface of the topics of communication and testing. There are two XP books going through the publication process now, and even they will not cover everything.

But what you should be able to taste is this: you can greatly improve your chances of success by communicating intensively on the plan, on progress, on design, and on the code; by doing everything simply but in an order that reduces risk; by testing intensively.

In the true story above, customers and management knew so much about what was really happening that their confidence rode right over a mistake that could have crippled another project.

We will make more mistakes in the future, and so will you. But by following the practices of Extreme Programming, we’re sure we’ll survive ours - and we’re sure that you can do so as well.

Check out Extreme Programming, learn about it, and build it into your process. You’ll be glad you did.

Summary

If you want more software, with higher quality, in less time, for less money, and with more certainty - then why not do it with fewer people, less management, less complexity, fewer tools - and developer commitment?

That’s the promise of Extreme Programming, a humanistic discipline of software development that says, “Act like all you have to do is write the program.”

There’s a lot more to it than that, as we’ll see. Yet Extreme Programming is a simple discipline that can help you build quality software in an environment of trust and productivity.

Extreme Programming has a focus on simplicity. We say “Do the simplest thing that could possibly work.”

Extreme Programming has a focus on building for NOW instead of tomorrow. We say, “You’re not gonna need it.”

Extreme Programming has a tight customer / developer relationship, where customers specify what has to be done, and developers provide estimates of how long it will take, in our Iteration Planning process.

Extreme Programming projects manage themselves, through their Commitment Schedule and project values.

Some Questions

But if we do everything simply, the system might be unmaintainable. Not at all. XP teams produce simple, clear code that is relentlessly tested and easily enhanced.

But if we focus on today, we might encounter hard problems, the system might not work, it might be poorly designed. On the contrary, XP teams attack the worst problems first. XP teams design constantly, using CRC design. XP teams use comprehensive functional tests to show everyone that the system is working correctly.

But if we let the customer say what has to be done, the project might never get finished. On the contrary: XP teams provide a simple clear schedule and progress report that lets the customer make good decisions on what to leave in, and more importantly what to leave out, to release a successful product on time.

But if the team manages itself, people might not know what to do. XP teams use User Stories to know what to do. They brainstorm Engineering Tasks for each story. They sign up and take responsibility for implementing each task. And nothing is done until the tests run.