Book Review: Questioning Extreme Programming
eff, an experienced XP coach, takes a hard look at Pete McBreen’s book, and finds it to contain good questions. He feels that the answers fall short – perhaps due to Pete’s inexperience with XP. 
Questioning Extreme Programming
After reading Software Craftmanship by Pete McBreen, and being impressed by many of the ideals it contained, I was anxious to see his take on XP. I approached Questioning Extreme Programming with an open mind, hoping to glean some legitimate viewpoints that challenge some of the concepts of XP. Worst case, I would at least know what to expect in terms of resistance when discussing XP. After reading the book, I was confused and disappointed.
Tainting the book immediately is the author’s admission in the preface that “I have never worked on a full XP project.” This becomes clear with some of the brief diatribes against XP practices that are not borne out in their practical application. By the book’s conclusion, I felt that the biggest point of book was to give the author (and perhaps the reader) plenty of excuses to never do XP. McBreen foreshadows his conclusions early in Chapter 1: “You are likely to discover a difference between what you think is important and the items that XP assumes are important.” (p10)
On page 12, McBreen says, “Changing processes is hard. Attempt it only when the benefits massively outweigh the costs.” This is an obvious assertion, yet it is repeated a few times in the book. Another example from page 53: “If your current approach to software development is not broken, it is unlikely that you can recoup the cost of changing to a different process.” But McBreen doesn’t appear to believe that most software development is broken, so it is clear what decision he has already made with respect to trying XP. A very brief list of “signs that your process is broken” appears on page 9. This list seems small, again suggesting that perhaps the processes out there are working just fine. However, McBreen doesn’t mention many other signs that you may encounter, such as:
- team members don't improve in capability
- large amounts of dusty documentation lie around
- overstaffed or overworked help desk
- heavy volumes of defects; high-end defect tracking systems
- minimized communication; lots of headphones
- large number of chiefs involved in any given project
- disdainful comments from the business on the capability of the developers
- mistrust, evidenced by excessive "CYA" emails
- segmented knowledge that creates bad dependencies on individuals
From what I’ve seen, most software development is broken, or at least bent. But it is very hard for organizations to admit there is a problem, as Weinberg tells us in Secrets of Consulting. McBreen is suggesting that there is not a problem.
Chapter 4, “What do other methodologies consider important?” provides a good summary of the various development methodologies in use today. It compares and contrasts several examples of software engineering, open source, and agile processes.
Chapter 5, “What is important for your projects?” is an interesting discussion of how to assess the role of methodologies in your specific project. It ends with a list of key questions (p52) to ask yourself to determine whether an approach is right for your organization.
On page 60, the author talks about the XP planning game: “The accuracy of the estimates produced during the planning game needs to be investigated, especially for organizations that are just adopting XP. … I wonder how long it takes a new XP team to get good at the Planning Game.” Here McBreen misses the point of the heavy feedback cycles in the planning game. Initial estimates are going to be inaccurate in any process. In XP, the team has lots of opportunities to estimate and to learn how to do it well–the team hones their estimating skills every two weeks.
In any case, no other process provides a better solution to estimation. No matter how much planning, and how experienced a team is, software developers cannot predict the next hour accurately, let alone the thousands of hours that represent a typical project. The point of the planning game is to allow teams to get good at planning, something that is missing in processes that do not accommodate replanning.
Another theme in the book is related to XP’s attempts to fix things that the author admits are clearly broken. The author fears that certain changes just won’t happen, so why should XP even try? As an example, XP stipulates that developers have the right to make their own estimates, as opposed to being told–by a non-technical person–how long something should take. In response, the author states that “Imposed estimates are such an endemic problem in software development that I suspect that few organizations will be able to use the planning game successfully.” McBreen’s defeatist attitude is a key part of the problem.
Of the XP practices, McBreen reacts most critically to the short-cycle incremental design practices of simple design and refactoring. These disciplines emphasize constant care to ensure system design is always optimal, as opposed to producing a complete design up front and assuming it remains correct. But McBreen sees “parallels between undisciplined hacking and XP,” (p66) and appears to lean on the side of developers who view incremental design as “an inefficient, questionable practice, possibly bordering on malpractice.” (p68) Which is malpractice: presuming that significant up-front design is always perfect, or being vigilant and ensuring that the code and design always stay clean?
McBreen’s whole discussion on incremental design is where my confusion began. McBreen wrote an entire book, Software Craftmanship, discussing how what most of us do today is not software engineering; it is instead craft. The premise of that book is that software is malleable, and we can easily shape or craft it from nothing into a robust product. In contrast, engineering is presented as a paradigm where we do significant planning up front because either we must or because it is too costly to change things later. (Many real engineers will refute this statement, though.) And thus we do not engineer software, we craft it with care, largely because we are able to. Personally I loved the book.
Yet Questioning XP seems to refute the craft metaphor, as McBreen wishes for a process where we can do perfect design up front. He also wishes for developers not to be burdened by having to show constant love and care for their craft. Unfortunately, that’s what craft is all about: craft will fail if its practitioners are not disciplined. If McBreen wants us to believe him in Questioning XP, then he should refute his earlier conclusions in Software Craftmanship.
Besides, McBreen is just wrong, demonstrating his lack of personal experience with XP. XP does support up-front design–it’s just that it recommends that design always proceed in the presence of code. The more effort we spend on design without feedback, the more likely that our design is incorrect. Design is done during the planning game, in addition to being done constantly throughout development. XP by no means prohibits us from sketching out UML diagrams prior to beginning an iteration, a task, or a test case. But we learn as we repeat, and our need to physically sketch diminishes as our understanding of good OO design and the system increases.
The brevity and title of Chapter 8, “Are We Done Yet?” hints that the author is wearying of all the XP practices by this point. McBreen includes a brief discussion of test-first development and acceptance testing as part of the last few practices covered. This is unfortunate, since the implication is that testing is an afterthought in XP as well. In fact, these are the two technical practices that will make or break an XP effort, and they should really be viewed as the technical drivers in XP.
I also note that McBreen is inconsistent in his naming of the unit-testing practice, using “test-first design” once or twice earlier, and “test-first development” elsewhere. This is to be forgiven, since the XP community keeps changing the name. The preferred term is now “test-driven development,” which technically encompasses the older term test-first design–not test-first development–plus the refactoring practice. The important thing to note is the reason it is (was) called test-first design: writing tests first has a huge impact on design. Were McBreen to have gotten the name right and have done a bit more research, he could have helped explain in the previous chapter that test-first design is one more way that design is always happening in XP.
Some of the side arguments in the book are appalling. In the chapter “Truly Incremental Development,” McBreen argues that making code readable and maintainable will “dumb down” the software (p67). “Tight, well-written code … can look obfuscated to people with less knowledge of the programming language.” First, clever code has little place in modern software that must be maintained by people at all levels of ability. Second, XP does not promote dumbing down software to the level of incompetent developers. What it does promote is making code clear wherever possible. I’ve been burned by too many high-priced consultants, with an attitude like McBreen’s, who left behind unmaintainable messes. I’m sure most of you have too.
The discussion on clever code brings up the issue that there are few true craftsmen in the software development industry. The majority of developers are adequate at best. McBreen’s discussion on pair programming is brief, like the test-first design topic, and only scratches the surface of its huge role in an XP effort. Both practices do wonders for bringing up the level of competence on a team. Pair programming educates all developers far faster than individuals working alone in their cube. Even the most experienced developers learn more through pairing. Feedback from the quick cycles of test-first design also serve to bolster and reinforce the body of knowledge of a developer.
The bulk of the chapters in section 4, “Questioning XP Concepts” contain well-reasoned discussions of concepts such as test-first development, source code as design, and large teams. Chapter 16, “Requirements: Documentation or Conversation” re-emphasizes one of the author’s main concerns about XP: the reality of finding an on-site customer with enough knowledge to be able to accurately feed the development team. Indeed, it is difficult in many organizations to find such a resource. Yet, the ability to precisely pin down requirements is a key factor in making any project successful. Once again, the author seems to be dismissing an XP concept because it’s just too hard to try and fix things that even he admits are broken. If the organization values the ability to deliver what the business is looking for, then it must learn how to grow the body of people able to drive a project as customer.
In chapter 12, “Test First Development,” McBreen discusses the overall value of testing in the XP process compared to other processes. This is a valuable chapter that discusses automated testing vs. manual testing, as well as the relationship between the development team and the test team. McBreen admits that XP teams are producing software where “the quality of the resulting applications is better.” (p110) The chapter ends on the note that testers should be an integral part of software development, as XP promotes, “rather than after-the-fact commentators and critics, as in the traditional approach.” Excellent observation.
On page 130-131, the author states: “When all of the practices are being used effectively, XP teams report that their workdays are productive and enjoyable.” (emphasis added.) The good part about this admission is that McBreen answers his own question from page 66: “Is a high level of discipline sustainable in the long run?”
McBreen’s defeatism surfaces again in a discussion on documentation and XP maintenance teams (Chapter 17, “Is Oral Documentation Enough?”). McBreen admits that there is high value to acceptance tests and unit tests as documentation. But he doesn’t believe that maintenance teams will keep the tests up to date, based on his experience that shows him maintenance developers will not keep documentation up to date. Quite a leap of logic; nonetheless, why knock XP for something which there is supposedly no solution (if there is one, McBreen doesn’t tell us what it is).
Part V, “Understanding the XP Community,” talks about the divisive nature of XP jargon and developer culture. Granted, XP proponents are often overly enthusiastic, and are good at being provocative regarding other questionable practices, such as big design up front. Should we hold enthusiasm against them? McBreen seems annoyed that XP has its own terminology. What process does not?
Part V ends with a weird chapter that tries to come up with a way of yanking extreme programming away from developers, supposedly to force them into a more rigorous, heavyweight approach. McBreen states that this a difficult proposition because the developers love working the XP way so much, and because the customers love the productivity they get out of the development teams. If everyone likes XP, what’s the problem McBreen is trying to solve?
The final section, part VI, presents the author’s conclusions. The basic conclusion is that XP is too narrowly focused to be applicable to most people. McBreen presents a good checklist of ten criteria to use to determine whether XP is applicable to your efforts. But the answers given by McBreen do everything to convince you that the answer to most of the questions is no. Above all, fear seems to drive McBreen’s recommendations, and if we follow his lead, we are best off not changing anything.
Also included in the conclusions are assertions that aren’t discussed anywhere in the book. For example (p175): “If the majority of your projects involve writing life- or safety-critical embedded software, please don’t even think about using XP.” Please tell me why.
I appreciated Questioning XP, because it presents many good challenges that need to be addressed. But McBreen tarnished his effort with too many unverified claims. It also seemed like the primary audience was McBreen himself–perhaps he wanted to be convinced by book end that he had no reason to try XP. McBreen even hoped that XP will be soon supplanted by something else, so he wouldn’t have to hear the rhetoric any longer (p168).
In summary, Questioning XP did not appear to be backed by enough meaningful research or experience to provide a truly honest critique of XP. Its conclusions did not seem to be in line with the evidence presented in the rest of the book. However, I do recommend it for XP coaches–it does provide a thorough awareness of the issues that will be faced on an XP effort.