All software development projects have to deal with risk. Most projects are unique, containing at least one, if not several elements that are new. While companies and the ground they and their development projects cover certainly overlap, each have their own complex peculiarities:
The hard thing, I think, for a lot of us writing business systems ... is that it's very difficult to understand how to structure these business systems well. ... It's one thing to design GUI systems or operating systems or database systems; those are inherently logical things. But the way business is run, like say, a payroll system, is inherently illogical - and that's what makes it so much harder to do that kind of thing. (Fowler "Future of Software Development" 25:30)
Most companies are about money, not building a logical, consistent infrastructure. Doing anything for customers to make a buck does not usually yield a neat, tidy stack of business rules; the rules change as the opportunities for a profit change. We can’t complain too loudly lest we bite off the hand that feeds us, but acknowledging the difficulty of corporate development problems is important.
In addition, the people and technology resources used to solve these problems greatly vary from project to project. Finding people who can understand how the business works and manipulate the technology at hand, which often changes too fast, can be difficult and certainly is risky. Moving forward with a thick handful of unknown issues requires sound risk management.
Risk Management Strategies
Forbes magazine in August, 1997 published an article titled, “Resilience v. Anticipation.” In it, the author quotes UC-Berkeley political scientist Aaron Wildavsky, from his work titled “Searching for Safety.” Wildavsky lists two categories of risk management, anticipation and resilience.
Anticipation is a mode of control by a central mind; efforts are made to predict and prevent potential dangers before damage is done. Resilience is the capacity to cope with unanticipated dangers after they have become manifest, learning to bounce back. ... Anticipation seeks to preserve stability: the less fluctuation, the better. Resilience accommodates variability; ... The positive side of anticipation is that it encourages imagination and deep thought. And it is good at eliminating known risks. It can build confidence. But anticipation doesn't work when the world changes rapidly, and in unexpected ways. It encourages two types of error: hubristic central planning and overcaution. (Postrel)
Change in Software Development
Since the presence of change has an impact on the effectiveness of risk management strategy, how important is dealing with change in software development?
Change is a fact of life with every program. ... even during development of the first version of a program, you'll add unanticipated features that require changes. Error corrections also introduce changes. (McConnell 98)
On a typical sample of roughly one million instructions, a study at IBM found that the average project experiences a 25 percent change in requirements during development. (McConnell 30)
[A]ccept the fact of change as a way of life, rather than an untoward and annoying exception. ... the programmer delivers satisfaction of a user need rather than any tangible product. And both the actual need and the user's perception of that need will change as programs are built, tested, and used. (Brooks 117)
An anticipation methodology attempts to identify and solve as many, if not all, problems prior to coding. (Fowler “The New Methodology”) A full write-up of requirements is commissioned, followed by a period of architecture development. The hope is that the requirements and design stages will have uncovered all major problems that could occur during the subsequent coding phase. (McConnell 27). There are some problems with this. The customer probably has limited knowledge of the business. Business processes are executed by numerous people. The knowledge of the entire process usually only resides in the collective minds of the employees. Even if one or a few people has an inclusive grasp on the business processes, they usually have a limited ability to fully recall everything known about the business beforehand. “On a typical project … the customer can’t reliably describe what is needed before the code is written. … The development process helps customers better understand their own needs, and this is a major source of requirements changes.” (McConnell 30). Likewise, the programmers on a team have either limited knowledge of the technology to be used on the project or at least the same inability to fully recall every issue that will arise during the course of the project. Not so? Consider optimization of code. Today’s common sense approach of properly optimizing code is to not start with optimizations. Attempting to predict where bottlenecks will occur prior to coding has been shown to be fruitless in enough cases that it’s usually a waste of time. Waiting to profile finished code is a much more effective way of identifying bottlenecks (McConnell 681). The business itself can also change during the course of development. If a new client can be brought on board resulting in a 100% increase of revenue at the price of changing some internal processes, more than likely the choice will be made by management to pay the price, and the programmers will be the one to pay it. Anticipation seems a poor choice in software development, because it depends on an unrealistic amount of mental ability and a small amount of change. (There are some development situations where the technology is not new and the customers and programmers both have prior experience with a very similar project. In these cases, an anticipation approach is more feasible). Adding to the problem is the fact that change within an anticipation methodolgy can be quite expensive. “Data from TRW shows that a change in the early stages of a project, in requirements or architecture, costs 50 to 200 times less than the same change later, in construction or maintenance. Studies at IBM have shown the same thing.” (McConnell 25). Anticipation methodology has goals that are both hard to meet and expensive if not met.
Handling Changes During Development
If anticipation suffers from putting too much distance between the time a problem is created (e.g. requirement not discovered), what are some approaches to dealing with this?
Steve McConnell lists 5 ways of handling requirements changes during the coding phase of a project. Oddly, 3 of them don’t really address handling a change during the coding phase. The first suggestion is to use a ‘pre-flight’ checklist of common requirement categories to ensure the requirements are good. The second suggestion is to disuade the customer from making changes by rationally explaining how expensive a change can be. The last of the 5 options is to “dump the project”! (McConnell 31).
The two items that actually involve changes during coding ironically reflect a resilience risk management approach rather than anticipation. He recommends setting up change-control procedures as well as adopting a prototyping approach. (McConnell 32). In fact, support for iterative development techniques abound:
Evolutionary delivery is an approach that delivers your system in stages. You can build a little, get a little feedback from your users, adjust your design a little, make a few changes, and build a little more. The key is using short development cycles so that you can respond to your users quickly. (McConnell 32).
One always has, at every stage in the [iterative] process, a working system. I find that teams can grow much more complex entities in four months that they can build. ... one of the most promising of the current technological efforts, and one which attacks the essence, ... is the development of approaches and tools for rapid prototyping of systems as part of the iterative specification of requirements. (Brooks 199-201)
The most important .. part [of handling change] is to know accurately where we are. We need an honest feedback mechanism which can accurately tell us what the situation is at frequent intervals. The key to this feedback is iterative development. (Fowler "The New Methodology").
McConnell also has a section in his book Code Complete covering Evolutionary Delivery (664) which further illustrates iterative development.
XP is Resilient
It's unfortunate that no one has ever written a book about incremental approaches to software development because it would be a potent collection of techniques. (McConnell 654)
Kent Beck, one of the founders of XP, authored an introductory work titled Extreme Programming Explained: Embrace Change which attempts to do what McConnell wished for.
XP has four values which serve as a foundation for the XP practices.
XP encourages extreme communication. If the customer needs to see the program in action to fully formulate requirements, put the customer on the development team and churn out working versions every 2 to 4 weeks, developing requirements along the way. If the programmers need clarification on requirements, the customer is a member of the team – lean back in your chair and ask the customer. Programmers work in pairs: two people to every one machine. Pair members rotate regularly. All code is owned by the team, not by individuals. This promotes communication of technical knowledge throughout the team. When technical challenges arise, the team is more able to address the problem. Pair programming also is excellent in matching up programmers of differing abilities. Less experienced members are constantly mentored, and the risk of less experienced code being added to the application is minimized.
The faster change is identified, the faster it can be dealt with. “In general, the principle is to find an error as close as possible to the time at which it was introduced.” (McConnell 25). XP strives for the fastest feedback possible in every aspect of the project. Unit Tests are written for most every piece of production code. Unit tests must run at 100% before any code is checked in. When production code needs to be changed, side-effect problems are identified. After new code is checked in, it’s immediately integrated with the latest changes from the rest of the team and again, all unit tests must be made to run at 100%. Acceptance Tests are written with the customer to verify the application is doing what the customer needs it to do. Pair Programming provides constant code reviews. No more dreary code review meetings – put two sets of eyes on the code as it’s written. Collective Ownership of the code by all members of the team helps ensure even more eyes will see the code, increasing the amount of code review performed.
The customer is the ultimate driving force in XP. Do what the customer needs, as simply as possible, and nothing else. Take on a mindset to reduce complexity from the beginning. All code should be refactored as often as possible. Refactoring is a process of improving a code’s structure without changing its functionality. Refactoring produces highly decoupled objects which makes them easy to test, easy to use, more flexible, and therefore, more changable.
No more fragile code. With smooth communication, quick feedback and simple code, programmers have the support they need to dive into changes when they come … and they will come.
Beck, Kent. Extreme Programming Explained: Embrace Change, Addison-Wesley, ISBN 0-201-61641-6, 2000.
Brooks, Frederick P., Jr., The Mythical Man-Month, Anniversary Ed., Addison Wesley, ISBN 0-201-83595-9, 1995.
Fowler, Martin. “The New Methodology”. 12 Sept 2000 https://www.martinfowler.com/articles/newMethodology.html
Future of Software Development, The, video stream from Dr. Dobb’s technetcast of Software Development 2000 West panel discussion, https://www.technetcast.com/tnc_play_stream.html?stream_id=227
McConnell, Steven C., Code Complete, Microsoft Press, ISBN 1-55615-484-4, 1993.
Postrel, Virginia I. “Resilience vs. Anticipation” Forbes ASAP 25 Aug 1997. 11 Sept 2000 https://www.forbes.com/asap/97/0825/056.htm <hr>(c) Chris Morris, 2000. All rights reserved.