Fixed-Everything: Agile?

We take as our text this morning, the following Tweet from @allenholub:

I hear: the real world is fixed-scope projects, how can I do that in an Agile way. You can’t. Agile exists because scope always changes as work progresses. It’s central. Fixed objectives? Sure. Fixed time? Sure. But we learn as we work. Your contract must reflect that.

We are here to discuss¹ the notions in the above. We’ll start with the values of Agile, then move on to the principles and some Agile frameworks and practices. In an act of unusual mercy, I’ll not be covering every single notion in the Agile pantheon, just enough to make my point and then hammer it home.

Allen refers to fixed-scope projects, which we presume means efforts where there is a comprehensive list of capabilities (“requirements”) which must be met. He doesn’t mention fixed price and fixed time, but in my experience, one rarely sees a contract that reads “Do all this stuff in your own time and charge me whatever you want”. Let’s imagine a contract where all three of content, deadline, and price are fixed. This is a harder problem than Allen’s and if Agile works here, it’ll work in the simpler “fixed-scope” situation.

There is Still Variability

Change Control

Even in a fixed-fixed-fixed situation, there is always² variability. Most such contracts, in my experience, include “change control” provisions. One of the standard tropes used between the parties to such contracts is that the vendor will build something, the buyer will object that it doesn’t meet the scope definition, they’ll argue, with the intention of the vendor being to extract more money when it is finally determined that the original scope wasn’t clear and this is a change.

This trick is used quite cynically by a number of famous fixed-price bidders. They bid low, fully intending to make their money on changes. It seems to me that everyone must know this going in. I am sure the vendor knows it, and I strongly suspect that most buyers know it. They’re constrained to select the vendor with the lowest price, but once things are under way, they anticipate being able to get more funding once things start going wonky.

You’d have to be a newbie to fixed price contracting, or incredibly naive, not to understand this pattern. For the sake of argument, let’s assume that there are some naive newbies who don’t know this. It doesn’t matter: it’s still going to happen. There will be deviations from what they think they asked for, and there will be negotiations, and there will be change.

Design and Build Options

Given any desired capability, there are many ways to build it. There are many architectures, many designs, and many levels of robustness and quality (most of them less than one might want). Under the pressure of a fixed effort, choices on all those dimensions will be made by the vendor, attempting to maximize profits, minimize loss, get done on time, and so on.

The more onerous the contract, the more corner-cutting changes will be made. Many of these will be invisible to the buyer until it is far too late. Even more unfortunately, however, many of the corners cut will be invisible to the contractor as well.

When you put engineers and developers under pressure to deliver more than they can deliver at your desired level of quality, they will turn the secret screws in the development process, in an attempt to go faster by skimping on quality. This trick never³ works.

Missing the Deadline

Fixed contracts are notorious for missing their deadline. Let’s face it: that’s a change. Any theory that these deals are unchanging is mistaken. Not hitting the deadline is a big and horrible change. It costs everyone time and money and sometimes costs some people their jobs.

When it becomes clear that the deadline is going to be missed, what happens? Well, most commonly, we begin to cut scope. We begin by reducing testing. You can imagine how that works. Quickly, however, we need to decide to “go live” with fewer capabilities than we had hoped for. Which capabilities are dropped or cut? Whichever ones happen not to be done. In a non-Agile effort, that’s likely to be most of them, and certainly won’t be the chosen ones of least value.

Renegotiation

Sometimes, the trouble on the horizon becomes obvious early enough to permit a somewhat organized renegotiation of the effort. Sometimes scope is reduced. Sometimes the price is increased. “If you pay us more we can put more people on the project and go faster.” This trick also never works, by the way.

Nonetheless, quite often supposedly fixed projects get renegotiated one way or another.

Change Happens

Bottom line, these “fixed” efforts do have a lot of very common and very visible ways to change. Anyone who has done a number of these is likely to have hit some, if not all, of the above types of change. And there are surely others unmentioned.

Let’s move on to whether, and how, “Agile” might fit in.

Values

First, we’ll tick briefly through the Agile values and take a quick look at how each still applies in the fixed situation.

Individuals and Interaction over Processes and Tools

By the nature of the hypothesis, our fixed project is difficult. This Agile Value suggests putting good people on the effort, encouraging them to interact well, rather than just relying on whatever processes and tools⁴ you have at hand.

Working Software over Comprehensive Documentation

Now on your fixed project, you probably will have a cubic bunload of comprehensive documentation. You can’t compile and run that documentation to solve any projects, and while you might look at the Gantt charts or Rally Reports to see how you’re really doing, having a running tested version of whatever is really done so far seems like it might be a solid idea. I know I would prefer to see real software over any self-interested management report.

Customer Collaboration over Contract Negotiation

It was contract negotiation that got us into this mess, and even given the best change control clauses in history, it’s not going to get us out. What might help would be to build up and maintain a solid working relationship with the buyer, and to use that to focus on what’s really important, and to prepare them for the outcome in the best possible way: by showing them our working software.

Responding to Change over Following a Plan

Things are not going to go according to plan. (I know a few people who disagree with this, and feel free to believe them if you want to. My experience is that plans inevitably go off the rails.) We need to detect as soon as possible that we’re leaving the rails. We do that by observing the working software. We need to respond do that situation wisely. We do that by interacting with the individuals inside the project, and by collaborating with our customers.

It seems to me that dropping any of the Agile values from a fixed effort could only be to our disadvantage. I wouldn’t do it.

Principles

Let’s look at a few of the Manifesto’s Principles and comment a bit:

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

If we can build things and make them work early, we can begin to build up customer satisfaction well before the deadline. We might even find them completely satisfied long before the originally imagined scope has been completed.

If things go less well than intended, as is likely, the visibility will help, and the customer will at least get partial value from the effort. Partial value is better than a canceled effort.

Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

Agile efforts are better able to accommodate change. When there is change, this means the vendor’s costs will be lower, and there will be higher profit from any change control revenue that may accrue. And the buyer’s needs are met better and sooner. Win-win.

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

Fixed contract situations often slow down. Grand results are reported at the beginning, and bad news shows up at the end. An Agile approach produces better information throughout, as we’ve already talked about. But in addition, the Agile effort tends to operate at a more consistent pace throughout. This minimizes bad news and maximizes the ability to project results.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

We’ll return to this, but clearly a fixed effort needs all the help and all the improvement it can get, just to get out alive. This is an Agile principle I’d not want to miss out on.

Read all the Manifesto Principles. Think for yourself along these lines, considering how they can help. My bottom line in the Principles area is that every single one of the Manifesto Principles is one that I would want at hand in a fixed contract situation. If you can think of one you’d prefer to drop, please let me know: I’d like to understand your reasoning.

Processes and Practices

Scrum

Let’s start with a look at Scrum, which is pretty much the minimal “Agile” framework. I’ll leave comparison with a Kanban style to the reader, but it’s much the same.

Scrum would have you deliver a working, tested, shippable Product Increment every Sprint. (Think Sprint = week here in 2019.) We’ve talked about the value of having the actual software to tell you how you’re doing. This is where you get it. (Yes, it’s difficult. I know that. We’re not talking about whether Agile is easy: it isn’t. We’re talking about whether you should do it: I think you should.)

Scrum would have you plan your week’s work using no more than two hours of time. Five percent to plan a week. You can afford this. Can you afford not to plan your weeks? How long do you want to be out of control before you find out?

Scrum would have you spend another couple of hours, during the week, with a few of the team members, preparing for next week’s planning, making sure that the two hour planning meeting goes smoothly. You can afford this, and if you skip it, your two hour meeting will extend, probably to all day. Do the prep and your planning meeting can turn into an hour or less.

Scrum would have you show your running tested software to your stakeholders every week. This review should scale to about an hour. At that meeting, all the principals in your organization, and your customer representatives, find themselves on the same page as to what is really going on, and they get to provide feedback on how they’d like to see things change.

Scrum would have you spend another hour every week, looking back and finding ways to improve. What is “improve”? Oh, nothing important, just things like go faster, eliminate delays, reduce misunderstandings, reduce defects. I don’t know, on second thought this seems like a good idea.

Scrum also calls for a single voice guiding the team as to what to do next, setting forth the most important problems to solve, and letting them figure out how to solve them. Makes sense to me. Maybe you’d prefer a lot of people shouting orders and commands. If so, well, go for it. I’d choose the Agile approach.

You can look for yourself at the Scrum Guide. I think you’ll find that it contains a lot of ideas like the above, where if you ask yourself whether you’d rather have those benefits or not, you’ll come down wanting them. YMMV, I’ve only been doing software for a half century or so, so I don’t know everything

Technical Practices (XP)

Let’s take a look now at some famous Agile technical practices centered in Extreme Programming:

Acceptance-Test-Driven Development

This practice expresses each requirement as one or more concrete executable acceptance tests, agreed between customer and vendor. When the tests run, the feature is done. It seems this could eliminate a lot of confusion and argumentation, replacing wondering how things are with confidence that we know how things are.

Test-Driven Development

This practice provides that we write all the code, and only the code, that is required in order to pass a growing collection of executable checks. The result is that we gain very high confidence that the code is doing what we expect it to do. That is, we gain very high confidence that it is free of defects.

ATDD and TDD together provide a two-layer safety net that assures us, at all times, that the growing system does what we believe it does. These two practices are the way we provide the “Working Software” of the Manifesto and the “Product Increment” of Scrum. These practices may not be the only way to do this, but so far, they appear to be the best way we know of.

Refactoring

As any system grows, its design needs to grow with it. We might do design “in our heads” for more than we have built, but even then we can only put in the real design incrementally, over time. Some efforts might try to build an architecture or framework and then build the application on top. What often⁵ happens in this case is that too much time is consumed building the scaffolding, leaving too little time to actually build the application.

Refactoring is needed even in the framework-first approach, as more framework has to be levered in. And refactoring is known to be essential when we build the application and the scaffolding together, as we must do if we want the benefits of Working Software.

Pairing and Mobbing

It’s important to have our developers be as broadly capable as possible, and important to have them learning as rapidly as possible as the effort goes on. They need to learn the domain, the requirements, and, most importantly, the software. Working together via pair programming or mob programming serves this need marvelously, and produces more reliable programs as fast as, or faster than working individually.

As I think about dropping any of these Agile technical practices, it makes me shiver. How could I afford to wait a long time to see real working software? How could I afford to give up knowing that the product meets the requirements? How could I afford to slow down determining whether the code actually works? How could I accept slowing down as we have to adjust to and change the design? How could I accept building up silos of knowledge, making me vulnerable to people leaving, and creating queues and slowing me down even when people stay?

My bottom line: I don’t want to give up any of those things.

Fin

And really, that is the bottom line. Agile values, principles, frameworks and practices all provide benefits that are critical to giving us the best possible chance to succeed. They don’t guarantee success: nothing does.

I’ve been doing, leading, and coaching software for over a half century, and I would use “Agile” in every situation I’ve ever seen. It would be difficult, always, and very difficult sometimes. But I’d never give it up.

You, of course, can do as you wish. Good luck!