I’m not planning to produce a new brand-named framework, but I am interested in ways of deriving, expressing, describing, and developing the ideas that surround “Agile”1 software development. I’ve been running on at length about the “Increment”, and today I’m wondering whether the Increment could be a driver for successful software development in the “Agile” style. Walk with me – we’ll explore the idea.
Here’s a draft definition:
At regular intervals, preferably every week or two, the developers provide an integrated, running, tested version of the software, the “Increment”, containing all the features and capabilities which have been completed so far in the project. The Increment must be in usable condition and must meet all the agreed standards of completeness and correctness.2
In each interval, the stakeholders provide priorities for what’s next and the developers select what they’ll do next from the front of that list. At the end they review the Increment with stakeholders, showing what they did.
Driving Good Software Development from the Increment
Let’s look at some issues that an organization might have with software development, and explore whether attention to the Increment might help them. If your favorite issue isn’t listed here, please drop me a email and I’ll see about adding it. My current plan is to update this draft several times, to see where it leads.
We never know what those software guys are doing or when they’ll be done.
This complaint could come from anywhere. The part of the organization paying for the software is a likely source. I’ve had my company’s sales people complaining that they didn’t know when promised things would be done. The managers of the developers often express this concern.
When I hear a concern like this, I ask whether the developers are providing an Increment. The usual answer is “a what?”. I ask whether, every week or so, the developers are providing a running tested integrated version of the product that the person with the concern could look at. The usual answer is “no”. We explore then whether, if the developers did provide such a thing, the questioner would better know what’s going on and have a better sense of when things would be done. Generally the answer is “yes”.
If the organization is doing Scrum – and they usually are, because that’s the kind of person I usually hang with – I inform them that they have a right, and a duty, to insist that they be shown working software after every single Sprint.
The Increment, it seems to me, would quickly start solving the “we never know” problem.
We have an increment, but we don’t know what’s in it
Are you and the other stakeholders meeting regularly with the developers to see what is in the current Increment? If not, well, you’re supposed to be. If you did, you’d probably know.
We have an increment, but it doesn’t have what we want in it
Are you and the other stakeholders meeting regularly with the developers to guide what they do next? If not, you’re allowed to do that. In fact, the rules of the increment require you to do it. If you did that, you’d probably have suitable influence over the product.
We meet with them but they don’t do what we ask
How are their priorities being set? Is there an ordered list of what to do next? Are the developers working from the front of that list? If there is no list, you stakeholders need to devise a way of created an ordered list. Often this is done by putting a single individual in charge of getting the best possible value from the team. We call that person the Product Champion.
We have an ordered list, but the developers say first they have to build infrastructure or architecture or something about frameworks and stacks and Dockers or Nikes or something. They tell us they’ll work on our list after that.
That’s interesting. It turns out that using “old-fashioned last-century” development techniques, we used to do a lot of architecture and infrastructure and framework kinds of things up front, and then, we hoped, we’d build the features in later, when it would be “easy” because of the wonderful all-powerful architecture infrastructure framework. It turns out that almost never worked.
What we know how to do here in the 21st century is to deliver business capability every couple of weeks, while building up the underpinnings as we go. I say “we”. What I mean is that there are known and practical techniques for doing that. Quite probably, your team doesn’t know how to do that. If they’re telling you that those ideas won’t work in your context, you know they don’t know how to do it, because I know people who can and do deliver Increments in exactly your context.
So your team needs help. Beating on them probably doesn’t constitute help. Providing them with training, coaching, members who do know how to do these things, and time to learn, probably would constitute help. It takes some creativity to turn a no-Increment team into an Increment team, and it won’t happen overnight. With help, it should begin to happen quickly. Provide that help.
The team can’t really do very much in only a couple of weeks. Our needs are large, so it can take months before they have anything worth looking at.
I’d like to suggest that you’re conflating two ideas, “worth shipping”, and “worth having”. Many large features, suitable for shipping, do take up more than one two-week iteration. But that doesn’t mean that the bits done in each iteration don’t have value to you. Your first responsibility in providing that ordered list of things to do is to learn to make those things small enough to be done in a couple of weeks (and a couple of days is better, and usually possible). And each of those things you ask for needs to have visible value to you. Don’t settle for two weeks of getting the database ready or something. That’s last century again.
Think of some feature you want. Maybe we’ll do an example later but pick one of your own for now. Pick something important, that your experience tells you they can’t do in two weeks. Now look at that big feature and pick out some things that you care about, that are somewhat separable. Try to think of two that are of utmost importance in your big feature. Tell yourself that without both of these, the thing is worthless. Therefore, nothing of value to you can be done in two weeks.
On the contrary. Now imagine that some time has passed, and the team has done everything for you except the second of your two important ideas. Without that idea, your product is worthless, no one would use it. With that idea, people will use it and find it valuable. Do you see, now, that that second important idea has value?
Now turn things around. They build everything, including the second important idea, but not the first. The product is worthless, no one would use it. But if they’d just put in that first idea, the product is valuable! Do you see, now, that the first idea has value?
Your job is to have ideas such that each one has value – because they all add up to have value and therefore each must have some – and to slice those ideas down until they can each be done in a couple of weeks or a couple of days.
Those are the ideas you put on your list of things to do. Each one has value to you – because it contributes to the overall value of the product – and therefore each one is worth your thinking about and reviewing after the two-week interval where it gets done.
Our job as stakeholders is to make every Increment valuable to us, by selecting things that can be done in a single Increment and putting them at the head of the list.
They’re showing us what they’ve got, and it’s good, but they’re not going fast enough. We need to go faster.
Odds are, they’re going as fast as they know how to go. If you push them to go faster, internal quality will probably decline, and defects will rise, causing rework, and as the code gets dirtier, you’ll slow down further.
Your first job as stakeholders is to observe the real progress, and make your decisions based on what happens, not what you wish. It would be easy to get a perfect product if you could just wish for stuff and it happened. To accomplish that, find a Genie somewhere and make some wishes.3
Absent a Genie, your first strategy in response to real progress is to be sure you are picking the most important things to do, and to slice them down as small as possible. Your job is to get the most valuable possible product, at the end of each iteration. You do that by carefully selecting what to do and what to defer.4
As a secondary job, after prioritizing, you might also want to arrange for development to “go faster”. This is not trivially easy. Beating on the team probably doesn’t constitute help.5
Possibly the team needs to slow down. Paradoxically, when they try to go too fast, they leave dirt in the code and this leads to rework, and slow work. You and your team may need coaching on this topic.
Possibly the team needs skills that aren’t available on the team. Maybe they are doing a lot of database work but none of them are expert. Maybe no one on the team understands the legacy system you’re trying to interface with. Maybe you’ve thrown some new framework at them and there are no framework experts on the team.
Maybe you’re providing the necessary support using a separate group. Maybe there’s a framework team or a database admin department. Great, you’re probably saving a lot of money while you wait for important, valuable, money-making features to be done. Take a look at the “value stream”, the flow of features through your team. Note in particular all the places where things stall while they wait for something to be done, something to be decided, someone to arrive to help them. Consider whether you could get more of what you want by putting all the necessary skills into the team. If the answer is that you can’t afford it, well, then your problem isn’t that they’re going too slow, it is that you’ve decided not to go faster.
Possibly the team needs to have higher skills. Building software Incrementally isn’t easy. They may need training or coaching. There may need to be investment to get more speed.
Possibly, though seldom, more people would help. If you have a dedicated team of a half-dozen skilled people, really cranking out feature-filled Increments on a regular basis, perhaps you could add a second team. This approach is called Feature Teams and it has been understood for 40 years or more. Mind you, if you have multiple Feature Teams, they will need to learn not to step on each others’ feet.
When we go to add a new feature that they’ve asked for, it’s usually large and complicated and takes a long time to put in.
A large feature is almost invariably made up of lots of small features. Take a look at the acceptance tests6 for the feature. Is there more than one test or does it contain a lot of paragraphs? Those are seams where you can break that feature up into smaller ones.7 Break big features down until you can do them in a couple of days, far less than a couple of weeks. This is generally possible, even in your context.
We do what they say at the beginning of the iteration, but at the end they don’t like what we did. They seem to be changing their minds.
Well, if they see what you did and don’t like it, one possibility is that you did exactly what they asked for and now they’ve learned that they don’t like it. That’s a good thing, that they’re learning. It’s not a good thing if they act like you made a mistake not doing what they would have wanted if they had been as smart two weeks ago as they are now. In this case, we need to be sure we have a solid shared understanding of what will be done before we start. This can often be done using concrete acceptance tests and perhaps some simple but clear drawings of the user interface.
Another possibility is that you didn’t do what they asked for, that you somehow didn’t understand the idea. This, too, can often be resolved by better tests, better sketches, and a bit more conversation about what’s being asked for.
It’s important to move beyond “blaming” and arguing about whether they did or did not ask for what they want now. Often it helps to have a team rule saying “if the agreed acceptance test runs, we did what we were asked to do”. We’re still not trying to affix blame, we’re trying to see clearly what needs to be improved. Does the test need improving? Did we learn something? What happened? Looking at the expected Increment and the actual Increment can keep us focused on reality and help us understand how to improve.
Even the list of small features they give us takes more time than a couple of weeks. We just can’t produce one of these Increment things in our context.
Forgive me if I’m a bit impatient with this concern. It’s because I’m really confident that when you get it together, you can produce an increment in a couple of weeks. Often I approach this concern this way:
Q: OK, what if you only took on half as many of these small features. Could you produce an increment then?
A: No, they would still take too long.
Q: OK, what if you only took one one feature, could you do it then?
A: No, even one would take too long.
Q: OK, can you build a version of the software with no changes in two weeks?
Q: OK, let’s work on your build, and then on a small change like fixing a spelling error in a message.
When we go through paring down this problem this way, we look at the flow of no features, to see if we need to fix the build. Then we look at one feature. Often we’ll find a delay because we have to wait for something. We fix that, by deciding not to do features until whatever we’re waiting for is ready, or by doing the work ourselves. If “they” won’t let us do the work, then the problem with delivery isn’t in development, it’s in “they”. Let’s sit with “they” and sort this out. Either “they” want effective development, or they should quit complaining that they don’t have it.
Sometimes as we do this exercise, we find that if we only did one feature, then some of the team couldn’t be kept busy. This is often a sign of too much specialization. Maybe for each feature Bill has to do the GUI and then Sam has to build the middle tier, and then Suzie has to hook it up to the database. This takes one day for Bill, and two for Sam and Suzie, if we’re lucky. So if we only did one story, most everyone would be waiting, ultimately, on Suzie. So instead we pile in more than one story and make sure everyone is busy.
If this worked well enough to finish one feature per week, I might not object too much but it seems really dumb to wait three weeks to get a feature done. Note that if you didn’t need to serialize these things, you could do them without handoffs and get things done only a week after they’re asked for instead of three.
The Increment is telling you there’s a better way. It’s up to you to find it and decide whether you want to do it.
OK, I’m tired.
Enough for now. It seems we can use our focus on the Increment to discover better things to do. I’ve not driven all the way to specific practices, but we can see planning, and the idea of the Product Champion showing up. We can see Cross-Functional Team, and Acceptance Tests. It won’t be hard to derive Programmer Tests.
What do you see in your striving to do “Agile” that I’ve not mentioned? Extra credit for things that a solid Increment just wouldn’t help with. Drop me an email or a tweet!
I’ll revise this later and probably move its date forward.
You’re probably wondering why I so often put the word “Agile” in quotes. This is my passive-aggressive way of responding to the fact that the term means many things to many people. I myself use the rough definition that Agile software development is an approach that is in good accord with the values and principles of the Agile Manifesto. ↩
This definition is intentionally quite close to the definition provided by Scrum, in the Scrum Guide. This is one of the many things which Scrum gets right, in my opinion. ↩
Watch out, though. The Genie thing almost never turns out well. ↩
I first learned this phrasing from Kent Beck. ↩
This notion may have come up before. Seriously, beatings will not improve morale, and pressure will not give you a decent product sooner. We’ve known this for decades. Try to remember, 21st century, OK? ↩
Don’t have acceptance tests? We’ll talk about that elsewhere, but I bet your stakeholders often question whether you’ve done the thing they wanted. I bet you get into long arguments about better requirements documents. ↩