Incremental Development
@GeePawHill tweeted a thread, later blogged in a more readable form here.
Embedded in the Twitter thread is a conversation between me and Ihe Onwuka. I’m sure I can’t do Ihe’s ideas justice, so read the thread, but the basic notion I got was that Ihe supports a lot of serious design, external to the coding of the program, and I … well, not so much.
Anyway, that exchange inspired this article. Here goes:
The story of Big Design is that we have some carefully examined set of perfect requirements, and that we create, using some fairly formal method, a beautiful design for implementing those requirements. That design, shining like a golden sun, illuminates the programming world, so that the programmers can, more or less without deep thought, build the perfect implementation of our perfect design for those perfect requirements.
Even if this were true – spoiler alert: it is not – Big Design wouldn’t meet our needs in many cases. I would say it doesn’t meet our needs in any cases, but it certainly doesn’t meet our needs in the case I care about:
Have Working Software Always
I’ve learned many things in my over half a century of building software on my own and with teams. If I could go back in time and take only one idea with me, it would be this:
Whatever your product is, provide real working software every two weeks, every day if possible.
Almost every bad situation I’ve found myself in, in over a half a century of doing this stuff, would have been substantially improved by having a real working version of the product, with reduced functionality of course, always in hand. We can discuss elsewhere why that’s so, but the simple explanation is that it would have given me the ability to show progress, and to change the conversation from “do more” to “here’s what’s next”. We’d have shipped the good ideas sooner and cancelled the bad ones sooner.
So, to me, the big development question becomes, OK, how can we do this software every week or so thing? And it’s clear on the face of it that Big Design. as I’ve defined it here, just can’t do that. We can’t get all the requirements and a good design in two weeks. Therefore, that kind of Big Design is a non-starter.
So how can we do it?
We require a good big design
Here’s a seeming contradiction, and it’s one that keeps a lot of people from considering incremental development seriously. It’s always “Our problem and solution are so large and complex that we absolutely require a good big design. Therefore we cannot deliver working software next week”.
What those people might have been better off to say is “Our problem and solution are so large and complex that we absolutely require a good big design. But we don’t know how to do that while deliverin working software every week. In fact, we think it’s impossible.”
Well, in my very thoughtful and experienced opinion, it’s not impossible. Since it’s not, and since I need working software every week or so, I have to figure out how to do it.
But one part of what those people say is absolutely true:
We absolutely do require a good, big design, for a large and complex problem and its solution.
So how can we possibly do those two things, starting from zero, building software every week or so, and wind up with a good, big design? Thanks for asking. Here’s a sketch of how we can do that.
Incremental Development and Design
We choose to have our development drive our design, and our design drive our development. We do development and design at the same time. It goes like this:
Informed by our vision of the requirements – not fully fleshed out at the beginning, but the parts we can see – we build a tiny fraction of the product, the part we can do in a week or so. As in the first little square in the picture, the features might not even fully work yet.
In the second iteration, we make the features, however spare, really work, at least most of them.
In the third, we add some more key features, and get everything working. Moving forward, we repeat. We add the next few most important features, as we then understand them, and we keep the whole thing working, all the time.
What could possibly go wrong?
Well, since we know that a good, big design is necessary, what could go wrong is that we wouldn’t have a good design, and progress would grind to a halt, while defects shoot toward the sky, because that’s what happens when you have a bad design. It would look like this:
Obviously, the project shown above is doomed. Its design has gotten more and more crufty until it is hopelessly tangled up. We’re surely screwed.
Make it go right!
Here’s the trick.
- It’s easy to have a good design for no features;
- When we build the next feature, we will mess up our good design;
- Because we just added one feature, and just took a few days to do it, we can’t mess the design up very much;
- Before we call that feature done, we will improve the design, back to good;
- Rinse, repeat.
The product grows; the design grows; the design stays good.
We wind up with a good, big design. The progress looks like this:
The process of moving from the inevitably crufty design, to a good design, is called “refactoring”, of course. The patterns and practices are well-documented.
But is there enough time? Of course there is enough time. If the process of building a feature is to make it work, then make it right, then features take the right amount of time, and the design stays good. If we try to speed up feature delivery by cutting down on making it right, well, then the design will deteriorate, we will slow down, and we’ll inject more defects as the code becomes less and less clear.
Don’t do that.
But won’t we go too slow if we do this? Wouldn’t we go faster with more Big Design?
Well, on the face of it, the full Big Design approach doesn’t deliver any software until the requirements are all in and the design is all done. We’re delivering little bits of usable software every week or two. We have an incredible head start, often amounting to months compared to Big Design.
We also have the advantage that our stakeholders and customers can see the actual product forming. They choose the features that seem next most important, and they can try all the features that they choose, so that when, inevitably, they choose wrongly, they get early feedback and can change direction.
This is the “Agile Way”, in my view. It’s never easy – they never seem to give us easy problems to solve. But it is the best and most survivable way that I know of, and believe me, I have studied them all and tried most of them.
You get to choose how much software to deliver, and when. You get to choose how much analysis, how much design to do before you start, and at any point along the way. That’s up to you.
What I would do is find a way to deliver working software every few weeks. I’m quite confident, based on two decades of work in Agile Software Development, that it can be done.
Since it can be done, I’d do it. You … well, you do you, but you can guess what I’d advise were I giving advice.
Related articles: