The State of Agile and Scrum

My first computing mentor, Bill Rogers, got me started in programming when I was an intern at Strategic Air Command, quite likely before you were born. As you can imagine, things got odd in a civil service organization inside a military one, and when the going got weird, Bill used to say “You have to either laugh or cry.” Though he never explained further, it was clear that he preferred to laugh at the idiocy that often surrounded us.

Let’s talk about the state of Agile, especially Scrum.¹

Prologue

When I was introduced to Agile ideas, it was through Kent Beck’s Extreme Programming ideas. This was in 1996. I had been programming for more than three decades at that point. I had worked hard to keep up, and and studied and tried most every new idea that came along in the programming business. I had had some great successes and some amazing flops.

Some of the flops, I feel sure, were my own darn fault, mostly relating to not knowing how to build things in a way that pleased management. Most everything we worked on was doing well, and doing what they wanted, but they couldn’t see our progress and I couldn’t convince them things were on track. We used big-bang first releases, of course, and they always came too late for management’s satisfaction. As I say, my mistakes.

Some of the flops had to do with issues that were more about management above me and parallel to me. Strategic directions would change. Marketing would get some new wild hair. These seemed at the time to be outside my control, even though I was pretty much informed on the thinking of my fellow executives. I’d push back against the changes, but sometimes they would happen. I suppose these were my fault as well,² as I look at it now. I could have read the writing on the wall, shifted technical direction, and so on. Or I could have been more convincing.

Nonetheless, I kept trying to learn, trying to do better. I love my profession, whatever it is, despite often not enjoying the actual job as much as I might.

An XP Awakening

When the C3 project and XP came along, the team immediately fell into a rapport with our “Customer” that I had never seen before. She would pick a few next things to do, and a few weeks later, we would demonstrate them. We started from nothing, and evolved a very powerful architecture, using tests that told us that features were working, tests that told us objects and methods were working, and by continually refactoring the code to make it better and better. We worked together in pairs, and as a group. We all worked on anything and everything in the whole application. There were specialists in that there were people who knew more about some area of the domain or the code, but because we paired, and worked together, everyone could work effectively in every area.

It was more fun, and more productive, than any way I had worked in the past, and I had worked in just about every known style and language. It was really fine.

My naive belief was that if you just did the things we did, you, too, could have those fine results. As it turns out, after a trip around the circle, I still believe that, but it’s trickier than I thought. For example, after changing out Every Single Manager But One on both sides of the C3 project (IT and Finance), the project got canceled. Sixteen million dollars and several years later, they managed to mostly duplicate on PeopleSoft what we had already done.

Agile “Failure” Modes

The Chrysler story turns out to be one of the standard “failure modes” of Agile: everything is going fine and then a tsunami of new management hits and it’s all swept away.

Another standard failure mode is even more sad: the organization never really does it. Let’s look briefly at each of these.

Swept Away

I hear more than a few stories of Agile / Scrum teams who are going along just fine and then a change of management comes along and the new broom sweeps it all away. I hear fewer such stories about XP, mostly because there’s not much XP going on within my hearing (Hi, Rachel!), so I don’t know if it happens there as much, or not. There’s no doubt in my mind that it could happen anywhere.

If an organization is doing Agile/Scrum/XP well, their progress is totally visible, they have close interactions with their stakeholders, and they’re doing exactly what stakeholders have been asking for. Surely this is about as good as you can hope for in the times when new management comes flying in with Wagner in the background. It might not be good enough, but then nothing is always good enough.

Never Had It, Never Will

This, of course, is the “state of Agile” that we all deplore. There are teams and organizations who say they’re doing Agile – usually Scrum – but who really are not, for whatever reasons.

What do I mean by really doing it? Here are some things you’re supposed to do if you’re doing Agile, drawn from the Agile Manifesto, in my own words and emphasis:³

• You have direct contact between business people and developers, every day.
• You work is guided by the business people selecting the next things to do, based on their perceived value of the things.
• You get those things truly done and shippable, every couple of weeks.
• You work with the business people, looking at the completed work, to decide what to do next.
• You review how your process is working, and you improve it, every couple of weeks.

If you’re doing these things, you’re at least on the road to doing Agile / Scrum / Whatever fairly well. If you’re a long way from these, you might still be doing great, if not in the “Agile” style. If you’re not doing great, well, I might advise from here that you try doing those things, in particular looking at how you have done every couple of weeks and truly improving what you do. You know, not like “try harder to write fewer bugs” but like “test as we go”. Like “get with stakeholders every Sprint” and come on, how about having a real accountable business-side person guiding what you work on?

The goal of a team should be to do well at whatever they’re supposed to be delivering, not to “do Agile” or “do Scrum”. Those are means to the end, if what you’re supposed to be doing is shipping software. Useful means, but not the only way to go and not the only way to succeed.

Are these failures? If so, what kind?

I want to suggest here that when things go badly, it’s not really helpful to say “I failed”, “we failed”, “they failed”, or “it failed”. So whether the team failed, the company failed, or “Agile”, done well or poorly, failed, I find that word too final. I also note that people tend to choose “they failed”, or “it failed” far more often than they choose “we” or “I”. Just remember, when we point our finger at someone else, three are pointing back at us.

So I’d suggest not saying “X failed” at all. You go ahead, though, if you like.

Well, these are sure not good things. What are they, then?

Well, if you were scoring well on the bullet points above, and your process was swept away by new management, I guess it’s safe to say that you’re not doing Agile any more. Some might want to say “Agile failed”. I’ve mentioned elsewhere that unless you had been planning to say “Agile succeeded” when everything went well, it might be better to look closer to home. But, yes, being swept away is not a good thing for a good process.

If you thought you were scoring well on the bullets, and old management swept the process away, there’s definitely something to look into. They must not have been happy for some reason, and if it was unhappiness with how things were going, I’d want to ask about your frequent delivery of truly shippable software, your frequent demonstrations of completed work to your stakeholders, and the daily connection between the business stakeholders and the team, where they select things for you to do every two weeks. If “Agile” got swept away with all those bullet points running in the green, I’d like to know a lot more about what happened.

In these situations, you can say “Agile failed” if you like. Perhaps “we failed at Agile” would be more accurate: think about it. Or even “we tried to do Agile and failed to do it very well”. Only you know how well these notions apply in your situation.

If, on the other hand, you were working toward those bullet points, but never really got into the green, I honestly think its fair to say that, while you may have been trying to follow an “Agile” approach, you never really got there. Again, call it “failed” and put whatever noun or pronoun on the front that you care to. Me, I might tend more toward “well, that wasn’t the outcome I had wanted, I’d better try again, fail again, fail better.”⁴

“You’re just saying Agile can never fail.” – Somebody, probably

Quick, someone bring up “No True Scotsman”!

No, I’m not saying that at all. When could “Agile” fail? It could fail in the same way that Lean could fail, or Christianity, or Democracy, or Zoroastrianism. Those things are all ideas, complex nests of ideas, and they can only fail when they vanish from the face of the earth. There’s an old science fiction / fantasy trope where gods are alive so long as they have at least one believer, and then they die. That’s how abstractions die.

This is quite different from Christianity failing in individuals who do not follow its dictates, or Democracy failing in a country where the elections are fixed, or Agile failing in a company that never adhered to its bullet points. Agile’s not dead – yet. It’s not dead as long as some people try it, follow it, and find value in it. When it has no more adherents, then it will die. Agile will fail.

What about all the flaccid Agile out there today?

Certainly it’s not a good sign when there’s “Agile in name only” all around us, just as we should be concerned about “Democracy in name only”. But it was ever thus: good ideas are adopted and used well by a few, while others do not walk the walk at all. But it is a good sign when, here and there⁵ we find pockets of people and organizations who are making excellent use of the ideas, principles, and practices that we had in mind back when we wrote the Manifesto. Agile is alive, so long as those things happen.

I try to be happy about the successes, and try to help people who are trying to get there. I try to help people who are interested, with ideas that may inspire them or assist them in trying things. The ones who want my help know my email and can get in touch.

The perpetual nay-sayers, and the innumerable flaccid, would-be, poser Agile projects? “You have to either laugh or cry.” I’m not here to cry about the fails, or flaccidities, or whatever you want to call them. I’m here to laugh in joy with the people who’ve tried these ideas and found them valuable.

Hey, can’t save everyone. I did save this one kitten, right here. A couple of you have made good use of what I helped you learn about. That’ll have to do.

Yeah but what about the state of Agile?

It’s good here. It has ideas that are useful everywhere, if my half-century of experience is any evidence. Try it, learn it, build on it. Or don’t. It’s up to you.