Dark Scrum: Is It 'No True Scotsman'?

A Medium article with an interesting mix of observation and questionable opinion inspires me to write what I hope will be the same.¹

Fair warning: this baby is long and rambling. Maybe someday I’ll boil it down to three Tweets, but for now this is what I’ve got.

In the article linked above, the author, one Jan Wischweh, argues “the currently perceived shortcomings of Agile and Scrum are not just ”faux scrum” but inherent to the agile approach as practiced today and a superficial understanding of the nature of Agile.” We’ll discuss some details, probably mostly in the order of M. Wischweh’s article, and I expect to find some agreement and some areas where we differ. We begin by agreeing:

I agree that [many of] the currently perceived shortcomings of Agile and Scrum are inherent to the agile approach as [often] practiced today, and to a superficial understanding of the nature of Agile. I added “many of”, because I have no doubt that some of the perceived shortcomings of Agile and Scrum are present in the purest form one could find. No work of humanity is perfect.

I emphasize “as often practiced today” for two reasons. “Often” is a quibble with which I hope Wischweh would agree: sometimes Agile and Scrum work out just fine. Often, they do not.

The emphasis is also there because, in my view, Agile and Scrum are often practiced improperly today, with Agile and Scrum taking the blame for bad results. When those results are bad enough, we move from Faux Agile to Dark Agile. Jan Wischweh refers to the “No True Scotsman” defense here, which I’ll address shortly.

In any case, I do agree that there are real problems in current would-be Agile and Scrum practice. Where we may differ is that I am quite confident that many of those difficulties stem either from failing to do what the method you’re using says, or, in some instances, because practice can’t overcome values, and the organization’s values oppose effective Agility.

Let’s explore further.

Post-Scrum; Myths; Foundation

Wischweh intends to “back up the call for a post-scrum era² in software development, address some myths in the current perception of Agile and will try to build a theoretical foundation for developers not happy with the current state of our industry”.

Post-Scrum

I’m honestly not sure about whether a “post-Scrum era” is what we need, but frequent readers are surely aware that I believe that Scrum often goes wrong, mostly for a few common reasons, and that I believe that developers can improve their lives within real or fake Scrum environments.

My ideas on how I’d improve my life in a fake Scrum situation revolve around using Agile practices (specifically XP and Scrum) to produce tangible software increments, and to use those increments to change the conversation with management, to the mutual benefit of the team and the organization. We’ll discuss Jan Wischweh’s ideas anon.

I’m open to the possibility that the Scrum (and Agile?) names have become so polluted and diluted as to make them no longer useful for what I’d call Real Agile. I’m resisting that, but I might be trying to hold back the sea.

Which Crisis?

Wischweh, if I understand this part, provides a bit of history, and correctly identifies the values of the Agile Manifesto, and is probably close to correct by saying that Agile = Scrum to 85% of the self-identified “Agile” efforts.³

The Mainstream: Flaccid Agile and the Agile-Industrial Complex

Here, Wischweh paraphrases Martin Fowler’s concerns:

• Brief training from the Agile-Industrial Complex;
• Emphasis on formal rules over technical excellence;
• Project focus rather than product focus.

I agree with Martin here, and we’re not yet sure of Wischweh’s own position. I’m getting tempted to skip ahead, but lets carry on. So far, I have little to disagree with, but observe that some of what I’m agreeing with is just what the other Manifesto authors have been saying, and we largely agree on the state of the world.

The Agile Split: Technical vs Business Practices

In the interest of keeping this article of finite size, I’ll let you read Wischweh’s observations here. I agree that Agile as often done today has not healed the divide between business and tech, and I agree that one big cause of that is the old-school “product manager” approach.

The Failure of Agile

Yes, Andy Hunt did say that, and yes, beginners often do follow rules without regard to context. I remarked yesterday, in a conversation with Chet Hendrickson and Hugo Lourenco⁴, that I’ve seen a tendency in myself and many others, when working with new ideas, to stay within the rules as I understand them, because I’m uncertain and afraid of what might happen if I step outside the rules.

An example may help: Some years ago, Scrum suggested tracking Sprints and the entire effort with a “burndown chart”, a simple diagram showing how much work was done compared to how much needed to be done.⁵ In that era, creative teams began using cards on the wall, called “kanban boards”, to track their work. Many Scrum experts at the time decried this idea because it didn’t follow Scrum’s rules. Fortunately, that particular mistake didn’t last too long, because card walls are so useful, but it exemplifies the concern with newer practitioners of any art.

Can Agile Be Saved?

Here, Wischweh says that the “critical practitioners” (I guess he means people like me) agree that people aren’t doing it right. And all of the folks I know who understand Agile rather well pretty much do agree that people aren’t doing it right. I think here’s where we’re about to diverge. Let’s find out.⁶

Dark Scrum: The True Scotsman

This section is largely about me and my meaning for “Dark Scrum”, a term which I believe I coined to describe Scrum gone horribly wrong, usually to the detriment of the team and the product. He concludes his comments this way:

Jeffries is convinced Dark Scrum can be moved to real Scrum. By an effort of the developers and by improving education on Scrum in general.

It is a true Scotsman.

I am not exactly convinced that developer practices can cure Dark Scrum, but I do think it may be developers’ only hope. I’m not sure what “It is a true Scotsman” was intended to mean, but I will say here that when I call out a Dark Scrum practice, it is either explicitly in opposition to values and principles of the Manifesto, or in contradiction of Scrum’s definition, or both.

For example, I recently heard of an unhappy team who kept falling short of their Sprint commitment every Sprint. Asked why they didn’t take on less work, they replied “We are told how much work to take on”. Well, boys and girls, it says right here in the Scrum Guide:

The number of items selected from the Product Backlog for the Sprint is solely up to the Development Team. Only the Development Team can assess what it can accomplish over the upcoming Sprint.

We don’t have a Scotsman problem here. Those people aren’t doing it right. Prosecution rests.

Now, would letting the team select how much work they can do fix this team? Very likely not. Very likely there are further problems that we’d have to identify and fix to get them swinging. But Scrum says to Inspect and Adapt the process of doing the work. This team isn’t doing that right either. Most likely, if I had to bet, they’re held back by that “project management” attitude we spoke of earlier, and they haven’t been allowed to fix the problem.

No Scotsman problem there either: this problem was known to the unhappy team, and it didn’t get fixed. Scrum said fix it. They did not, probably because it would have taken a bigger hammer than they had. But still, it’s not Scrum if you don’t Inspect and Adapt as needed.

I’d bet a day’s consulting fee that in a day I could find more ways in which this organization isn’t doing the Scrum they say they are. Furthermore, I’d bet that I could find most of them by comparison with Scrum recommendations, and that the causes would mostly be cultural, impelled by a project management focus.

If somehow that team got “fixed”, you might well see fewer deviations from Scrum, and certainly fewer from Manifesto values and principles. Would that be because Scrum fixed them? Absolutely not. Scrum is a lens that shows you your problems. Fixing those problems doesn’t come from bearing down on Scrum, it comes from finding what’s really causing the problem and addressing that.

Now, remember our agreement above that newish practitioners tend to adhere to the rules. This means that many ScrumMasters, many coaches, and many practitioners would say: “Scrum says the team decides how much to do, do that”. If that were done, my prediction is that the team would continue to miss its Sprint commitments. Why? Because whoever the people are in that organization are who are pushing too much work won’t stop pushing. They’ll turn from saying “Do twenty stories” to “This team has to go faster”, or other ways of inducing the team to take on more work than they can do.

The fix isn’t the practice of team pulling work. The fix is somewhere else and the result is the team pulling work.

But in my view, the Scrum lens usually doesn’t lie. If we examine Scrum teams who are not thriving, we will usually find a number of serious differences between what they’re doing and what Scrum says. They really are doing it wrong.

But while Scrum would say “The dev team decides”, the fix isn’t just “let the dev team decide”. The fix is to address the cultural and organizational issues that resulted in the team being told how much to do.

My bottom line here is that “You’re doing it wrong” is not a No True Scotsman fallacy. In most cases, there is a visible deviation between what’s going on and what the method says. That doesn’t mean do what the method says. It’s smoke. You have to find the fire and put it out.

Wischweh concludes this section saying that it doesn’t make sense to try to get back to “Real Agile”. Here, I disagree. I think “Real Agile” works as well as anything I’ve ever seen, and that the values and principles of the Agile Manifesto are well worth learning and holding.

The Imposition of Agile Processes

The point here seems to be that imposition is bad. It is, yes. Then there’s something about whether the emperor is naked. Let’s read on …

Agile and the rationalization of software production

Here Wischweh tries out the assertion that Agile is just another form of Taylorism. I can assure you that no Manifesto author would agree that the thing we’re talking about is a form of Taylorism. We might agree that things done in the name of Agile smack of Taylorism. Those are misuses, and are the reason why terms such as Faux Agile and Fake Agile and Dark Agile have been coined.

Wischweh asserts that Agile turns into assembly line work:

• Work is broken down into the smallest and easiest steps possible;
• The pace of the work is controlled, measured and managed.

Yes, but the question is how. In a Taylor situation, Taylor figures out the work breakdown and hands it to the workers. In Agile, the workers are supposed to figure out the breakdown, because they are the experts. The Scrum Guide is clear on that, and the Manifesto is as well, in particular the focus on self-organizing teams.

Yes, Faux Agile approaches often spoon feed tiny stories to their dev teams (and tell them how many to accomplish). That’s bad. It’s also counter to what the Manifesto Authors had in mind, and counter to what they have said.

We might well find, today, coaches and trainers who would condone or even teach this inferior way of working. I’d repudiate it, as would all the people I hang out with.

Aside

I feel that we’re having a problem here between the words “Agile” and “Agile”. It’s quite common that idea sets get polluted. I have in mind not just Agile, but, oh, ideas like religion, democracy or “The United States”. Things are said and done in the name of these that seem very counter to their original intention. I’ll say no more about that, I just wanted to point out that the concern isn’t limited to Agile.

Is “Agile” the thing that I say it is, that the Authors say it is, that the dozens of really good coaches and trainers I know say it is … or is “Agile” somehow defined by all the people who are, by my lights, “doing it wrong”?

End Aside

Wischweh then says:

The ultimate goal is to make the software-worker disposable by the process and even the gap between highly experienced engineers and less skilled members of the team. This is done for the benefit of productivity and predictable quality of the resulting product, in a way that aims to be as reproducible as possible. However, by doing this, Pioneers and Geniuses on the one end and Spaghetti-Script-Cowboys on the other end of the spectrum are not longer indispensable and are clearly out of date.

Well, not only no, but hell no. I don’t know anyone who has that as a goal. Certainly no one in the “Real Agile” community has that as a goal. The Manifesto and even Scrum are very clear that the work is to be done by experts. The Manifesto, for example, says “Continuous attention to technical excellence and good design enhances agility.”

Now sufficiently bad Faux Agile might have an effect of limiting the impact of “highly experienced engineers”, and certainly there is a perverted view of economics that might lead a company to try to get software done while hiring only inexperienced developers and telling them exactly what to do. But that won’t work, and it sure isn’t even implicit in any form of Agile (or even Fake Agile) that I know of.

I believe that Wischweh is seriously off base here. I do accept that probably some people and organizations would like to turn software development into a more factory-like process. I’d even agree that there are some aspects of software development that can benefit from that view. Let me give an example:

I’m writing this article in plain text with Markdown tags. When you see it, it will have been converted to an HTML file, and several additional pages on my site will have been updated to refer to it. Some years ago, I had to type the HTML and update the index manually. Now, every time I save this file, all that is done behind the scenes, by my web site factory. And when I’m ready to publish it, I just press a button and all the changed files are sent up to my ISP. Repeated work that can be automated probably should be, in my view.

Probably more of software development than is currently automated can be automated. Some years ago, there were no automated refactoring tools. Then there was one, then there were several. Now there are many. That’s a good thing, in my view.

But the real problem in software development isn’t typing in syntax, it’s understanding a user’s need, and devising a creative approach to solving that need. Old-style development recommended phases, analysis, design, code, test, and some people did seem to think that “code” just meant typing in whatever the analysis and design people said. (Wischweh references this old style in the Waterfall section, which we’ve skipped over.)

That phased approach has several problems that are addressed by Agile ideas:

• The phases inherently produce delay. A later phase cannot start until the preceding phase is complete. You can do partial phases and pass things on, but this of course results in feedback from design to analysis and code to them both, as issues are seen.
• The coding phase is absolutely precise. The code has to be just right or the program won’t work. The analysis phase and design phase, however, are not subject to the same rules, and their output is invariably vague, incomplete, and occasionally mistaken. Again, a loop has to be created, with testing and coding informing analysis and design.

You can’t get around these issues. Analysis, design, code, and test are all elements of the work to be done. Agile doesn’t get rid of those phases. Instead, Agile suggests two key changes over real or fake waterfall.

• Bring all the skills together. Create a “dev team” containing all the skills of analysis, design, coding, and testing required to build the product.
• Shorten the analyze-design-code-test cycle as much as possible, down to weeks or even a single day.

Shortening the cycle and working together means that feedback is rapid, but generally quite small. We devise a solution idea, start to build it, find out what’s wrong with it, and improve the idea, all in a matter of a day or less.

This is in no way working against technical excellence, in no way trying to make software developers disposable.

Are Faux Agile promoters trying to do that? Perhaps. If so, I’ll cast it under Dark and rail against it. Not a fake Scotsman. We really said not to do that.

The over-idealization of Agile needs to stop.

Let’s look at Wischweh’s words here, because I think they’re making my point rather than Wischweh’s.

Considered sober, the shift towards Agile changed the ways of the software industry. But at the end of the day, it is just a rationalization in software production comparable to other rationalization processes in other industries. It modernized how we are doing our work, but it is more driven by the quest for efficiency than by values. Ironically the efficiency cannot fully unfold when the values are disregarded.

Yep. I fully agree. You can’t gain efficiency without the values. That’s what I call Faux or Fake or Dark Agile.

In what follows this paragraph, Wischweh points out that the management - technical gap persists, and that the dysfunctions can be harder to see behind a wall of what I’d call Fake Agile.

Yes the gap exists. But calling the tail a leg doesn’t make it a leg, and calling that Agile doesn’t make it Agile. Unless we’ve lost the word war. Is this whole article exchange about deciding we need a new word, as well as an increased focus on the values? An interesting possibility … I look forward to seeing what I say next …

In a bit of a mini-screed, Wischweh says that we need to ask about inherent shortcomings of Agile. No doubt about that. If Agile stops “uncovering better ways”, then it is dead and we should move on. So far, in my view, that’s not the case. Attend a Deliver: Agile conference, or technical presentations at other Agile-focused conferences and you’ll see ideas and tools that are new and powerful. At the same time, however, you’ll see a lot of same old, a lot of by the book, and a lot of new ways to use Jira. That’s unfortunate, but it is an inevitable result of popularity.

Wischweh is beginning to wrap up now, it seems. Some quotes:

Agile started as kind of (project) management crisis and management still seems to be a hard problem in IT. We need to look deeper into the management crisis and the software crisis and revisit it. At the moment it seems, the ideals of the Agile Community are difficult to uphold in the harsh reality of the Business world. Calling upon the individual responsibility of every single IT professional just isn’t enough.

The distrust in Developers, which supposedly need to be disciplined and controlled by management, is another issue we need to address.

Here, I agree. But read on:

And the issue of trust cannot be addressed without looking at the problem of power. Agile, especially Scrum, is more about efficiency than about empowering developers and it is not a shift away from Taylorism. On closer inspection, this will be visible in every single conflict within companies trying to transform towards Agile. Quite the opposite is true: it makes people more replaceable and controllable and is a modern and competitive form of Management. But when the power holders become invincible and teams start to impose control on themselves, conflicts are avoided and responsibility becomes concealed. The humane promise of Agile is broken.

I agree that we have to look at the issue of power. I do not agree that Scrum (as defined) is more about efficiency and not a shift from Taylorism. On the contrary, Scrum clearly calls for dev team power over how the work is done, which is exactly NOT Tayloristic.

Nor do I see how Agile makes development workers more replaceable. Agile requires some skills that developers don’t have out of the box. Many people in the Agile community have written about the need for team members to become more broad in their skills, both technical and human, while not giving up whatever depth they may also have. This will make individuals more valuable and harder to replace, not easier.

I think the paragraph above is a bit histrionic, but I don’t begrudge the author that, I do it myself when I get on a roll. And certainly a nearly invincible power holder who uses power poorly is a big problem and you’ll likely not see Agile thriving there.

I’ll quote Wischweh’s concluding paragraphs in their entirety here before providing my own:

Within the scope of this article, I can only sketch out which questions need to be raised and where the Agile community has some blind spots. But I think the points, which we only reluctantly want to tackle, are probably most worthy of our attention. Keeping the software industry a great place to work requires a deep reflection on Agile and where we as developers and human beings want to go. The most important question still is how we want to unfold the power of the machines for the better of humankind. The digital Industry is already shaping the future of work, but how can we take better control of it?

This Article is a primer and invitation for discussion and a call for looking deeper into the Crisis. Please contribute and let me know your thoughts.

At this moment, I feel that if Jan Wischweh and I were to sit down over a few cups, we’d disagree on very little other than phrasing, and some of the items mentioned here so far. We might be pretty close to those. I’m feeling as if our differences lie in terms like “Agile community”, although of course everyone has blind spots.

If by “Agile community” we mean me and mine, then we are doing our best to see and work these issues. However, if by “Agile community” we mean the population at large, or some (but not all) members of the “Agile Industrial Complex”, or some (but not all) organizations rolling their own “Agile”, then yes, some (but not all) of those people really do seem to have blind spots and do seem reluctant to address the concerns.

Where would we mostly disagree? I am not quite as concerned as Wischweh appears to be regarding the desire to turn software development into a factory. That said, I expect we’d agree on automating that which can be well-automated. I don’t really know anyone who wants to create plug-compatible mindless developers, but if those people exist, they’re not going to get that job done, because software development is creative from top to bottom and you don’t automate creativity.

I certainly have seen individuals and organizations who have little or no appreciation for developers and development, and perhaps Jan has more recent or closer exposure to people like that. That could lead to a feeling that people would prefer that software developers are all drones who should be worked hard until they wear out. I think that’s more about specific individuals and industries. I’m sure it’s not about Agile as my community thinks of it.

Bottom Line

We’ve got over 4,000 words here at the moment, and I’ve got to wrap up, or try to.

I see many of the same problems as does Wischweh, and I freely state that I attribute most of them to people and organizations who are not following the Agile Values and Principles, and I think that if they are nominally doing Scrum, we would likely find explicit and undesirable deviations from what Scrum says.

What kind of deviations? Things like not having a cross-functional team that can ship a product increment. Things like telling them how much work to take on, or pressuring them for more. Things like not encouraging (and in some cases even pushing back against) important practices like testing and refactoring. Things like Product Owners bringing in canned solutions, instead of doing the collaboration that the Manifesto and Scrum both call for.

Again, I’m not saying that the transition from terrible results to good ones will magically happen if you let your dev team choose how much work to take on from the Do All This backlog. The magic, if you are to see it, will come when you’ve changed your organization such that having the dev team define and take on their own work in intimate collaboration with their Product Owner and other stakeholders.

And while Wischweh may be saying that terms like Faux and Dark do not help, at the moment I see a need to call out bad situations that call themselves Agile, yet are not living by the values and principles, and are, as a result, oppressing and harming the people on the teams, while at the same time harming the organization by getting far poorer results than if they were “doing it right”.

“Doing it right” and “doing it wrong” are difficult notions. They are sometimes subject to the “No True Scotsman” objection, but in my experience here, they usually are not. When Scrum goes bad, you can usually find things directly counter to what Scrum teaches. How that came about, and how to fix it, are not as simple as “do what Scrum teaches”. Unfortunately, the solution is always more like “Do what Scrum means”, which is a complex thing we can only learn over time.

If you’ve stuck with me for this massive tome, thank you. I welcome Twitter messages and I hope I’ll write something shorter and more pithy, founded in part on Jan Weschweh’s interesting and provocative article.