Dark Scrum: Hills: The Backlog
Scrum requires that the team have a backlog. Fact is, it requires that they have two backlogs, the Product Backlog, and the Sprint Backlog. Here, we’ll concern ourselves with the “Product Backlog”. The Product Backlog is a list of “everything that is known to be needed in the product”. But let’s quote a few paragraphs from the Scrum Guide:
A Product Backlog is never complete. The earliest development of it lays out the initially known and best-understood requirements. The Product Backlog evolves as the product and the environment in which it will be used evolves. The Product Backlog is dynamic; it constantly changes to identify what the product needs to be appropriate, competitive, and useful. If a product exists, its Product Backlog also exists.
The Product Backlog lists all features, functions, requirements, enhancements, and fixes that constitute the changes to be made to the product in future releases. Product Backlog items have the attributes of a description, order, estimate, and value. Product Backlog items often include test descriptions that will prove its completeness when “Done”.
As a product is used and gains value, and the marketplace provides feedback, the Product Backlog becomes a larger and more exhaustive list. Requirements never stop changing, so a Product Backlog is a living artifact. Changes in business requirements, market conditions, or technology may cause changes in the Product Backlog.
I think it was at least a decade ago when I first heard Mary Poppendieck saying that the Scrum backlog was a bad idea. My recollection is that her primary objection to the idea was the notion that the Product Owner created and defined things to do, and the team just had to do them. She often spoke of her idea of how product should be made with more of a “Product Champion” who set the team free to figure out how to solve the problem that the product was supposed to solve.
Now the truth is, many “Scrum” teams do work that way. The PO comes in with a long list of stuff that must be done, and the team cranks it out. Notions like “Definition of Ready” can seem to support this idea.
Let me say right here that if the PO creates some massive list of stuff to be done off line and then comes in and parcels it out to the team and says “just do it”, that’s not the ideal situation. Certainly no matter how creative the PO is, the creativity of the whole team is more creative. (Except in the rare case where someone on the team has negative creativity, somehow sucking all the creativity out of the room. Otherwise adding creativity, well, adds creativity.)
The Product Owner is supposed to be the person in the room with the best overall understanding of what needs to be done – and they have the final say on what’s to be done, so it would be unwise to have them be other than very good at knowing what the point of the product is. But Scrum leaves it quite open just how all this gets done:
The Product Owner is the sole person responsible for managing the Product Backlog. Product Backlog management includes:
Clearly expressing Product Backlog items;
Ordering the items in the Product Backlog to best achieve goals and missions;
Optimizing the value of the work the Development Team performs;
Ensuring that the Product Backlog is visible, transparent, and clear to all, and [that it] shows what the Scrum Team will work on next; and,
Ensuring the Development Team understands items in the Product Backlog to the level needed.
The Product Owner may do the above work, or have the Development Team do it1. However, the Product Owner remains accountable.
Thus Scrum, at least as described today, leaves the door open for a Product Owner to bring in nothing but vision, ideas, and problems, and to let the team figure out the entire product. It’s very likely that whatever your vision is for how things should be done, Scrum “allows” it. Here’s an example of a tweet from Allen Holub, just today:
A proper “sprint goal” is: put valuable software into our users hands. An actively destructive goal is to “commit” to delivering some set of stories ahead of time. The former is valuable, the latter will erode any agility that your team might have.
While I think it’s possible to use commitment to a set of stories well, our point here is that Scrum totally allows the approach that Allen recommends. Wise Scrum practitioners prefer that approach. Unwise ones, not so much.
This is a perfect example of the analysis we’re trying to do in this series. We want to be clear about the difference between what Scrum’s creators say and recommend, and what people do. If the creators say something really stupid, we need to correct them. If people trying to do Scrum do something really stupid, we need to help them do better. If the creators say something poorly, we need to help clarify what they say, and to help them say it better. We need to aim our objections – and even better, our help – at the right recipients.
Allen, of course, was not condemning Scrum in the quote above. He was describing two near-endpoints on the spectrum of how Scrum is done, and indicating which direction we should favor. I think he overstated his case a bit and I, of course, never would and never have done that. His point remains a good one.
A day or so ago, Dan North tweeted this:
Having a backlog, grooming a backlog, managing that much WIP is silly given how fast a team can turn round an idea and get real feedback.
If Dan had said “when”, instead of “given”, it would have been easier to agree. It’s not always possible to develop a product by trying an idea and getting feedback. Let me give an example from my history.
I worked on a tax preparation system for tax professionals in the dark years of my past2. The thing about a tax preparation system is that it needs to handle a large number of tax forms and a large number of states, or it is useless to any tax pro. And, in case you’ve not done it lately, every line of a tax form has a raft of instructions and laws backing it up, and these all need to be taken into account and done correctly when you write the software. So we had a raft of tax experts working out what had to be done, and in the end, everything on every form we needed had to work.
Oh, there was a little flexibility. You could skip an obscure form, since whatever a form produces gets written somewhere on a top-level form and added up. So you could allow a form to be done manually, if you at least provided a place up on the 1040 or Schedule D or somewhere, for values to be put in.
But by and large, you don’t have a tax product until all that stuff is done. So call the backlog WIP if you wish. Hate the long list – we certainly did. But the fact remains, in situations like that you need a backlog, and while you can experiment on how you build it, what you build is incredibly well defined.