Kate Oneal: Handling Defects
A few minutes before 9, the team was in the coffee room.
“What time’s the meeting?” said Gil. “Is she here?”
“Nine sharp,” said Susan. “I just went by her office. She’s not there.”
“Carl and I have been working in here since 8,” Gil said. “She hasn’t been here for a Diet Coke.”
“This is the day,” Carl said. “She’s late. Let’s go!”
When the team came into her office, Kate was by her window. She spun her wheelchair and scooted to her usual spot at the table.
“How do you do that?” said Gil.
“It’s easy. To spin, push one wheel, pull the other. To go forward, push both. To turn, grab one. To stop, grab both.”
“No, how do you always get to meetings right on time?”
Kate smiled. “That’s easy too: you leave wherever you are as many minutes as it’s going to take to get wherever you’re going.”
Carl said, “You have a Diet Coke.”
“Yes. Yes, I have.”
“It must be warm.”
Kate felt the can. “No, it’s nice and cold. The machine is working just fine.”
“I mean, how did you get it? Did you bring it to work with you?”
“Standard way. Put coins in slot, press the button. I suppose I could have thrown the coins across the room into the slot and tossed my exercise ball against the button, but I’d still have to go get the can.”
“No one could do that,” said Gil.
Kate smiled. “Maybe no one ever tried. What’s this meeting about?”
On his way to the coffee pot, Carl said, “We want to talk about what to do about bugs.”
“Great,” Kate said. “Let’s not have any.” She clapped her hands. “Meeting over?”
Susan laughed. “As Product Owner, I’m good with that. Move to adjourn.”
“Not so fast,” said Gil. “There are always going to be bugs. We have to handle them.”
“I knew it couldn’t be that easy,” said Kate. “What about these defects?”
Gil said, “Well, should we put them into a database? How should we decide which ones to fix?”
Kate said, “Who decides what features you work on?”
Gil said, “Susan, as you well know.”
Kate laughed. “Yes, as we all well know. So … who should decide which broken features to fix?”
Gil said, “Susan?”
“Good idea. Susan.”
Carl, having returned to the table, interrupted. “Oops. I forgot sugar. Sugar, please.”
Kate dipped two fingers into the bowl of sugar packets, and pulled one out. “Don’t move,” she said, and flicked the packet at Carl. It hit him in the chest and dropped into his shirt pocket.
“How did you do that?”
“Easy. Your nerd pack pulls your pocket open.”
Everyone laughed.
Carl frowned. “You never know when you might need a pen. Or a marker. Or a drawing pencil. Or another pen …”
“True that,” Kate said. “So what else about these defects?”
Susan said, “Well, we need to decide just how to handle the scheduling, how to count velocity, and things like that. And if there are a lot of them, how should we keep track of them?”
Kate said, “How do you schedule things now?”
Susan grinned. “Why do you ask questions you know the answers to? I write a short description on a card. If I need help deciding what to write, or need to be sure the idea makes sense, I talk about it with the team. Also, I meet with the testers to work out the correctness checks. We do that about once a week. Then at the planning meeting, I bring enough cards and checks to fill the week, we talk through each one, figure out how many the team can take on, and commit to the work.”
Kate paused, then said,”Why do I ask questions I know the answers to?”
Susan stared. Then, “Oh. If I want a bug fixed, I could write it on a card and bring it to the meeting?”
“Sure could,” said Kate. “But you mentioned something else that you bring.”
“The checks,” said Susan. “The testers and I could bring in some checks for the bug. They always figure out a way to reproduce it anyway.”
Kate smiled. “Sounds good to me. But isn’t there something special about these defect checks?”
Everyone looked confused. Finally, Charlie, a tester, spoke up. “Oh. We should have provided those checks the first time. We screwed up by not having them.”
Kate said, “I’m not into blame, and anyway we can’t go back on our own time line and edit the past. But we do have a good learning opportunity. What can we do to take advantage of that?”
Carl said, “The retrospective. We should look at the defects discovered and see how to do a better job of creating checks for them.”
“Good idea, Carl, and well put,” said Kate. “I noticed that you said ‘we’. Why did you put it that way?”
“Well, we programmers are in those story meetings too, and we help with the examples. Often a bug …”
Kate said, “Defect.”
“… yes, defect. Often a defect comes out in some odd coding case that a programmer might foresee and a tester might not. Anyway, we’re all in this together, so let’s all learn.”
Gil said, “There’s another chance to learn, sooner. That’s as soon as the bug – defect – is found. The testers usually grab a programmer right away, to be sure they understand the situation, and maybe to figure out how to reproduce it.
“Often, we see right away what is going on. That’s the moment to start learning.”
“Yes! Good point, Gil.” Kate looked delighted. “Now you said something about velocity?”
Susan said, “Yes. How should we handle velocity for bu … defect cards.”
Kate said, “Here’s a question I know the answer to: how do you use velocity?”
Susan said, “I use it in my product burn chart, to get a sense of how much we’ll have by the final ship date.”
Carl said, “And we use velocity to estimate how much work we can take on in the Sprint.”
“OK,” said Kate. “Two reasons: know how many features we’ll have by the ship date, and know how much work to take on in the Sprint. Do defects take time to fix? Can we estimate that time for the Sprint?”
Carl grinned. “They do, unfortunately, and we can usually estimate them pretty well. Sometimes we need to take some time to dig in a bit before we know.”
“If you don’t know how long a fix will take, what do you do?”
Carl went on. “Often we just guess how much time it’ll take or how many we can do. That works pretty well. If it’s a really weird one, we usually ask Susan to authorize a time-boxed exploration.”
“A spike,” Kate said.
“Yes. Then we usually know how long the thing will take. In fact, quite often we actually find and fix it inside the spike timebox.”
“OK,” Kate said, “it sounds like you’re totally on top of estimating defects into the Sprint. What about planning toward the release, Susan?”
Susan frowned a bit. “Defects are really a hassle. Usually they’re important, and I have no choice but to fix them. But they don’t fit into my planning at all.”
“Why not?”
“Well, even if I knew how long they take – and I think they probably average out well enough – I don’t know how many there are going to be.”
Kate said, “Couldn’t we just take that average as well? Like if you’re fixing five defects a Sprint, we just assume that if we have ten Sprints left, we’ll have fifty defects to fix?”
Susan said, “Well … we could …”
“So then,” Kate went on, “we could factor the defect fix time right into the planning and tracking. We could bump up the burn chart by fifty, and track the defects just like stories.”
Susan said, “We could, but that makes me feel kind of dirty. Defects are bad. Yes we can learn from them. For example, I learn how to say more clearly how things have to work, and I work closer with Charlie and the rest of the team coming up with examples that help us be sure things work. And I know the programmers are learning how to do their tests better, and better ways of designing things. But …”
“But what?” said Kate.
“I don’t want defects! When I have to schedule a defect fix, even when I can see it’s because of a test that I should have seen, I just don’t get the same thrill I do when I see the gang finish one of my features.”
Gil said, “You just want to punish us, or yourself, for making mistakes. Bugs are inevitable. They just happen.”
Kate said, “Hold on a second. Maybe some defects are inevitable. But aren’t you all learning ways to avoid them and make them less likely?”
Gil said, “Well, yes. Carl and I learned a new way of testing without going to the database last week, based on something Carl read about. That makes it a lot easier to write tests, and much faster to run them. I can identify a bug that I’m sure we would have had two weeks ago, avoided now because we can test better. But still, bugs will happen.”
Kate said, “They will. One reason I like to say ‘defect’, though, is to remind myself that whenever I look back at a ‘bug’ in the system, I can almost always find something concrete that I, or someone on the team, or all of us together, could have done to prevent it. Saying defect reminds me that most of them are up to us to avoid or prevent.”
Gil said, “There will always …”
Kate said, “Yes. There will always be defects. But do this for me. The next time one shows up that was absolutely unpreventable, bring it to a meeting with me. I’ll bet that by the time the meeting is over, we’ll know a way we could have prevented it … and a way that isn’t even all that costly. Meanwhile, let’s pretend that they are all preventable: most of them clearly are.”
Susan said, “So what does this say about planning?”
“Way to keep us on track, Susan. You’re right to feel bad about defects. There are two kinds of demand for work, according to the Lean and Systems Thinking people. Who knows what they are?”
Silence. Then, Susan said, “I don’t know the words, but there are the things our customers actually want and the things I have to do because the software isn’t right.”
Kate clapped. “Exactly right. The words they use are ‘value demand’ and ‘failure demand’. Value demand is the demand for what customers actually want. Failure demand is work we have to do because we are imperfect … because we have room for improvement.
“So I think you’re on the right track, Susan. I would suggest that you should not bump up your target by some expected number of defects, and you should not count defect fixes as features on the burn chart. If you do that, and there are defects, what will happen?”
Susan thought a moment. “My progress line won’t grow so fast. Each defect we fix takes away that much work on a feature. The chart will show us slowing down to the extent that we work on defects.”
“Right!”, said Kate. “Defects do slow us down, and in an unpredictable way. So it makes sense to me not to track them on your chart as if fixing them were a good thing. It’s more of a necessary evil.”
“But wait!” Carl, this time. “Those bugs can be hard to fix. We put in real work on those. We should get credit for that work.”
Kate grinned. “Remember the story lady? She comes in every Friday, looks on your shelves for features, and buys what you have, putting dollars in your pockets and lattes on your desks? What’s that beautiful redhead going to say when she finds a defect fix on the shelf? Am I going to pay for it?”
Carl smiled back. “I guess not. You’re going to say that you already paid for that feature and you want the fix for free.”
Kate said, “That’s right. Making me pay for defects ain’t workin’. I won’t pay money for nothin’. I want my fix for free.”
Everyone groaned.
“Welcome to my earworm. So, do we have a plan? Estimate for the Sprint as usual, don’t put defects on the Product Burn Chart, either in the target or in the progress?”
Everyone agreed.
“Great,” Kate said. “Meeting adjourned. Thanks for stopping by.”
As they started to leave, Carl turned. “One more question, Kate?”
“Sure?”
“Could you really throw your exercise ball and hit the Diet Coke button?”
“Is there money riding on it? Come over to the Bombay some evening and play a game of darts with me. Then we can talk about the exercise ball … and the coin slot.”