Technical Stories: We don't need 'em.
Teams often want to write "technical stories" to cover things like testing and improving the design or infrastructure. There is rarely a need to do this. Here's why.
The fundamental point of Scrum is for the PO to prioritize work for the team, so that the best possible product can be delivered within the desired time and budget. The PO will get the best product if work is chosen that is of high “business value”. (Low development cost is also a concern in choosing the work, but may will be substantially less important than value.)
Clearly, the PO is capable of driving out more value when a larger percentage of the team’s time comes under her control, since otherwise, she is directing a smaller flow of work, which will inherently reduce total value. Therefore it is undesirable to slice away a strip of the team’s development time and make it unavailable for stories.
However, so-called “technical stories” have a value proposition which is almost completely different from an ordinary story. A technical refactoring story may promise faster velocity sometime in the future, while an ordinary story is something that real users actually want. It is therefore very difficult, perhaps impossible, to compare an ordinary story and a technical story for value. Therefore it is undesirable to present technical stories at all, since they are so difficult to prioritize.
This seems, if one doesn’t think about it very hard, to be a dilemma. We can’t take away the PO’s time, we can’t prioritize the technical stories. We can’t have them at all. We are doomed.
Like most dilemmas, this one is solved by using a third alternative. It turns out that [almost?] every bit of technical work, if it should be done at all, is done in support of one or more stories. IN PARTICULAR, the kind of work the OP is talking about, creation of tests and creation of clean code, is inherently part of creating stories.
Why inherently, one might ask. Because for code to be DONE, it must work, and to know whether it works, we must test it. Because we are working incrementally, we will revisit much of the code many times and we might break it, so it is wise that the tests be recorded for reuse, i.e. automated.
Further, since we are building the system incrementally, we are building the design incrementally. We do not have the time (nor, likely, the ability) to define the design correctly up front. We do not have the time to build it up front even if we did have the ability to define it. We must, perforce, evolve the design. Therefore, for each feature to be DONE, its design must be fit for the current purpose.
Therefore, testing, and design improvement, are inherently part of the work of doing stories.
How does this help us with our dilemma? We have, through our human failings, fallen behind on the testing and design. What are we to do? We do not want to slice time away from the Product Owner. The Product Owner cannot correctly prioritize these strange technical stories. We are doomed.
No. All we need to do is to bear down a bit on improving the code we work on, doing all it takes to bring it back up to the proper definition of DONE, namely “well-tested and well-factored”. We make the code a bit better every time we pass through it.
At first, maybe this takes a little longer per story. This is not a real concern, as no one but the team really knows if the team can do twelve this Sprint or only ten. The team commits to what they can do. They do it, and the world becomes a better place.
Better yet, soon, surprisingly soon, it doesn’t take longer after all. Since we are improving the code and tests in the areas we work in, velocity picks up quite quickly in the areas we work in. We wind up going faster than before, because we are going better.
Our Product Owner retains her ability to prioritize the whole of our work, and our code improves. No confusion, no political discussion, no need to make promises about the future that we may not be able to keep.
We don’t need technical stories or tricky negotiations. We just do the job we were always supposed to do, namely make each story DONE, fully tested, with a good design in place.