Perhaps the biggest mistake in XP and Agile …

I take my theme this morning from something Chet said at last Tuesday night’s Friday Night Coding Zoom Ensemble. His point was that the early supporters of XP and Agile ideas thought of themselves as average developers. This was probably partly due to the fact that we had all been exposed to, and worked with, some programmers who were profoundly and obviously much better than we were. So even the most egotistical of us might have thought “well, I’m probably a bit above average”, we couldn’t get too full of ourselves, because we knew programmers who could code rings around us.

That sort of thing keeps you grounded.

Then, yesterday, I tweeted that I’m thinking of doing some work with Kotlin, and what did people think of that. One person replied, I believe with some intended sarcasm:

I think you should do a full blown mobile app project using a Javascript-ish environment that requires a lot of tooling, such as ionic or React Native.

And to make it more interesting, you have to deliver a valuable feature and a full, one-step CI/CD setup the first two weeks.

I replied:

I bet I could do that the second time. Wouldn’t surprise me not to be able to the first time, on tools I’ve never used.

Seems one could build an app shell to start new projects with?

Since I don’t know Kotlin, IntelliJ IDEA, Maven/Gradle, JavaFX, React, whatever all, it wouldn’t surprise me to need more than a couple of weeks to get a thing in place, working on my own. I’ve read enough about those things, and played just enough in the past few days to know that.

For that matter, no one has ever said that you have to have a full one-step CI/CD setup right out of the box. I do think a team should be able to manage something like a feature in a couple of weeks. But if they don’t already have a working CI/CD pipeline on their previous project, I’d expect them to be enhancing it for a long time.

No one ever said you have to be perfect on day one.

But you do need to be competent, and you do need to be good.

It would be insane to hire me as a developer in JavaScript, Java, C#, Kotlin … really anything but Codea Lua, where I’ve been spending most of my time for the past few years … because I don’t use those languages any more (or ever, in the case of Kotlin) and I’ve lost my chops, especially with all the associated tools, which I have always hated and where possible foisted off the work onto people who like that sort of thing.

However, if you brought me in to help you with the code in any of those languages, I fant’sy that I’d be useful very quickly, because I can read those languages well enough, and I know enough about how too build software to see issues and know what I’d do about them … even if I don’t know the exact keystrokes to type in.

Fortunately for us all, we’ll never know how useful I’d be, either as a programmer or in the role I suggest in the paragraph above, because no one is going to ask me to join them, and if they did, I would decline. So if it pleases you that you know Gradle and I don’t, or to think that your code is so horrible, or so lovely, that I’d run away crying as soon as I saw it, go ahead and think that. I really don’t mind.

However, I was talking about what may have been a mistake, thinking that everyday developers could thrive in XP, or even in Scrum. Let’s see what it takes, in a team, to thrive in an environment where you are supposed to produce discernible software value every week.

Iterative, Incremental

Scrum, the dominant death form of software development in the world today, unless you think that the even more deadly SAFe is dominant, asks the team to produce a working increment of software every “Sprint”, an interval of one or two weeks, four if you are truly awful.

The process is iterative: that is, it repeats in a cycle. Plan, do, release, that kind of thing.

The product is built incrementally. Feature A, then A+B, then A+B+C, that kind of thing.

And you’re supposed to add a few A’s and B’s and C’s every darn week or two.

I don’t think I’ve ever said that’s easy, but if I did, I’m sorry now. It’s not easy. It’s just the best way we know to survive and thrive in software development, unless you think that just keeping your head down and your resume out is better.

What we have to do:

To thrive in the iterative incremental world, we need to accomplish some difficult things:

  • We need to evolve the design from simple to more and more capable;
  • We need to test all the software all the time, because the design is changing and that can break things;
  • We need to add capability in tiny tiny slices, because you really can’t do a big feature in a week or two;
  • We need to build the system many times a week, many times a day, to be sure that we can be shippable at the end of every week;

I’m sure I could make the list more detailed, but those items make it hard enough.

This isn’t easy.

To evolve the design, we need to be able to refactor, and to do it well. We can’t rewrite the design every two weeks, we have to morph it smoothly, growing it, extending it, and keeping it a “good” design. This isn’t easy.

To evolve the design that way, we have to know what a good design is, and how to change a poor design to be more good. This isn’t easy.

To be sure the program is tested and working all the time, tests have to be automated, because no one wants an entire building full of people hammering keys to test the code. Providing tests, and the pipeline to run them, is required. This isn’t easy.

We have to figure out how to break big features down into sensible bits that show progress every week. This requires a very strong sense of design, and the ability to work toward a complex thing in small simple steps. This isn’t easy.

To build the system multiple times per day, we need powerful tools, and a strong command of them, and a continued focus on making the build/test cycle fast, fast, fast. This isn’t easy.

We need knowledge and skills.

Within our team we need all the knowledge and skills to accomplish all that stuff above, and of course ten or a hundred times more stuff than that. And every developer needs some of those skills.

Every developer needs to be able to build software with tests, so that the tests track with the system’s capability.

Every developer needs to be able to start with a small design and evolve to a good design. Therefore …

Every developer needs to know what a good design is, and what makes a design not so good. And …

Every developer needs to know how to change the design smoothly, without massive rewrites that would stall the project. They need to know how to refactor, and how to do it well.

And at least some of the developers need skill with specialized areas, including the CI/CD pipeline, configuring the IDE and other tools, figuring out why the build won’t run on Jack’s machine, and so on.

And most of the developers need skill with taking the vague and often contradictory ideas of product managers and customers, and turning them into sensible features, and then slicing those features into a size that allows them to be built up smoothly, with new capability appearing every week.

This Isn’t Easy

Really. It’s not easy. It’s not even easy all the time for those of us who have been doing it for years. What we’ve learned to do is to make things easier. Our main trick to make things easier is to do them more often. That has two useful effects.

First, if you do something often, you’re just about guaranteed to get good at it. If you pay attention at all, you’ll find things that work, things that don’t work, ways to hold your mouth so that the thread will go into the eye of the needle. You get good at it.

Second, if we do something often, it often gets easier just because of the shorter time between doing it. If you and I work on our own separate branches for a couple of weeks and then go to merge, there are going to be merge conflicts, and some of them will be massive.

If we have a rule on the team that whoever encounters a merge conflict has to fix it (perhaps with help), then all I have to do is merge before you do. (Similar to running faster than you, not faster than the bear.) But you know this strategy too, so you want to merge before I do.

Since we surely have the rule that you can’t push a release with broken tests, we race to merge every time the tests run. Soon we’re merging every few minutes. Wow, look, we’ve invented Trunk-Based Development, and we rarely have merge conflicts any more.

So, while this isn’t easy … it gets easier the more we do it, and it gets easier the better we become at doing it.

Bottom Line

This should be no surprise: the bottom line is, the better you are at doing your job, the better things are likely to go for you. There are no guarantees, of course, the universe is definitely plotting your demise, but, by and large, the better we are, the better we go.

If I gave you the impression you could thrive without getting good at your job, I do apologize. In my defense, I’ve been writing for years, decades, about how to get good at this stuff, and showing that we can get better in small, relatively easy steps, up a very tall mountain of possibility.

It isn’t easy … but it gets easier with learning and practice … and learning and practice can be a lot of fun.

What you do with that, well, it’s up to you.