Design?

I mentioned that the dungeon program is boring me on Twitter and asked for ideas. Andreas asked about software design.

Andreas Schätti asked:

Insights into the SW design mystery, please!

In our exchange, I said:

Aside from slicing out easy bits, it’s magic. And slicing out easy bits is also magic.

How do you get magic? Lots of experience doing magic.

Andreas replied:

That’s what I fear. I hope I can learn something from design experts like Nigel Cross and transfer it to the SW world. Or from general problem solving.

And I said:

that’s how you do it.
you read about design.
you do design.
you write about design.
slowly, over time, you learn to design.

keep going!

That’s Not Very Helpful …

Well, I’d have to agree, and I’ll try to say a bit more here, and when occasions to design things arise in future. But I’m inclined to stand pretty pat on the two notions above. Let’s put them in the opposite order, and in different words.

To learn how to do software design:

1. Study software design;
2. Do software design;
3. Write about software design.

Slowly, you’ll learn how to do it, and if you’re like most of us, you will often be unable to say how you do it. (The magic part.)

Most of us, as we mature in software design, can do a pretty good job of describing an existing design in words and pictures. I can probably do a pretty decent job of telling you how the Dungeon program works. And because of how I write these articles, you can sometimes get a picture of what I think I’m thinking as I go along.

In the case of this program, Dung as I fondly call it, I was about 20 articles in with one design, when I decided to scrap the whole thing and start over from a new premise.

I had started with the same basic problem as Bryan Beecham, from our Zoom Ensemble, was solving: throwing random rooms into space and then connecting them with hallways. Rooms were just rectangles. Somewhere in our Zoom discussion, GeePaw Hill suggested a focus on tiles instead of rectangles, and that caused me to do a little outside reading, and to do an experiment.

The reading and experiment had a different focus from the original pigsXXX rectangles in space idea. It’s basically a space of tiles that you can think of as solid stone, and then one carves out rectangular rooms, and then one digs hallways between them. I liked how that experiment went, and essentially started over. I don’t remember now whether I threw everything away, or just the underlying dungeon bits. Probably I preserved what I could.

Lesson: Try, fix, or replace.

So this is a design lesson. Try things, and if they seem OK fix them up to be nice, and if they turn out not to be working, replace them.

I think we can never design without making mistakes. Well, I say “we”. I think I can never design without making mistakes. I have to see how a design shapes up in the code, because a vague line on a diagram may look just fine, but when I see that across that line a thousand messages are flowing, I get a dose of reality that helps me improve the design.

Get to code soon.

You may have had this experience if you’ve ever been handed a design to implement, or if you have tried to build a program using some new framework. You’re going along fine until suddenly you bump up against a problem caused by the design or framework, and you are dead certain that whoever created this thing must have never used it to do what you’re doing.

Kent Beck once said that he liked to let the code participate in the process of design, and it’s a good notion in my view.

Now, “obviously”, if I have a really bad design idea, there’s no point coding it up: I already know it’s bad. But as soon as I start thinking it’s pretty good, I can benefit from coding it up, in as light-weight a fashion as I can, because the code is how a design really comes to life.

In my work over the last couple of decades, it has been my practice to go to code essentially instantly. In most of these programming articles, there’s running code in the very first one, or at least quite soon. I choose to do that because I want to learn what happens when we go to code too soon. We already know what happens when we go to code too late: our design has flaws that make it hard to code.

What could possibly go wrong with starting to code too early. I can think of two things. Both assume that the design isn’t good enough. If it is, it’s surely time to code …

First, you might waste time, if you could have discovered the design flaws without the code. True enough, no sense coding for weeks to discover what you could have seen with a few cards on the table in a few hours. So don’t do that. Code a little, just enough to test your design questions.

Second, you might wind up committed to the code. Having invested X amount of time in the code, you might be unwilling to throw it away, and since the design isn’t right, it’s not good as it stands, and if you do fix the design problems, the code won’t match the design and you’ll have to fight it until it does or until you work around the design you just created.

In short: waste. The code might be wasted. Yes. So we learn to do just enough to discover problems.

Let me give an example. A colleague was going to try something challenging with a team they were working with. The team asked, given a clean sheet of paper, how would my colleague build the app they have. The app, of course, has microservices and database collections and all that jazz.

I asked my colleague whether they’d like to hear what I’d do in that situation and they said yes, so I said that I’d build the app, not with services or databases, but as a simple program manipulating memory collections containing objects like those out there on the database. I said that I thought that would let me discover the natural shape of the application.

So I create a design in as simple a way as I can, and I test an existing design in as simple a way as I can. This ensures that my early move to code produces as little waste as possible.

TDD

I’d be remiss not to mention Test-Driven Development. Learning this practice–and I’m still learning–has made a huge difference in the pace at which I can develop a program. GeePaw Hill likes to say that TDD is the fastest way to program that he knows, and I agree.

It’s odd, because TDD seems like it would be much slower, and it even feels slow. You have this perfectly good idea, so instead of implementing it, you think of the smallest simplest reason why it doesn’t work, typically that the class you’ve thought of doesn’t even exist yet.

Then you write a test to tell you what you damn well know, that the class doesn’t exist. Then you run the test to be sure the class you’ve not even started doesn’t exist, and surprise! it doesn’t exist, so now you can type in

Foo = class()

And now you can code your idea? No, not remotely. Instead you think of the simplest thing that it still doesn’t do, like give the right answer if it has no data to concern itself with and you implement that … no. You write another test to prove that the code you haven’t written doesn’t work yet. And you run the test. And surprise! the code you haven’t written isn’t there.

Then you make the simple thing work. Repeat, next simplest thing … ad infinitum.

Bloody HELL that seems slow.

But it turns out that it isn’t slow. The reasons are complex, I think, but here’s a simple explanation.

By carving off tiny simple bits of the problem and solution, even though we have the whole solution pretty well figured out, we get two advantages. First, we gain lots of experience using our new code. We’ve thought a lot about how to write it, and our tests make us think about how to use it, how to set up the class, what methods we’d like, what to name them, and all the outer stuff.

Second, we’re carving off simple cases. Each one of those is easy to solve, and if they can be carved off, they’re likely essential bits of the solution. That creates nice modularity in our new class. Often it even shows us that we could derive a simpler helper class along the way.

And as we carve off simple cases, we usually leave a simpler problem behind as well. If the problem partitions nicely into cases, our code will also partition nicely into cases. And where the partitioning isn’t appropriate, we’ll automatically extend existing code to handle a more broad range of the problem.

The result of this way of working is that we follow a path of many small simple steps, even when we’re creating a fairly large complicated thing.

And that’s almost invariably faster than just diving in, writing a mass of code that we’ve thought about, and then testing and debugging it.

We get simpler designs that are tested well enough to give us confidence that they work. And that’s what going fast is really about.

This Isn’t Automatic!

However.

When a less experienced person does these things, or even the oldest, if not most experienced person I know does these things, they do not work perfectly.

I always make mistakes in the designs I imagine. I always waste time working with code that is telling me it’s not right. I always have trouble writing tests and I write tests that are fragile and I miss tests that I need, and every darn thing that can go wrong goes wrong.

The difference between me today and me a half century ago, aside from the fact that I’m a hell of a lot more tired now than I was then, is that I create less waste, I have less trouble, my tests are less fragile and more often suffice. Every darn thing that can go wrong still goes wrong … and I tend to catch it sooner.

It’s almost as if I’m not better at design, I’m just better at spotting bad design sooner. You’d hope so, I’ve certainly done a lot of bad design on my way to some occasionally decent design.

Practice, Man, Practice

You know the old joke. But it’s true. We learn to design by reading about it, studying it, doing it, and writing about it.

One of the advantages to being incredibly old is that I have been exposed to almost all of the thinking about software design throughout history. I learned languages with specialized purposes to allow design of particular kinds of programs. I learned to draw flow charts–I even had a bunch of those templates! I went to classes on UML. I learned structured design directly from Larry Constantine. I learned about object-oriented programming from the very best in the trade.

If I’m any good at this stuff now, it’s because some of that information stuck, and because I kept trying and trying and trying to use it.

Practice, folks, practice!