I haven’t fully recovered from my winter snowboarding vacation, so I haven’t quite gotten back into the writing groove yet. (It also doesn’t help that I’m trying to help a First Robotics team get started for the first time, so yesterday I sat down to write something up and instead skimmed through this year’s rules manual…) But that’s okay, because one thing I did do over vacation was review the last year of posts on this blog, and start thinking about organizing things into a book outline. So let’s talk status update!

The good news is that I think I’ve got a good pile of topics that’s going to make a great book. The bad news is this is hell to figure out how to organize. I was hoping that by writing these posts, I’d get some more clarity into the order things could be presented in, but everything is just related to everything else.

So today’s post is a little bit of “how the sausage is made.” I’m going to talk about the organizational problems, mostly in the hope that it’ll help me figure them out. Using my blog to do rubber-ducky debugging, as it were.

Themes aren’t chapters

One of the first problems I had to sort out in my own head was that I often organized my thoughts in terms of themes. I had written about the book in terms of themes in the (now old, in need of updating… later this month) book description. For example, one important theme concerns the universal, mathematically-rooted aspects of design. This includes how the expression problem gives us 3 basic ways of designing types. These are eternal things that won’t ever change when it comes to designing software.

But they don’t really constitute a chapter, or their own distinct part of the book. There is a significant concentration of this theme when it comes to types, and types might be a chapter, but there are other universals that don’t make sense there. And there’s going to be more about types in this book than just universal truths.

So in the first place, I needed to ban myself from trying to concentrate certain things in certain places, and leave posts on those themes free to float apart. Although I probably can’t resist the temptation to try to discuss these alternative organizations in the introduction…

Dependency loops aren’t just for programming

Part of talking about testing wants me to talk about types, and part of talking about types leads me to want to talk about testing. Hooray!

The good news here is I think the most intransigent of these loops can be handled just because this won’t be a book for novice programmers. I can talk about testing before the chapter about testing, because you’ve written programs and test suites. You have experience here. We’re not starting from scratch.

General organizational structure

One of the tempting things to do is start with the basics, and work your way up to the higher-level topics. But I think we actually want to do the opposite with a book about design. A part of why software design is hard is that your head is filled with the details when you’re programming, and as a result, you’re not actually seeing the bigger picture.

So I want to organize the book around the more important topics first: seeing the bigger design picture, and then we can zoom in on certain aspects of it here and there.

A tentative outline

Here is my current attempt at a series of chapters:

  1. Design, context, and trade-offs. Instead of being vague about context and trade-offs, let’s dive into what they are and how to deal with them. “Everything is a trade-off” might be true, but it’s terribly unhelpful advice. Example: System boundaries.
  2. Unlearning from experience. This book isn’t for complete novices, but that means we have to unlearn some of the things we think we’ve learned from experience. Example: Inheritance and OO design.
  3. Getting the human brain to behave. “Missing the forest for the trees” is probably the biggest problem humans have designing things, so we need techniques to help us focus on what matters. Example: Documentation as design tool.
  4. Modularity. We’ve got to think about the bigger picture, so let’s start with putting that picture together. Example: Module diagramming.
  5. Abstraction. Each module exposes some abstractions, so how do we think about them? Example: Various kinds of interfaces.
  6. Types. Once we’re below the level of modules and dependencies, design is all about types, even in dynamic languages. Example: Variance.
  7. Composition. One of the most important ways we can design types is with composition in mind. Example: Forms of composition.
  8. Testing. Although one might try to consider it a special topic, testing is of paramount importance for humans. We’re just too flawed at writing programs; we need to design around testing for bugs. Example: Property testing.
  9. Focused topics. So far we’ve covered on this blog:
    • Concurrency.
    • UI / MVC.
    • Security.
    • More coming!

So what’s coming this year?

With that (extremely subject to change) outline, I’ve got a few thoughts on things that are missing.

  • I need to write more on designing for composition. I have some plans to talk through a little bit of category theory as part of that.

  • I’m considering some additional focused topics: distributed systems has a few “low hanging fruits” that should be better understood by the masses (the end-to-end principle, for example), not just the experts at building such things. There might also be some things in data structure design or parallelization that might also be general enough to talk about to a mass (programming) audience. Possibly even a broad overview of what web frameworks consist of, I’m not sure. I wonder if I have anything to say about ORM…

  • There are some things that aren’t quite fitting right now. For instance, I want to have a tutorial in this book about foreign-function interfaces. Right now, we have a little intro to that in my blog post about property testing. But I’d like to expand that a bit, and also discuss it in different contexts: for example, some suggested practices for evolving prototypes in dynamic languages into more maturely engineered systems. The use of an FFI may be part of that. I’m not sure where this would fit in the outline I have so far, though.

  • I also want to fit in some thoughts on over-engineering and “future proofing” that I haven’t gotten to yet. Probably this would fit into the section on making human brains behave, since this is a kind of human misbehavior.

  • I want to comment on other design fads, like SOLID. And “design patterns” deserves some critique, too.

  • I want to do a few more reviews of older papers about designing programs. I have a backlog of a few of these, but I haven’t really gotten around to that kind of post, yet.

  • One reader writes asking me to comment on dependent types a bit more, and… I have to think about that.

So, that’s at least some of what you might expect to see on the blog in the next few months. If you’re hoping to hear about a topic and I haven’t mentioned it yet, let me know! Or, if you’re just excited for something in particular, I always enjoy that feedback, too.