Get started

This writing project has been going on for over 6 months now (at time of writing in 2018). While the full archive should be good reading, here are some of the most important pieces:

Why do I want this book?

I think there is a substantial gap in how we learn programming. In the beginning, we stumble through the basics, like our first programs, data structures, and basic tooling like version control. In the middle, we briefly get a little exposure to the idea that code is designed, often in the form of some boiled-down object-oriented ideology. (Fields in your classes should be private! Why? Well public fields are just immoral of course.) In the end, we branch out into a breadth of introduction to lots of specialized topics, like operating systems, algorithms, web development, programming languages, artificial intelligence, and so on. And then, the real education begins, when we go out and get experience writing (and maintaining) real programs.

In case it’s not obvious, I think that middle bit needs some work. Part of the problem is that it’s very hard to teach design without experience. Part of the problem is that object-oriented ideology has too strong of a grip. (What are you going to go on to write? C++, Java, Python, Javascript. It’s not that it doesn’t make sense to talk about OO, but it’s terribly incomplete as a way to think.) And what we do teach about design is often done badly: unjustified rote mimicking of rules handed down from on high, or “good designs” that are actually just workarounds for the bad designs we’re stuck with. There’s also some things we should talk about when it comes to design that, inexplicably, we just don’t. Maybe it comes back to the experience thing, I don’t know.

Or maybe we just don’t want to portray our field as being a giant collection of hacks for working around decades-old poor decisions that we’re now permanently stuck with.

The best advice we give programmers is to go out and learn several different programming languages. Languages frequently codify a particular approach to thinking, often called a “paradigm” (though I’m not sure I like how people talk about “programming paradigms”). If we really knew the perfect way to design programs, maybe it’d eventually manifest itself in a perfect programming language. We obviously don’t have such a thing. But we do see value in learning multiple different languages. We can think better about design, even if we don’t have a language to perfectly express these thoughts with.

What is this book about?

I want to write a book about code design. It’s not a book for beginners, though I hope they’ll still find it valuable. It’s not necessarily a book for “software architects,” who are (it seems?) sometimes too far removed from the actual programming to care. I want this book to be an excellent tool for experienced programmers who want to get better at programming – programming, not software engineering process, not management, not specialized subject material (OS, PL, AI, etc), not individual tools or languages, just general programming. Specifically when it comes to design: APIs, interfaces, libraries, languages, any time we have to create something and then get stuck with maintaining that backwards compatibility forever.

I’ve taken a couple of cheap shots at “object-orientation” but it’s not really my intention to disparage it. The issue I take is that there’s an ideology that comes with OO that I think doesn’t belong. It’s just one particular way of looking at design. A great many things are best structured in ways that sometimes map poorly to a purely object-oriented world-view (or just to our current object-oriented languages). Thinking about them as they are, instead of how some our languages force us to conceptualize them, can be helpful. Even if we still write OO programs, we should keep our thinking straight between the solution and the mapping of that solution to within the constraints of the language.

Partly, this blog is a tool to figure out exactly what will be in this book as I go. But I can give my current thoughts about things I plan to include:

  • Universal, mathematically-rooted, concepts. Sometimes software and design is about deeply flawed, very human-invented problems. But sometimes there are parts of design that are rooted in something deeper. Understanding these things will withstand the test of time, never becoming obsolete. As an example, I’ve written about the expression problem, and its effect on how we design our types. (And part 2.) Or about how variance affects design.

  • Tricks our minds play on us, that aren’t going away. We’re humans writing programs, not just any old intelligent creature. And humans are quirky. There are a lot of ways we trick ourselves into having the wrong priorities. As an example, the most important part of the design of an abstraction is its properties, but we usually fawn over how powerful it is instead.

  • Fundamental, but high-level, concepts in programming. One of the things that can sometimes make high-level mathematics inscrutable is that to understand a high-level abstraction, you need to be able to come up with lots of different little concrete examples. There is, I think, a similar problem for programming, except to be able to come up with those concrete examples, you might need a few years of experience. They aren’t as simple. This makes it hard to ever teach high-level abstractions. (See, for example, all of the flailing about on the internet about monads.) We should nevertheless have good resources for experienced programmers (3-5 years perhaps) to come back and learn these things. And it seems there’s plenty here we could be teaching that doesn’t need extensive experience, either. In my post about the expression problem, many people found it revelatory that the “ADT” design approach permits the a library author to introduce new variants and new operations in the future. This isn’t rocket science, but it helps to have someone point it out, and apparently for many people, nobody had.

  • What we can learn from other “programming paradigms”, distilled. One of the best ways in which we try to instruct programmers is by encouraging them to learn other programming languages, with different “paradigms” for solving problems. Some amount of this simply has to be done themselves: it’s hard to define “blue” to someone who’s never seen it. Experience is the foundation of everything. But we should also provide a decent road-map: show what can be learned from different languages, and what’s interesting (and perhaps, too, what’s actually rather mundane, but could easily be mistaken for profound). You may enjoy my in-depth discussion on the problem with inheritance in object oriented languages.

  • Careful discussion of design models, and design case studies on various systems. We often send programmers into the world having taught them little about how to actually do design. We should do a better job of looking at good and bad designs of various libraries, especially standard libraries, where programmers might often look for examples of how to do things right (and often get lead astray). Many programmers can say what MVC is, and many can also do more than just cargo-cult what they’ve seen before. But remarkably few (though granted, I have high expectations…) can explain why. Most disagreements about how to do MVC (fat controller? fat model?) devolve into talking past each other. We should do a better job of looking at many common models for constructing programs, and look at why these models were developed the way they were. Of essays so far, you might be interested in compiler design, build system design, containers, or the Unix philosophy.

I am also considering including some discussion of designing secure software. Sometimes “security” is a specialized topic, but I think it has some interesting overlap with design more generally. And there are some things everyone should keep in mind.

But this plan is not set in stone. Part of the reason I’m here promising to write essays over the next 1–2 years is to give myself time to really think about what should be here. And as a bonus, to allow people to yell at me early on, if I starting going in the wrong direction.

Is this book still happening? (A 2023 update)

Yes, uhhh… eventually.