Design is a Search Problem

Design, code, and writing all feel like everything you do is in a nightmarish state of total failure until the moment it is not. What I'm going to talk about is not so much these design principles but instead how we can establish a process for design that recognizes the inevitability of failure and the necessity of trial and error. We need to explore as many possibilities as efficiently as possible in whatever time that we have. We need to be able to spend a lot of time exploring things that aren't going to work in order to find the things that do work. Hopefully, if you have a robust creative process like this, you can succeed even if you don't know what you're doing. Design is a Search Problem. Keynote presentation for OpenVis Conference 2014 by Mike Bostock. youtube

Talking about your process and talking about design [comes with a] fear that you will discover that there is no method to the madness, and that whatever past success that you've had isn't really the result of any underlying reason. It's just lightning striking. It's happening to be lucky. When we talk about our own work it's convenient to sort of ignore all that intermediate failure and the mistakes that were made and that uncertainty and just to just look at the polished end result and to brush away those mistakes that were made. That sort of lets you pretend that you know what you're doing. But, of course, that's not really helpful if you want to learn design.

If you want to learn design, it's helpful to understand that everyone struggles with it and makes mistakes. So I'm going to talk about design anyway because I know at least one thing about design which is that we collectively do not know many things about design. I'm going to say that design is hard and because design is hard, it is challenging to articulate these effective design principles. You know they require careful interpretation, thoughtful application, and ultimately you never even know if the design principles that you hold dear are the reasons for your success. So what I'm going to talk about is not so much these design principles but instead how we can establish a process for design that recognizes the inevitability of failure and the necessity of trial and error.

The title of this talk and the thesis of this talk is that design is a search process, a search problem. Which is to say that we have these principles for design and they are necessary to guide us but they're also not sufficient they don't dictate what a successful design will be they're not blueprints. There are ultimately too many unknowns for the principles to be sufficient to tell us exactly what we need to do. What we need in a successful design process is to be able to explore as many of these possibilities as efficiently as possible so that in whatever time that we have—because we're always facing particular deadlines, whether it's a daily graphic that we're working on, or a longer-term project where we might have a couple weeks or even a month, we need to be able to spend a lot of time exploring things that aren't going to work in order to find the things that do work and that hopefully if you have a robust creative process like this that you can succeed even if you don't know what you're doing, like me.

This is an extraordinary talk. Bostock begins with a powerful metaphor. Then shows visualizations of several different sort algorithms in a way that warms the viewer's brain for the cognitive work of perceiving a visual representation while developing a mental model, even a felt sense of the algorithm visualized. Then shows visualizations of maze construction and visualizations of pathfinding through generated mazes to develop an intuition about what he really means by the title of the talk, that design is a search problem. He's shown different algorithmic approaches for searching for a path through a maze. With that metaphor so firmly established he proceeds to demonstrate and talk about the tools and process used at the New York Times to search for a good design to communicate complex data and ideas to NYT subscribers. Finally he concludes with big ideas about how to apply these insights to design a process to enable wide exploration and how to develop a sense of when in the lifecycle of a design to begin attenuating variation in order to meet production deadlines.