Design and Uncertainty

“We want a designer who really BELIEVES in their design. They should be willing to fight for it.”

“But what if your belief is wrong?”

When I was graduating from CMU’s HCI program a while back, I was asked to chat with some of the prospective new members of the program about why I found the program valuable. CMU put a great emphasis on research and usability testing; having seen the power of data to improve my ideas, I had learned that I was wrong a lot of the time. I told the students that the HCI program taught me that I could be wrong. It taught me humility.

At the time I was coming off my Ayn Rand phase. I had believed it was morally virtuous to have an opinion that I fought for against all the world. Now, in my old age, I still believe this is true: but only if that opinion is formed by a relentless dedication to the Truth. The “problem” is that the Truth is an “arch wherethrough gleams that untravelled world, whose margin fades for ever and for ever when I move” (Tennyson, Ulysses). Our measured knowledge of the world is imperfect and localized. We are biased in what we think we should measure. We have limited time and resources to objectively measure, with FDA-like rigor, our software design decisions.

It is a Good Thing that in the software industry we’ve recently gotten obsessed with data. I briefly knew DJ Patil at LinkedIn and loved the work he did. It wasn’t a challenge to my authority; it was a marvelous input that helped me do a better job. A good designer, a good product manager, a good businessperson: they’ll all seek to enrich their positions with knowledge of the world.

The people who are endowed with the title of “designer,” or “UX person,” are often the people who are charged with challenging the data. They’re the people at the organization who are asked to look to deep understandings of the human condition and from that derive a set of recommendations about what the product should do or be. At the same time, our recommendations only have clout (that’s with a “c”, not a “k”) if they’re more real, more Truthful, than the measured data we can collect. The value of a great UX / product person lies in them being able, presumably, to see deeper human and social truths than are afforded by the statistics we collect. And at the same time, the statistics and usability studies hone our intuitions and our knowledge of what people actually do when they encounter our products.

I’ve recently been re-reading French Women Don’t Get Fat by Mirelle Guiliano. Her philosophy, the French philosophy, shuns much of modern statistics and medicine about weight management. Michael Pollan similarly argues that if you want to be fit and healthy you should reject most modern food science and, to paraphrase: don’t eat something your great-grandmother wouldn’t recognize; eat real food; not too much; mostly plants. Despite our plethora of research into food, nutrition, and vitamins, our “intuition” and “ancestral learnings” about food really seem to be the best way to stay healthy and fit. Vitamin this or carbohydrate that; those isolated experiments are still too crude to give us better answers than our “intuitive” understandings of what to eat. It’s because what we eat is deeply entwined with our human needs for pleasure, sociability, nutrition, and psyche in a way that we don’t yet know how to measure accurately.

Much of the HCI curriculum when I was at CMU was focused on helping software designers understand domains that they didn’t understand. This was in the late 90s, concurrent with the rise of consumer internet, but before tech got involved deeply in our lives. We focused on research to understand domains in which we weren’t expert, like medical, weapons, or manufacturing technology. Our mantra was “I am not the user.” Once designers started working on all these consumer technologies, our direct real-life “my mom” intuitions could often, successfully, be leveraged to create good products. Gmail is a great example of this: it’s a marvelously designed product that was largely based on the correct, complex intuitions of its original team.

The problems come when you don’t admit, as a designer or product person, that intuitions based on your mom or yourself may or may not extend to what most other people actually do. So a designer who seems like a hotshot Howard Roark out of college may be great for that one particular project. But when you ask him or her to work on a design for a domain that they don’t “intuitively” understand (since they don’t have years of experience being within that particular community) they’ll flail if they don’t know how to turn to research and data to inform their opinions.

Now that I’ve been in industry for a while I’ve honed my intuitions. I’ve had a lot more data to inform my notions of what works and what doesn’t. I’ve been fortunate to see my designs launched: some of them were epic wins and some of them were failures. Personally, this has created in me a quest to continuously refine my intuitions based on real measurable data.

When I was at ChoiceVendor, the startup that got acquired by LinkedIn, I learned that I wasn’t thinking vigorously enough about the big picture: what makes someone realize they need your service? Were we being too optimistic: did they really need our product as much as we hoped they would need it — enough for us to create a business out of it? I was overly focused on the intimate details of usability for what a startup needed; we didn’t focus enough on our value proposition and on acquiring new users; I spent too much time on the internal usability of our product — which was really good, but not enough.

When I was at LinkedIn I redesigned the signed-out homepage. Instead of articulating three bullet-pointed value propositions, my design showed people telling a visitor what they used LinkedIn to do. Why? Because I had talked to people who had heard they should use LinkedIn, but didn’t really understand why or how. So we showed visitors what they could get from using LinkedIn, and how it would help them be great at their jobs. (The conversion rate went from about 4% to 9%.)

As a designer, as a UX person, because I’ve seen my shit fail, I’ll only rarely guarantee that what I’m recommending will work. It’s very rare that I’ll back against a wall and fight to the death for a particular design. Because, you know, I may be proven wrong.

But I will always say: based on what I know right now, this is the best design for what we’re trying to do. I’ve thought this shit through, pulling from the best of all my experiences, criticizing myself and my assumptions at every step, admitting where my (our) knowledge is imperfect and flawed. And with all this, I think it creates the effect that we want: makes sense to the people who use it, solves a real need; and solves it in a way that makes them efficient, cheerful, and empowered. They leave their experience with the system thinking “that was cool. I’m glad I got that done.”

But show me the data and I’ll throw all of it in the trash.

More: Design like you’re right, listen like you’re wrong by John Lilly.