Oh Sh#*! (Or Managing The Unknown Unknowns)

If you’ve ever worked on something even remotely disruptive or innovative, you know about unknown unknowns. They are the unexpected things that happen that it is impossible to predict. Customer feedback that doesn’t align with your assumptions. Unexpected alternative uses of your idea for other purposes. The competition reaction to your new offering. None of which you can predict.

So then how do you deal with them when they arise?

Types of Unknown Unknowns

First it is important to recognize that not all Unknown Unknowns are created equal. Some are truly “unknowable”, while others stem from our own internal biases and assumptions. How and when you deal with these different types is different.

Known Unknowns

While these are not actually Unknown Unknowns, they are sometimes lumped in with them. Known Unknowns are things that we know we don’t know at the outset. These can be things we are going to deal with through agile iteration or experimentation such as design details or feature details.

The only reason I mention them is that they need to be planned for. When you identify known unknowns at the outset of a project, make sure you make room in the plan to work through them at the right time. They are only a critical risk if you don’t plan to deal with them.

Be ready to be surprised by the outcomes of your experiments and ready to pivot to address new information. Known unknowns have a tendency to create churn if you are married to your original hypotheses.

Unknown Unknowns from Biases or Assumptions

We all like to believe that when the unknown unknowns arise that they couldn’t possibly have been planned for, but the reality is that there are many of them that we simply didn’t preflect for.

In many cases our own assumptions of our industry or customer or our own biases about our definition of a problem get in our own way. We think of things through traditional lenses for our industry. We take our own personal experiences and extrapolate them to the broader customer base.

Industry Assumption “Unknown Unknowns”

For example, one project I worked on was the implementation of a new credit card system for a major financial institution who had recently acquired a new card portfolio. I was brought in late in the program because it was struggling to get to completion. When I got engaged, the teams were fighting over reporting formatting, struggling with the supporting operating model, and struggling to understand the customer base.

Most of the problems they had stemmed from the fact that they had an existing credit card business that was thriving, an existing credit card platform that had a set of features and capabilities, and an existing operating model for how they supported it. They had lots of round holes and they were struggling to fit the square pegs into them.

In this case, the team working on the project failed to preflect on the new customer business they had acquired. This customer base was significantly different to the one they currently serviced, and their expectations were different for how they would be treated. Many of the details that the teams were struggling with were not important to new customers, and many issues that were critical to success were not being considered.

In the program, these issues seemed huge and were stopping progress. They also seemed like issues that couldn’t have been predicted. In reality it was the lack of preflection and questioning that prevented the team from getting focused on what mattered. And many of these issues could have been prevented with a premortem.

Personal Experience “Unknown Unknowns”

Another project I was working on related to another organization and a service experience they were trying to improve. It was a competitive market and a commodity product, so the intent was to focus on the customer experience to create differentiation.

In this case the team took a preflective approach and did direct customer interviews and mapped the customer journey before they got started. They used external resources to capture and hear what customers liked and didn’t like about the existing experience with both customers and non-customers. They examined their own customer base to learn what made them choose other options when they elected to purchase elsewhere.

And then they ignored it all.

While the feedback clearly showed that while the product was a commodity, their customers didn’t understand it. It was complicated, a major purchase, and something they were significantly worried about because of the commitment they made with their purchase. It was a long-term purchase decision, and the fear of regret made it challenging.

Customers wanted to be helped through the process. They wanted someone to help them understand what they were buying in simple language. They didn’t care about the technicalities of the product that were available, they wanted it translated into what it meant to their own needs.

But the team working on the problem had their own personal perceptions based on their own experiences. They didn’t recognize that their own deep understanding of the product meant that their issues weren’t reflective of the average customer.

So while their real customers asked for help in simplifying the experience and helping them to understand it, the team focused on technical challenges which created pains in their own experience.

Ultimately customers were underwhelmed with the new changes. Sure things were faster, more streamlined, and more digital. But they didn’t care about any of that. The product was still complicated. The purchase was still complex. And the fear of regret was still there.

In this case, an outsider view would have helped the team focus on the customer needs and avoid the pitfalls of their own experiences.

Unknown Unknowns that are Unknowable

Sometimes there are truly unpredictable events that happen in our work. Hardware might fail causing a service outage. Fires break out preventing you from opening the store. A power outage takes out an entire region at the worst possible time.

You truly can’t predict some risks. These are “black swan” events. But you need to know how to deal with them when they arise.

First, recognize that you can’t plan for the truly unknown unknowns. No amount of project slack or preparation will help you find them in advance. If you try, you’ll just waste time getting started on the important work you had planned.

But when they arise, don’t ignore them or delay dealing with them. If you have a major unexpected event you need to address it immediately.

The best way to deal with the unknown unknowns is as a group. This is very suited to our agile work groups, but it does mean parking your backlog to address the bigger issues.

Pull everyone together to focus on the problem and then:

1. Start with open minds. This is an unexpected position to be in. You need to eliminate biases and assumptions before you begin.
2. Lay out what you know and what you don’t. Don’t ignore the important elements of the unknown unknown that haven’t surfaced yet.
3. Brainstorm on the right questions to be asking. This is often called “questionstorming” and it is extremely useful to make sure you’re dealing with the right problem.
4. Have the group assign out the required analysis to the people best suited to bring back the answers to your questions.
5. Once you expand what is “known”, you can brainstorm the problem statement you need to deal with.
6. Now iterate and work through the issues. Make sure you have regular and short iterations with checkpoints to make sure that you remain on the right path. If something doesn’t feel right or you learn something new that challenges your original assumption, start again.

Plan and React to Unknown Unknowns

If you start by classifying the known unknowns and plan for them, address unknown unknowns caused by assumptions and personal experience through preflection and premortems, and have a prepared plan to deal with real unknown unknowns you can manage what comes at you.

You can’t plan for everything, so don’t try. Do the work. Be prepared to respond. Address the things you can in advance. You will maximize your chances of success.