Prioritizing Assumptions

In a previous blog I talked about the importance of stating and validating strategic hypotheses and assumptions. Once the team gets the hang of thinking in these terms and writing down hypotheses, the extensive list of what you don’t know, or what you think you know but are not sure about, can be quite daunting. In early stages of product development, it seems there is more we don’t know than we do. So what do you do when there are too many hypotheses to reasonably validate? My team faced just this challenge when working in a tech company that was creating a new product.

A Segmentation Matrix

For our research to be most impactful, prioritization was in order. We didn’t have the time to pursue all of the assumptions, but it was unclear which assumptions were the most important to research. My team tried several different ways to prioritize, but finally landed on an approach that mapped all our assumptions on a 2×2 matrix with two key questions:

• What is the impact to our solution if this assumption is wrong? If we’re wrong, does the reason for the product evaporate? Do we lose an entire market segment or a single customer? Do we impact the cost by a few million dollars or a few cents? We placed items on a spectrum of minimal impact to significant impact.
• How likely are we to base our solution on this assumption? This is another way of asking how core this is to our solution. Is it a fundamental assumption, or just something we’ve used to justify an ancillary feature? We ranked this dimension on the spectrum of unlikely to likely.

Prioritizing with the Matrix

While the placement can be somewhat subjective and imprecise, it was fairly easy for the team to characterize our assumptions with these key questions. What results is a mapping of items into four quadrants:

1. Significant and Likely
2. Significant but Unlikely
3. Minimal and Likely
4. Minimal and Unlikely

These four quadrants then determine the priority order. The most important things to validate are those priority 1 items that are both significant and likely. Only if you have more time and resources you could address priorities 2 (Significant but Unlikely) and 3 (Minimal and Likely).

Priority 4 items (Minimal and Unlikely) can be initially ignored, but it is good to keep them around and periodically reassess to see if they move into one of the other quadrants.

Assumptions, Hypotheses and Using the Matrix

While certainly Mr. Webster would cringe, we used the terms “assumption” and “hypothesis” interchangeably. For the team, “assumption” was gentler and less formal than “hypothesis” which carried with it the suggestion of a formal scientific process the team was hesitant to presume. The question “What are our hypotheses about the customer problem?” was met with blank stares, but when the question changed to “What have we assumed about what’s important to our customers”, the responses started to flow. Since we didn’t want the language to get in the way of our prioritization goal, we used the more approachable term “assumption” to represent a statement we hope to be true or valid. Assumptions can be about anything, but it was helpful to consider the three areas of 1) the customer problem, 2) our proposed solution, 3) our proposed implementation.

Here are a few examples mapped onto the matrix.

• “Customers would like a solution to problem <x>.” If our product plans to provide a solution to problem <x>, then we are likely to base the product on this assumption.   If it is a key selling point, then the impact if we’re wrong will be significant. Thus, this assumption falls into the Priority 1 quadrant, and should be tackled first.
• “Small businesses struggle with problem <y>”. For us this was a priority 4 assumption, since small businesses were not our primary target and a feature addressed at problem <y> was ancillary.
• “Customer will adopt a mobile solution” was a Priority 1 assumption for us, since “mobile first” was our core strategy. It does, however, point out a nuance in this prioritization approach. Since plenty of research exists pointing to increasing adoption of mobile in our customer segment, we were willing to stipulate this as given without conducting additional primary research; we labeled this a “validated assumption”. However, I urge caution in stipulating assumptions, as an overabundance of certainty is exactly what can bite you later.