Why use confidence scoring?

Confidence scoring is a method for understanding checkpoints of information needed in order to make a decision.

Our natural tendency is to believe we need all the information available to make a decision, but that's far from the truth. We just need enough information. In fact, spending too much time collecting information is costly and damaging.

Confidence scoring helps us understand when enough is enough since our brains will constantly search to fill the knowledge gap making us spend unnecessary time and paralyzing the decision making process.

The purpose of the confidence score is to segment decisions at a high level between those that can rely on lightweight process and lean heavier on intuition vs decisions that need more information. This method pairs well with the speed score. If the speed score lets us know how quickly we can make a decision, the confidence score gives us an idea of what tasks are required to build confidence past the threshold.

The confidence score

Initially, we want to get a gut reaction about how confident we are in what we know about a decision. 'What we know' is not necessarily the amount of information we have, but how confident the team is that:

• They have the right experience in the space or have made countless identical decisions.
• There aren't any 'known unknowns'. They have the information they need.
• Nothing can change their mind.

Though every team may have a different definition of confidence, asking these three questions based on the criteria above is a good start.

• Do we believe we have expertise in this space?
• Do we believe any new information will change our mind?
• Do we believe we're aware of the risk?

If we don't have a ton of expertise (maybe we're looking to enter a new market or implement a new technology), believe that there's quite a bit of information that could change our minds and we see quite a bit of risk, then we'd see a low confidence score.

The confidence threshold

The confidence threshold is helpful for drawing a hard line at 'just enough' information, but also combating action bias when we're not as confident as we might think.

The threshold should be a statement that articulates what it means to be confident in a decision. It shouldn't be rigid. In our example above, if we don't think we have the expertise, then we need to go include expertise. For this criteria, our confidence threshold isn't to "Interview 5 experts", but more-so "We feel we've had expert opinions". It's not meant to be a goal, but something that makes you think. For one decision, this may be a single interview; for others, this may be an entire research effort.

These are relatively simple statements to score on a 1-5 scale and help tremendously when segmenting decisions. If we're under the threshold, we need to understand what will get us closer to the threshold. If we're over the threshold, there's no more need to spend time getting more information.

How to get started

Start with a single question with every decision; Is there any more information that would change our mind? The answer to that question can immediately segment the decisions that truly need more information and those that don't. On top of that, you can build on some of the basic questions we've proposed here. For example, having more relevant expertise means we can lean more into intuition over information.

For questions like "Do we believe we're aware of the risk?", using a Johari Window for Donald Rumsfeld's 'known knowns' is a helpful tool. You can learn more about that method here.

‍