Deciding when the data won't settle it

There is a comfortable myth in data work: that if you analyze hard enough, the answer appears and the decision makes itself. I believed that for a while. Then I spent enough years close to real decisions to notice how rarely it happens.

What actually happens is this. The analysis gets you to a certain point. It rules out the obviously wrong options, sizes the rough shape of the thing, and then it stops. And in the space that is left, two reasonable people can look at the same numbers and want to do different things. That is not a failure of the analysis. That is the normal condition of any decision worth making.

The mistake is treating that moment as a signal to go get more data. Usually it is a signal that the decision has arrived.

The tell that more analysis will not help

I have learned to watch for a specific pattern. Someone asks for more data. You produce it. The conversation does not change. Then someone asks for a different cut. You produce that too. The conversation still does not change.

At that point the disagreement is not about the number. It is about something the number cannot resolve: different beliefs about the future, different appetites for risk, or different ideas about what the company should be. Running one more query is a way of avoiding that conversation while looking productive.

So the useful question, when a decision is stuck, is not "what else can we analyze?" It is "if this analysis came back either way, would anyone change their mind?" If the honest answer is no, you have found the real disagreement, and it is not a data problem.

Ask what you would have to believe

The single most useful move I know in a stuck decision is to flip it around. Instead of asking "what does the data say," ask "what would have to be true for this to be the right call?"

It changes the conversation completely. Suddenly you are not arguing about the number, you are listing the assumptions underneath each option. And assumptions can be examined. Some turn out to be things you actually can check cheaply. Some turn out to be beliefs about the future that nobody can check, in which case at least everyone can see clearly what they are betting on.

I have watched this dissolve arguments that three more weeks of analysis would not have touched. The disagreement was never about the data. It was that one person believed customers would tolerate a change and another did not, and nobody had said so out loud.

Decide how reversible it is

Not every decision deserves the same rigor, and treating them as if they do is its own failure. Before spending effort, I try to sort the decision into one of two buckets.

Some decisions are easy to walk back. Try it, watch what happens, undo it if it goes badly. For these, the cost of deliberating usually exceeds the cost of being wrong, and the fastest way to learn is to just do it.

Other decisions are expensive or impossible to reverse. Rebuilding on a new platform, restructuring a team, changing pricing for existing customers. These deserve the slow, careful version.

Most of the pain I have seen came from getting this backwards: agonizing for weeks over something easily undone, and rushing something that could not be. Sorting the decision first is often more valuable than any analysis you do afterward.

Write down what you expect

Here is a habit that costs almost nothing and pays for itself repeatedly. Before you act, write down what you expect to happen, and what would tell you that you were wrong.

Two sentences is enough. "We think this will lift conversion into the mid teens within a quarter. If it is still flat after six weeks, the targeting theory was wrong."

This does two things. It stops you from quietly rewriting the story afterward, which everyone does without meaning to, because a result that seemed surprising in advance always feels obvious in hindsight. And it makes it possible to actually learn. A decision with no written expectation cannot really be reviewed, because there is nothing to compare against.

Get comfortable saying "this is a judgment call"

The phrase people avoid, and probably the most honest one available, is: the data narrows this to two reasonable options, and I am choosing this one, for these reasons.

That sentence takes a certain amount of nerve, because it drops the pretense that the analysis decided. But it is far better than the alternative, which is dressing a judgment call up as an analytical conclusion. That dressing-up is what erodes trust in data teams over time, because when the call goes badly, and some will, everyone discovers the analysis was doing less work than it appeared.

Owning the judgment is also what makes it reviewable later. "We chose this because we believed X" can be examined. "The data said so" cannot.

What decision intelligence actually is, to me

Not a class of tools. A practice. It is knowing what the analysis can settle and what it cannot. Making the assumptions visible instead of letting them hide inside people's heads. Matching the effort to how reversible the choice is. Writing down what you expect so you can learn something afterward. And being willing to say out loud that at some point, a person decided.

The goal was never to remove judgment from decisions. It was to make judgment better informed, more honest, and easier to learn from. The number gets you to the edge of the decision. Someone still has to make it.