CATalyst

Everyone wanted data, and the analytics team was the bottleneck. Dashboards answered last quarter's questions; anything new still went into a queue and came back days later. By then the decision had often been made without the number.

And the real issue wasn't access. People who got an answer still didn't always trust it enough to act. So even a fast answer didn't always change anything.

The hard part of self-serve data isn't fetching the number — it's getting someone to believe it. A correct answer with no explanation gets second-guessed; a confident wrong one is worse.

So this couldn't just be a 'type English, get SQL' toy. It had to show its work: what question it thought you asked, what it ran, and what it assumed.

You ask a question. The tool tells you how it understood it (which is where most mistakes hide — what counts as 'active', what 'last quarter' means), lets you correct that, then runs a query against vetted metric definitions and shows you both the answer and the SQL behind it.

Claude handles the language and query generation. A defined set of metrics and dimensions sits in the middle so it can't quietly invent its own version of a number.

React front end → a Node.js layer that orchestrates the request → Claude for understanding the question and drafting SQL → a curated set of metric definitions → BigQuery to actually run it.

The guardrails matter more than the model: every query is checked against the defined metrics, scoped to what the user is allowed to see, and bounded so a vague question can't trigger an expensive scan.

Always show the SQL. Treating transparency as a feature, not debug output, is what turned skeptics into repeat users.

Constrain it to defined metrics instead of opening the whole schema. Narrower, but far more trustworthy — and trust was the whole point.

Let it say 'I'm not sure.' When a question was ambiguous, asking back beat guessing confidently.

It reached 40–50 stakeholders and effectively ended the ad-hoc analyst request — everyday questions got a self-serve answer instead of a queue, which freed the analysts for the genuinely hard work.

The bigger shift was cultural: people started checking their own assumptions in the data because doing so had become cheap and safe.

In data tools, trust is the product and accuracy is just the price of entry. The explanation layer drove adoption more than model quality did.

The boring foundation — agreed-on metric definitions — is what makes any of this safe. I'd invest in that earlier next time.

The hardest thing to design was teaching the tool to be honestly unsure instead of smoothly confident.