The Missing Denominator

Cold open · A/B

The number on your desk

It is the morning of the budget review. An AI summary lands in your inbox with a headline and a recommendation already attached: “Churn dropped 40% in Q3 after the new onboarding — double the budget.” The room will want to act. Here are two ways to handle a confident, fluent number. One of them is gambling. Pick the one you would trust to carry a real decision — then check your reasoning.

!

The fluency of a claim tells you nothing about whether the number underneath is sound. An AI phrases a figure built on two data points with exactly the same polish it gives one built on two million — so your usual instinct (“this sounds shaky”) can’t catch a bad one. The rest of this lab is one move, run for real: ask “out of what?” before “so what?”, shepherding this exact claim through its lifecycle while a Trust Meter tracks how much weight it has actually earned.

The lens · order & map it

Five questions to run on any number

You don’t need to memorise a statistics textbook to interrogate a claim. You need five plain questions you can run in about thirty seconds. They spell D.E.N.O.M. — and the first one is the one most people skip, which is exactly why it names the lab.

D

Denominator

Out of what? Count or rate — what’s the base?

E

Edges

What window or group — and what sits just outside it?

N

Numerator

What is actually being counted, and how defined?

O

Origin

From where, by what method, how many?

M

Meaning

Does the conclusion follow? Name the confound.

1. Put the lens in reading order

The five questions are shuffled below. Arrange them into the order the lab runs them — start at the base the number rests on, and end at the conclusion drawn from it. The insistence of this lab: “out of what?” comes before “what does it mean?”

2. Which letter catches each trap?

Every classic data trap is caught by one of the five questions. Sort each named trap into the D.E.N.O.M. letter whose question would have stopped it. Click an item to move it between bins, then check.

D — Denominator out of what?

E — Edges window / group

O — Origin source / sample

M — Meaning does it follow?

Unsorted traps

Multi-round simulation · carried state

The Trust Pipeline

Here is the live loop. The claim — “churn dropped 40% in Q3” — starts at 50% trust: plausible, unproven. Each round you choose how to handle one stage of its lifecycle. Interrogate it well and the number earns trust toward the decision-grade line. But the meter can move down. Accept a figure on its fluency, or wave a stage through, and you stake trust the number never earned — which you’ll pay back the moment the missing denominator surfaces.

Trust this number can bear 50%

Stage 1 of 4. Run the lens on each stage, or wave it through.

The claim under interrogation: “Churn dropped 40% in Q3 after the new onboarding launched — we should double the onboarding budget.”

Stage — what you’re inspecting Stage 1 · Origin

Assemble the rate · read the chart

Read it like an auditor

1. Build the rate — the denominator is a choice

The headline “40% drop” is one of several true sentences you could write from the same data. The number swings depending on what you put in the numerator, what base you divide by, and which window you measure. Assemble the statistic yourself and watch the headline move.

−40% churn change, as the AI framed it

Numerator — what counts as “churn”?

Denominator — out of what?

Window

How decision-grade is this framing? 20%

2. Click the lie in the chart

The AI also built a slide to sell the 40%. A tall green bar towers over a short grey one; the shape persuades before anyone reads the small print. Read the axis and the caption before you read the bars. Click every span that breaks honest charting, then check.

Click each problem you can find, then press Check.

Causation · sample · who’s missing

Confounds and survivors

1. Which read of the churn drop is stronger?

The redesign launched in Q3 and churn fell. The tempting sentence writes itself: the redesign worked. But something else changed that quarter too. Pick the framing that would survive a skeptic, then check.

2. How many even finished onboarding?

The “40% drop” was measured among completers — users who finished the new onboarding. Of 10,000 Q3 signups, what share do you think actually completed it? Drag your guess, then reveal the number that decides whether this metric describes your users or just a sliver of them.

The denominator of the celebrated metric is “completers.” If most users never complete, the headline describes almost no one.

Your guess: 50% of signups completed onboarding

0%50%100%

3. Spot every group missing from the data

A growth analyst defends the budget: “Our retained power users nearly all completed the new onboarding — it drives retention.” That sentence studies survivors only. Select every reason the data can’t yet credit onboarding for retention, then check — the skill is exhaustiveness.

Make the call · footnote · keep this

The verdict

1. Your call to the VP

You’ve traced the number to its source, sample, denominator, and chart. Now the VP wants a recommendation on the onboarding budget. Data literacy ends in a calibrated decision, not a binary — each call has a consequence. Choose, watch it play out, then try the others.

What happens next

2. Write the honest footnote

Every number you share should travel with its caveat — the denominator, the window, the sample. Write the one-sentence footnote you’d attach to the churn figure in the deck. Then self-assess against the checklist and compare to a model footnote.

Your one-sentence footnote on the churn number:

3. The verifier’s pocket card

Five questions you can run against any number in about thirty seconds. Screenshot this.

D.E.N.O.M. — interrogate the number

DDenominator — Out of what? Is this a count or a rate, and what’s the base?

EEdges — What window or group was chosen, and what sits just outside it?

NNumerator — What is actually being counted, and how is it defined?

OOrigin — Where’s the data from, how was it gathered, and how many?

MMeaning — Does the conclusion follow? Name the confound.

4. Field guide

One page to keep. The named traps the lab walked through, and why AI sharpens them.

1No denominator

A count or % with nothing to compare against
“1,200 reports” — out of how many?
The rate is almost always the story

2Bent edges

Cherry-picked start date (“up since April’s low”)
Truncated y-axis exaggerating a small gap
Ask what sits just outside the window

3Vague numerator / origin

“Confusing” lumps ticket + tick-box together
n=22 self-selected beta users isn’t “users”
Divide 100 by n to see what one person is worth

4Cause from coincidence

“Moves together” sold as “causes”
Name the confound; check if the arrow reverses
State it as an association to test

5Hidden splits & survivors

Simpson’s: pooled total can reverse the subgroups
Survivorship: the dropouts were never measured
Ask who fell out before the count

6Why AI sharpens it

Fluent confidence is independent of data quality
No range, non-reproducible, launders baked-in flaws
A laundered number travels further, faster

Reflect — a statistic you repeated this week: what was its denominator, and did you know it at the time? (autosaved on this device):

You interrogated the number.

You took one fluent AI claim and ran it through its lifecycle — traced the source, found the missing denominator, read the chart honestly, named the confound, and landed on a verdict the number had actually earned. You won’t carry a checklist into every meeting. What you carry is the reflex: a half-second pause where “out of what?” arrives before “so what?” — cultivated, not commanded.

Back to the Lab overview