Metric reviews that change decisions
A metric review is only worth its hour if it changes what someone does next. This lesson shows you how to build a dashboard around decisions, diagnose a sudden drop or spike in a fixed order (data first, then size, mix and segments), tell signal from ordinary noise, and run a weekly business review that ends with owners, actions and dates.
By the end you can
- Design a dashboard that answers "are we on track, and where do we act?"
- Diagnose a sudden change in a fixed order: is it real, how big, where, why
- Tell a data bug from a real change, and a change in mix from a change in behaviour
- Judge whether a change is a signal or ordinary week-to-week variation
- Run a weekly business review and write a one-page diagnosis that ends in a decision
The everyday picture
A patient arrives at a clinic with a temperature of 39.5 °C (about 103 °F). A good doctor doesn't start with antibiotics. First she checks the thermometer: does it work, was it in place long enough? Then she asks when the fever started and what changed: a trip, new food, someone at home with the same thing. Then she examines one part of the body at a time until one shows the problem clearly. Only then does she treat, and she books a check-up to see whether the treatment worked.
Most bad decisions made from dashboards skip the first step. A team sees a number fall, holds an urgent meeting and starts fixing stores, riders or the app, when the thermometer was broken all along. The rest skip the last step: they act, and never check whether the action changed anything.
Dashboards built for decisions
A team's main dashboard should answer two questions in under a minute: are we on track? and where should we act? The easiest way is to lay it out like the metric tree from pm-21: the north star at the top, the inputs below it, the guardrails beside them, each with its owner.
Every chart on it needs five things:
- A definition in one line, with its source (pm-24).
- A target or limit, so nobody has to remember what "good" is.
- Comparisons: last week, the same week last year, and the plan.
- Enough history to see normal. Amazon's weekly business review, as Colin Bryar describes it, uses a "6-12" chart for most metrics: the trailing six weeks on the left, the trailing twelve months on the right, with last year as a faint line behind.
- Annotations for releases, campaigns, festivals, heavy rain and data incidents, so the next person doesn't rediscover them.
Keep the main page short and put the slices (by store, hour, app version) on pages underneath. Separate monitoring from review: anything that needs action within hours, such as sync failures or a city's on-time rate falling below 85%, belongs in an alert to the person who can act. The weekly review is for trends and decisions, not for discovering that something broke on Tuesday.
Quick check
Which change makes the Store app dashboard more useful for decisions?
Targets, comparisons and annotations let a reader tell at a glance whether a change matters and what might explain it. More metrics bury the important ones. Hourly charts suit monitoring, but in a weekly review they turn ordinary noise into false alarms.