Product Management line, station 5 of 11
Metrics & Product Analytics
Pick the numbers that matter, read funnels and cohorts, trust your data, run experiments and pilots, and turn a metric drop into a decision.
- 6lessons
- 4 hto read and do
- Intermediatelevel
- 6podcast episodes
- 6checked exercises
- 5simulators
Tools you'll use in this course
The lessons open these with a template ready. They work for your own job too, no course needed.
About this course
Most product teams have plenty of dashboards and very few decisions that came from them. This course teaches the measurement habits strong PMs use every week: a north star with a metric tree of inputs teams can move and guardrails that catch gaming, funnels and activation, cohort retention that shows the leak behind growing totals, a tracking plan that survives offline handhelds and data bugs, A/B tests and store-level pilots read with confidence intervals, and a diagnosis method for the day a number drops.
Every lesson works through Mangrove Market, a fictional grocery chain with 320 stores, a 30-minute delivery app and 1,800 partner shops. You follow Imran, the PM for the Store app and the Partner app, as he builds the metric tree for on-time delivery, finds where stores drop out of the Store app rollout, discovers that festival stock transfers were inflating adoption, sizes an A/B test and runs a matched store pilot, and traces a sudden drop in partner orders to a sales bonus. Real cases from Amazon, Microsoft and Bing, DoorDash, Segment and others show the same ideas outside the company.
Internal tools get full attention: adoption measured as work done in the tool, retention by rollout wave, and pilots when there are only 320 stores to test on. Five simulators let you move the inputs yourself, auto-checked SQL exercises run in your browser, and each lesson ends with a template you can copy, practice tasks with model answers and a podcast episode.
What you will be able to do
- Build a metric tree from a north star down to input metrics teams can move, with sensitivity, owners and guardrails
- Read funnels and retention cohorts to find where and when users drop off, and measure adoption of an internal tool honestly
- Write a tracking plan with exact triggers, a source of truth for every number, and checks that survive offline devices and late or duplicate events
- Design an A/B test, or a matched store-level pilot when you can't randomise, size it, and read the result with a confidence interval without peeking
- Diagnose a sudden metric change in a fixed order, from data bug to mix to segments, and run a weekly review that ends in decisions
Syllabus
6 stops, about 4 h in total. Each stop fills in when you complete the lesson.
North star, input and guardrail metrics
One number should tell everyone whether customers are getting the value you promise. Below it, a tree of input metrics tells each team what it can move this month, and guardrails stop anyone winning by breaking something else. This lesson shows you how to pick the number, build the tree, find the input that matters most and check that the tree still holds.
North star metricsMetric treesInput metricsGuardrail metrics
Funnels, activation and adoption
A funnel shows where people drop off on the way to value, and which fix would help most. This lesson teaches you to define and read a funnel, pick an activation moment and measure time to value, and measure adoption of an internal tool in a way you can trust, including the stock transfers that can make a headline adoption number look better than the work on the dock.
Funnel analysisActivationAdoption metricsSQL for PMs
Retention and cohorts
New customers fill the bucket; retention decides whether it holds water. This lesson shows you how to define retention so the number means something, build and read a cohort table, judge whether a retention curve flattens and compare it fairly with benchmarks, and measure retention for partner shops and internal tools.
Retention analysisCohort tablesB2B retentionSQL for PMs
Instrumentation: tracking plans and data you can trust
Every metric in this course is only as good as the events behind it. This lesson shows you how to write a tracking plan that starts from the questions you need answered, name events so they stay consistent, choose the source of truth for each number, handle handhelds that go offline and events that arrive late or twice, and catch data bugs before they reach a dashboard.
Tracking plansEvent designData qualitySQL for PMs
Experiments: A/B tests, holdouts and store pilots
An experiment is the most reliable way to know that a change caused a result. This lesson shows you how to plan an A/B test (hypothesis, metric, sample size and duration), read a result with a confidence interval in plain words, avoid the traps that fool smart teams, and run holdouts and store-level pilots when you can't randomise users.
Experiment designA/B testingStatistics for PMsSQL for PMs
Metric reviews that change decisions
A metric review is only worth its hour if it changes what someone does next. This lesson shows you how to build a dashboard around decisions, diagnose a sudden drop or spike in a fixed order (data first, then size, mix and segments), tell signal from ordinary noise, and run a weekly business review that ends with owners, actions and dates.
Metric diagnosisDashboardsBusiness reviewsSQL for PMs
Capstone: Prove the 30-minute promise
A realistic project with a rubric and a model answer.
Final exam and certificate
10 questions drawn at random, 70% to pass.
Capstone project
Prove the 30-minute promise
The 30-minute "Now" delivery is Mangrove Market's headline promise, and the managing director wants one clear view of whether it is being kept and what to do next. Build the measurement system for it and use it. Start with a metric tree from weekly on-time orders down to inputs teams can move, including picking time, rider assignment time and stock accuracy, with owners, sensitivity and guardrails. Write the tracking plan for the events from checkout to the doorstep, deciding which come from the apps and which from the server, and how offline Rider app phones are handled. Use the sample cohort table in the retention simulator (or your own) to find which customers stop ordering and when. Design an experiment for "suggest a substitute before packing", where a picker offers the customer a one-tap swap when an item is out of stock, taking care that pickers and riders are shared between groups. Finally, open the diagnosis simulator, pick one of its weeks where on-time fell from 91% to 79% in one city, and write the one-page diagnosis, deciding between rainy evenings, a new app version and a data bug, ending with a recommendation.
You will hand in
- A metric tree for weekly on-time orders with today's numbers, owners, the +1% sensitivity of each input, causal links marked as hypotheses, and three to five guardrails with limits
- A tracking plan from checkout to doorstep: 8 to 15 events with exact triggers, properties, app or server source, a source of truth for each number, a settling window and daily data checks
- A one-page retention readout: definition, the typical curve and its floor, the cohort that differs and why, and one decision
- An experiment plan for substitutes before packing: unit and why, primary metric, guardrails, minimum detectable effect, sample size and duration, and the decision rule
- A one-page diagnosis of the 12-point on-time drop in the diagnosis order, ending with a decision, an owner, a date and a follow-up
Check what you know
Already know some of this? Answer 6 quick questions, one from each lesson, and we will mark the lessons you can skim. It takes about 3 minutes and nothing is graded.
Questions
Do I need to know SQL?
No, but it helps. Each lesson has an auto-checked SQL exercise (a funnel, a cohort table, a tracking check, an A/B summary, a data-bug hunt) that runs in your browser, and the starter code shows the shape. If you want a refresher first, take SQL you'll actually use in the SQL & Data Modeling course.
Is there much statistics?
Only what a PM needs, explained in plain words: sample size and the minimum detectable effect, confidence intervals, p-values, why peeking breaks a test, and how much a rate wobbles by chance. You use a rule of thumb and a simulator, not formulas to memorise.
I build internal tools with a few hundred users. Do experiments even apply?
Yes, in a different form. The experiments lesson shows store-level pilots with matched control stores and a difference in differences, and why the sample size is the number of stores, not transactions. The adoption and retention lessons measure internal tools as work done in the tool, by rollout wave.
How does this course connect to OKRs and launches?
It builds on them. OKRs and outcome goals (Strategy, Priorities & Roadmaps) introduced leading, lagging and guardrail metrics; this course turns them into a full metric tree. Launching (From Spec to Launch) asked for an adoption funnel at the post-launch review; here you learn to build and read one.
Will this help with PM interviews?
Yes. Choosing success metrics, diagnosing a metric drop and designing an A/B test are among the most common execution questions, and five of the six lessons include an 'In interviews' note on what a strong answer covers.
Is it really free?
Yes. Every lesson, podcast episode, exercise, quiz, final exam and certificate is free. The labs and their templates for real work are free too. Plus and Premium add every lab challenge, weekly extras and a mentor on top, and they never lock the tools or the courses.
How long does it take?
Each lesson shows an honest time for reading and doing. Most people finish a course in two to four weeks at three or four hours a week.