1. Courses
  2. Product Management
  3. Metrics & Product Analytics
  4. Stop 4

35 minIntermediate

Instrumentation: tracking plans and data you can trust

Every metric in this course is only as good as the events behind it. This lesson shows you how to write a tracking plan that starts from the questions you need answered, name events so they stay consistent, choose the source of truth for each number, handle handhelds that go offline and events that arrive late or twice, and catch data bugs before they reach a dashboard.

By the end you can

  • Write a tracking plan that starts from the metric tree and the funnel
  • Name events and properties consistently, and choose app or server for each one
  • Decide which system is the source of truth for each number, and reconcile the others with it
  • Handle offline devices and late events with two timestamps, deduplication and a settling window
  • Write SQL checks that catch duplicate, incomplete and late events

The everyday picture

Send a parcel with a courier and you can follow it: picked up, at the hub, out for delivery, delivered. Each line is an event: something that happened, at a time and a place, recorded by a scan.

The tracking page also shows what goes wrong. When a courier's scanner loses signal in a basement, the scans are stored and uploaded later, so "at the hub, 10:02" may only appear at 11:30. When a courier forgets to scan, the parcel looks lost while it sits on the right shelf. When a scan is sent twice, the parcel seems to arrive twice. And the courier company doesn't decide whether you paid by reading its scan log; it checks its billing system.

Product analytics is parcel tracking for what people do in your product. A good setup decides in advance which scans exist and what each records, which system is the final word on money and counts, and what to do when scans arrive late, twice or not at all.

Start from questions: the tracking plan

An is a record that something happened: a name, a time, and such as the store, the app version and the number of lines. A is the agreed list of every event you record. For each one it says:

  • Name, in the agreed convention.
  • Trigger: exactly when it fires. "When the server accepts the submitted receipt" is a trigger. "When receiving happens" is not.
  • Properties, with types and allowed values.
  • Source: sent from the app on the device, or from the server.
  • Metric served: the question or the box in the metric tree it feeds.
  • Owner: who changes it, and who is told when it changes.

Write the plan from your questions, not from your screens. Go through the metric tree (pm-21) and the funnels (pm-22), and ask for each box: which events do we need to count this? "Track everything" sounds safe and produces thousands of events, half of them half-defined, that nobody trusts. A plan of 20 well-defined events beats one of 400.

The plan is part of the spec. Add the events to each user story and to the Definition of Done (pm-16), and when an outside agency builds the app, make the plan part of what they deliver against. An event the vendor forgot is much cheaper to add before launch than after three months of missing data.

Quick check

Which trigger definition for the event "Receiving Submitted" is ready for the tracking plan?

Loading the rest of the lesson…