1. Courses
  2. System Design
  3. How the Web Works
  4. Stop 6

35 minBeginner

Measuring speed and reliability: p99, SLOs and nines

"Is the app fast?" and "Is it reliable?" only have useful answers when you measure them the right way. This lesson shows why averages hide the pain, how percentiles show it, and how SLOs and error budgets turn reliability into a number the business and engineers can agree on.

By the end you can

  • Explain why the average latency hides slow requests, and read p50, p95 and p99
  • Explain why one slow call makes a whole screen slow
  • Define an SLI, an SLO, an SLA and an error budget for a real endpoint
  • Convert "nines" into minutes of downtime, and combine the availability of several parts
  • Measure request durations in NestJS and compute percentiles in Postgres

The everyday picture

You take the same bus to work every day. On most days it comes within 5 to 10 minutes. The transport office proudly reports that the average wait is 9 minutes. But about once a week it does not come for 45 minutes, and you are late for the morning meeting.

Which number do you remember? The 45. And which number decides when you leave home? Also the 45, because you cannot afford to be late. The average is true, but it describes nobody's actual experience. It mixes many short waits with a few terrible ones into a number that sounds fine.

Software has the same problem. A dashboard that shows "average response time: 120 ms" can hide the fact that one request in fifty takes 3 seconds. Those are the requests that make a cashier tap twice, a customer give up, and a manager call your team.

Why averages lie, and what percentiles show

Request times are not spread evenly around the average. Most requests are fast, and a few are very slow: the database had to read from disk, a lock was held, a garbage collection pause hit, a packet was lost. The slow ones form a long on the right of the chart. The average gets pulled up by the tail but still hides how bad the tail is.

A answers a clearer question: "how long did the slowest X% of requests take?" Sort all requests from fastest to slowest:

  • p50, the median: half the requests were faster than this. It tells you the typical experience.
  • p95: 95 out of 100 requests were faster. One in twenty was slower.
  • p99: 99 out of 100 were faster. One in a hundred was slower. This is the tail.
How long 140,000 lookups took p50 80 avg 102 p95 254 p99 410 the tail 0 125 250 375 response time (ms) slowest few (450+ ms) not shown
Real shape, fake data: 140,000 simulated stock lookups. Half finish within 80 ms, and the average is 102 ms. But 1 in 20 takes over 254 ms and 1 in 100 over 410 ms. The average line sits in the crowd and says nothing about the tail.

Why care about 1 in 100? Because it is not rare for the people using the system. A cashier who scans 300 items in a shift, at p99 = 2 seconds, waits 2 seconds about 3 times per shift. A system with 320 stores making 1 million requests a day has 10,000 requests a day slower than p99. Someone is always experiencing your tail.

Two rules for reading percentiles. First, you cannot average them. The p95 of the whole day is not the average of 24 hourly p95 values, so compute percentiles from raw data or from a histogram. Second, always say which requests and which window: "p95 of GET /stock over the last 30 days, measured at the API".

Quick check

You have the p95 for each of the 24 hours of yesterday. How do you get yesterday's p95?

Loading the rest of the lesson…