Performance and load testing

35 minIntermediate

Performance and load testing

On festival day Mangrove's checkout gets about four times its normal orders, and every slow second costs carts. This lesson shows you how to turn that rush into a load test with a clear pass or fail: a goal written as an SLO, a workload model built from real numbers, a k6 script that sends load the way customers do, and the habit of reading percentiles, errors and saturation together until you find the bottleneck.

By the end you can

Pick the right test type (load, stress, spike, soak, breakpoint) for a question, and write its goal as an SLO
Turn daily order numbers into requests per second, with every assumption written down
Explain why closed-model tests hide slowness, and use k6's arrival-rate executors instead
Write a k6 script with scenarios, thresholds and checks, and read its end-of-test summary
Use Little's law and server metrics to find a bottleneck, fix it and prove the fix with the same test

The everyday picture

Ama wants to know whether her store's four tills can cope on festival eve. She tries two rehearsals. In the first, she asks ten friends to shop. Each friend pays, walks one lap of the store, and joins the queue again. When one till jams for five minutes, her friends are stuck in the queue, so nobody new arrives. After the jam, the queue looks short and the average wait looks fine.

In the second rehearsal, she hands out numbered tickets at the door: one new shopper every 20 seconds, jam or no jam. Now the jam shows up the way it will on the real night. Shoppers keep arriving, the queue grows out of the door, and the people who arrived during the jam wait fifteen minutes. Real customers behave like the second rehearsal. They do not wait politely for the till to recover before they walk in. A load test has to behave like them too, or it will tell you "fine" on exactly the day it matters.

Five questions, five test types, one written goal

"Is checkout fast enough?" is too vague to test. Each kind of performance test answers one sharper question, and you pick it by the question:

Load test: does checkout meet its goal at the traffic we expect? For Mangrove, that is the festival peak. This is the test you run before every big event.
Stress test: what happens above the expected peak, say at 1.5 or 2 times festival traffic? You want to see it slow down and shed load gracefully, with clear errors, rather than fall over and take payment with it.
Spike test: what happens when traffic jumps in seconds? A push notification to 2 million customers at 6 pm can do that. Spikes find autoscaling that reacts too late and connection pools that start too small.
Soak test: does it stay healthy for hours? Run 2 times normal traffic for 8 hours and watch memory, open connections and disk. Leaks only show up with time.
Breakpoint test: where is the limit? Ramp the load slowly until the goal fails, and note the rate. That number tells you how much headroom you have.

Before any of these, run a smoke test: the same script at one order a second for a minute, to prove the script, the test data and the environment work.

Then write the goal so the test can fail. A goal needs three parts: a load level, a latency limit at a percentile, and an error limit. You met SLOs in Measuring speed and reliability: p99, SLOs and nines. For Mangrove's festival test, the team agreed on this:

Output

At festival peak traffic (12 orders a second), held for 30 minutes:
  POST /orders      p95 under 800 ms, failed requests under 1%
  GET /orders/:id   p95 under 300 ms
  Checks (correct status and body) pass for over 99% of requests

A load level without a limit cannot fail, so nobody learns anything. A limit without a load level is meaningless, because every system is fast with one user. Write both, and the goal becomes a line in the test script that turns the run red or green.

Quick check

Marketing plans a push notification to 2 million customers at exactly 6 pm on festival eve. Which test answers "will checkout survive the first two minutes?"