Most designs are checked once, at today's traffic, and declared fine. Then a campaign goes out, traffic doubles, and the team finds the limit the hard way: in production, with customers watching.
A load test turns that around. You push steadily more traffic through the design until something gives, and you write down three things: the rate it held, what broke first, and what it cost. This post walks through doing exactly that in the System Design Studio with its Find my limit button. It is free with an account and takes about 15 minutes.
What "Find my limit" does
The Studio's simulator runs your design with steady traffic: the request mix from your brief, no spikes and no failures. It starts at 10 requests a second and raises the rate by 30% at each step. It stops at the first step where any of these happens:
- the p95 latency goes over your target (95% of requests must be faster than this),
- the error rate goes over your target,
- or queued work piles up faster than it is done.
Then it narrows down between the last rate that held and the first that broke, to within about 1%. The simulator is deterministic: the same design always gets the same answer, so you can change one thing and compare fairly.
You get back:
- Holds up to: the highest steady rate that met every target.
- About N active users: the rate times 10, assuming each active user sends a request every 10 seconds. It is a rough rule, but it turns req/s into something a product manager can picture.
- Costs per month, and per 1,000 users.
- What broke, and the bottleneck: the busiest part at the breaking point, whether it ran out of CPU, memory or disk, and what to do about it.
Try it: the 3-tier example
Open the Studio, choose Projects, then the 3-tier web app example. It is Mangrove Market's store pages for the website and the app: a load balancer, three app servers and a database. The brief says normal traffic is 500 requests a second, with a Thursday offers spike to 1,200. The targets are a p95 under 250 ms and errors under 1%, on a budget of $700 a month.
Press Find my limit. Here is what the free plan shows for the example as it ships:
- Holds up to 1,304 req/s, about 13,040 active users
- Costs $605 a month, $46.40 per 1,000 users
- What broke: at 1,316 req/s the p95 reached 256 ms
- Bottleneck: the app servers, limited by CPU
So the design passes on Thursday, with about 9% to spare above the 1,200 spike. That is not much. One good campaign and it is over the line.
Fix the bottleneck, and watch it move
The obvious fix is the one the bottleneck note suggests: more app servers. Select the app server, raise its replicas from 3 to 6, and test again.
- Holds up to 1,420 req/s. Only 9% more.
- Costs $785, and the cost per 1,000 users went up, to $55.28.
- Bottleneck: the database primary, limited by CPU, at 95% busy.
This is the most useful thing a load test teaches. You doubled the app servers and gained almost nothing, because the moment they stopped being the bottleneck, the database became it. Every design has exactly one first bottleneck at a time. Fixing anything else is spending money for no gain.
So take load off the database instead. Go back to 3 app servers, add a cache next to the database, and draw an arrow from the app server to it, so reads try the cache first and still fall through to the database. Then try 5 app servers:
- Holds up to 2,264 req/s, about 22,640 active users
- Costs $770, $34.01 per 1,000 users
- Bottleneck: the app servers again, so the next step is more of them
Nearly double the capacity of the original, and each user is cheaper to serve. The cache earned its $45 a month many times over, because it attacked the part that was actually full.
The budget says no
There is a catch: $770 is over the $700 budget. Look for the money before you look for more capacity.
Click the database. The example runs it with 2 replicas, and the Inspector explains that extra replicas of the primary are standbys: they take over in a few seconds if the primary dies, but they add no capacity. That standby costs $200 a month and doesn't change the limit at all.
So should you remove it? The Studio can't decide that for you, and that is the point. The standby is what keeps the store pages up when the database machine fails. Removing it brings the cost to $570 and keeps the same 2,264 req/s, but the next database failure becomes an outage. Either the budget grows, or the business accepts that risk in writing. A load test turns a vague argument ("we need more servers") into a clear choice with numbers on both sides.
What to take into real projects
- Test until it breaks, not until it passes. "It handled today's traffic" tells you nothing about next month. The limit and the headroom above your peak do.
- Fix one bottleneck, then test again. It will move. Expect it.
- Track cost per 1,000 users, not just total cost. A change that raises capacity and lowers this number is a good change.
- Know what each part is for. Some parts are there for capacity, some for survival. Don't cut the second kind to pay for the first without saying so.
Free and Plus
The free plan load-tests up to 10,000 req/s, which is plenty for every example and for most designs you will build while learning. It gives you up to 12 parts, one failure check and 5 saved projects. Plus raises the load and part limits, and adds zones, chaos mode, version history and the scenario library. The full comparison is in the practice labs announcement.
Want the theory first? Scaling up vs scaling out explains why some parts scale by adding copies and some don't, and Speed numbers and back-of-the-envelope math helps you guess the limit before you measure it. Then come back and check your guess.