Speed numbers and back-of-the-envelope math
Five minutes of rough math often tells you whether a design needs one server or fifty. This lesson gives you the few numbers to remember and a method for turning "how many users?" into requests per second and gigabytes per year.
By the end you can
- Place memory, SSD, network and disk on a ladder of speeds, by order of magnitude
- Explain the difference between latency and throughput
- Turn daily volumes into average and peak requests per second
- Estimate storage per year, including indexes
- Check your estimate against real data with SQL
The everyday picture
Ama manages a Mangrove store, and she is planning for the night before a big festival. Last year, the busiest hour that night had about 300 customers. One checkout takes about 2 minutes. So the counters need 300 × 2 = 600 counter-minutes in a 60-minute hour, which means 10 counters busy every single minute.
She does not plan for exactly 10. When every counter is busy all the time, any small delay, like a customer hunting for their mobile-wallet PIN, makes the queue grow and never shrink. So she opens 13 or 14 and keeps a spare cashier ready. On a normal evening with 120 customers an hour, 4 or 5 counters are enough.
She never measured to the second. She needed to know whether the answer was 4 counters or 40. That is back-of-the-envelope math: rough numbers, multiplied carefully, to get the size of the answer right.
The speed ladder
Different operations take wildly different amounts of time. Reading from memory is about a thousand times faster than reading from an SSD, and a trip to another continent is about a thousand times slower again. You do not need the exact values. Remember the order of the rungs and roughly how far apart they are.
The units, from small to large: a nanosecond (ns) is a billionth of a second, a microsecond (µs) is 1,000 ns, a millisecond (ms) is 1,000 µs, and a second is 1,000 ms.
Three lessons fall out of the ladder:
- Memory beats disk, by a lot. That is why databases keep hot data in memory, and why a cache helps so much (Caching). A query whose data is in memory can be a hundred times faster than the same query reading from disk.
- One network round trip costs more than lots of local work. A server can do a lot of computing in the 0.5 ms of one call to the database. Ten queries in a loop cost ten round trips, which is why the N+1 problem hurts (Reading query plans).
- For a phone, the network is everything. With 300 ms per round trip, five calls in a row cost 1.5 seconds before the server does anything (the request journey). A useful target: people feel a response under about 100 ms as instant.
Quick check
Roughly how many SSD random reads fit into the time of one round trip to another continent?
About 150 ms divided by about 100 µs is about 1,500, so "about a thousand" is the right size. This is why one extra trip across the world costs more than a lot of disk work on the server.