What the giants share, and what not to copy
Nine companies serving between hundreds of millions and billions of people use a surprisingly short list of ideas, and almost every idea is one you met in the earlier System Design courses. In 2026, OpenAI described the PostgreSQL behind ChatGPT running on a single primary with read replicas. This lesson pulls the patterns together, shows at what size each one starts to pay off, teaches you to read an engineering blog critically, and ends with a review template you can use for a real system, filled in for Mangrove Market.
By the end you can
- Name the eight patterns that repeat across the giants, and the lesson that teaches each
- Place a system on the scale ladder from its own numbers, and say what changes at 10× and 100×
- Read a big-tech engineering post and separate its context from its advice
- Run a "should we copy this?" check on any idea from a giant
- Fill in a one-page architecture review for a Mangrove Market system using three giants' ideas
The everyday picture
Walk through a big shopping mall on a Saturday afternoon, then into a corner grocery down the street. The mall has escalators, a central cold room, a loading bay with ten docks, security at every door and a control room with fifty cameras. The corner shop has a fridge, a shelf, one cash box and a shopkeeper who knows every customer.
Look closer and the corner shop already does what the mall does, in a small way. Fast sellers are near the counter (a cache). Stock arrives at night, before customers come (a fill window). Money and goods are counted at closing (reconciliation). The shopkeeper keeps a little extra rice in case the truck is late (spare capacity). What the shop does not need are the ten docks and the control room. Copying those would bankrupt it. This lesson is about telling the difference: which ideas are universal, and which are only answers to the mall's size.
Eight patterns that keep coming back
Across the nine teardowns in this course, the same eight ideas appear again and again. None of them is exotic. Each has an earlier lesson on Kolom where you built it at small scale.
- Cache everything that is read far more than it is written. Facebook's TAO answers the social graph from cache in front of MySQL (Facebook). Netflix's EVCache holds 2 trillion items (Netflix). Uber described a Redis layer in front of its MySQL-based Docstore serving over 40 million reads a second in 2024. OpenAI added a lock so only one reader per missing key goes to PostgreSQL (ChatGPT). Prompt caching reuses the identical start of a prompt (Claude). The mechanism is always cache-aside with a TTL, invalidation, and stampede protection (Caching).
- Split data by its natural key. Google splits its index by document (Google Search); Instagram put thousands of logical shards on Postgres schemas, keyed by user (Instagram); Uber cut the map into H3 hexagons and gave each an owner (Uber); DynamoDB is built around the partition key (Amazon). The hard part is never the sharding code; it is choosing the key that keeps most requests on one shard (Sharding).
- Put a queue between fast and slow. WhatsApp stores a message only until it is delivered (WhatsApp); Uber runs Kafka everywhere (Uber); Netflix fills its boxes in a night window (Netflix); the Batch API takes work that can wait (Claude). Accept fast, process later (Queues and jobs).
- Make every retry safe. AWS's guidance: a retry with the same token must return the same answer (Amazon). Netflix's key-value layer puts an idempotency token on every write (Netflix). WhatsApp pairs at-least-once delivery with an idempotent receiver (WhatsApp). Google hedges only reads or idempotent calls (Google Search). Idempotency keys (APIs done right) plus backoff (Timeouts and retries).
- Limit the blast radius. Amazon's cells (Amazon), Netflix's active-active regions and region evacuation in 8 minutes (Netflix), Anthropic's separate server pools by context length (Claude), OpenAI's high and low priority PostgreSQL instances (ChatGPT). Bulkheads (Timeouts and retries), at the size of a data centre.
- Separate the control plane from the data plane. Netflix decides in AWS and streams from Open Connect (Netflix). Google's Borg decides where every job runs (Google Search); WhatsApp's servers pass along encrypted media pointers while the files sit on separate media servers (WhatsApp). OpenAI's December 2024 outage happened when the two were tied together (ChatGPT). Your API deciding and a presigned URL carrying the bytes is the same idea (Files and CDNs).
- See everything, and test failure on purpose. Netflix splits cache misses into causes and runs Chaos Monkey (Netflix); Anthropic now runs quality evals on production (Claude); Google rolls out in stages (Google Search). Logs, metrics, traces (Observability) and safe rollouts (Shipping safely).
- Keep a boring database at the core. MySQL under TAO, Docstore and Schemaless; PostgreSQL under Instagram's shards and under ChatGPT; Erlang and Mnesia at WhatsApp. Each grew around a well-understood engine before replacing anything (Choosing a data store).
Quick check
Netflix's API hands out URLs while Open Connect sends the video; WhatsApp's chat servers pass small pointers while media sits on separate servers. Which pattern is that?
In both, one part decides who gets what and a different part carries the heavy bytes. Caching is close, since Open Connect boxes are caches, but the defining move is splitting the deciding from the carrying.