What system design really is
Welcome to Kolom. This is the first lesson of the System Design track, and you need no experience of system design to start it. System design is planning how the parts of a software system fit together so it keeps working at the numbers you expect: the users, the busy hours, the failures. This lesson gives you the words and the thinking loop that every later lesson uses.
By the end you can
- Split any feature request into functional and non-functional requirements
- Name the nine qualities that every design trades against each other
- Use one repeatable thinking loop to run a design discussion
- Draw the starting three-box system and say what each box does
- Write a one-page requirements sheet with real numbers in it
The everyday picture
Imagine you open a tea stall next to a bus stop. You buy a kettle, a gas burner, twenty cups and a bench. If more people come than you expected, they wait a few minutes or walk to the next stall. You never write a plan, and that is fine.
Now imagine you are opening a large supermarket in a busy neighbourhood. Before opening day, someone has to answer a long list of questions. How many customers come on a normal evening, and how many on the night before a big festival? How many checkout counters does that need? How much storeroom space for the stock that arrives from the warehouse? When the power cuts out, how long until the generator takes over? If the internet goes down, can the tills still sell?
None of these questions are about what the supermarket sells. They are about how well it works under real conditions: crowds, failures, money and time. Answering them on paper costs a few meetings. Answering them on festival night, with a queue out of the door, costs a lot more. Software works the same way. A to-do app for yourself is the tea stall. An app that hundreds of stores open every morning is the supermarket. System design is the planning you do before opening day.
Two kinds of requirements
Every request a team receives has two parts. The first part says what the system must do. This is a functional requirement (FR). The second part says how well it must do it. This is a non-functional requirement (NFR).
Take one feature: store staff scan a barcode and see how many units are in stock. The FR is "staff can see the stock of one product at their store". The NFRs are things like "the answer appears in under 300 ms for 95 out of 100 scans", "it works during the 7 pm rush on the night before a festival", and "the number is never more than 2 minutes behind the last sale".
Business teams usually describe FRs and leave the NFRs unsaid. Nobody writes "must be available 99.9% of the time", but everyone is angry when the app is down during the evening rush. A big part of the job, whether you are the engineer or the product manager in the room, is to bring the NFRs into the open and put a number on each one, because the numbers decide the design. A stock lookup used by 5 stores and one used by 500 stores at the same moment need very different systems, even though the FR is the same sentence.
Good NFR questions are short and always ask for a number:
- How many users or devices, and what does the busiest hour look like?
- How fast must it feel, and for what share of requests?
- What happens to the business if it is down for 10 minutes? For a whole day?
- How fresh must the data be: seconds, minutes, or yesterday?
- How much data, and how long do we keep it?
Quick check
The business asks for a "real-time sales dashboard". What is the most useful first question?
"Real-time" is an adjective, and the design depends on the number behind it: 5 seconds and 1 hour lead to very different systems. The library and the screen type are fair questions, but they come after you know how fresh the numbers must be.
The qualities you trade off
NFRs fall into a small set of qualities. Learn these nine words well, because engineers use them in every design review. Most of them get a full lesson later in the System Design track.
| Quality | Plain meaning | Mangrove example |
|---|---|---|
| Scalability | Keeps working as load grows | Works the same with 50 stores or 600 |
| Availability | Answers when you call it | The POS can reach the server at 9 pm on festival eve |
| Reliability | Gives correct results, again and again | The stock number shown is the real stock number |
| Latency | How long one request takes | The price appears 0.2 s after the scan |
| Consistency | Everyone sees the same data | Two tills agree there are 3 units left |
| Durability | Saved data is never lost | A sale saved at 8:01 pm survives a server crash at 8:02 pm |
| Maintainability | Easy for your team to change and fix | A new developer can add a report in a day |
| Cost | Money for servers, tools and people's time | The monthly cloud bill and the hours spent on support |
| Security | Only the right people see and change things | A partner shop sees its own orders, never another shop's |
Notice that availability and reliability are different. A server can be up and answering (available) while it shows stock numbers from last week (unreliable). Users feel both, and both need their own number.
These qualities pull against each other. Each improvement in one usually costs something in another. This is called a trade-off. Three examples you will meet again and again:
- Speed vs freshness. Keep a copy of stock numbers in memory and refresh it every minute. Lookups get much faster and the database works less. In exchange, the number can be up to a minute old (Caching).
- Safety vs speed. Wait until two separate servers have saved each sale before telling the cashier "done". A crash can no longer lose the sale, but every sale now waits for the slower of the two (Replication and read replicas).
- Availability vs cost and effort. Run a full second copy of the system in another data centre. A fire in one building no longer stops the stores, but you pay for two systems and your team has more to run.
This is why there is no "best" design. There is only the best design for a given set of numbers. When someone asks "what is the best database?", the honest answer starts with "for how many writes per second, how much data, and what happens if it is down?"
Quick check
The Store app answers every scan in 100 ms, but it shows prices from last week. Which quality is failing?
The app answers, and fast, so availability and latency are fine. The answers are wrong, so reliability is failing. Scalability is about handling more load, which is not the problem here.
Think first: the business asks for "real-time stock in the Partner app". What do you ask before anyone draws a box?
Turn "real-time" into numbers. How old can the stock number be before it causes a wrong order: 5 seconds, 2 minutes, 1 hour? How many partner shops look at stock at the same time during the busiest hour? Is it enough to show "In stock" or "Low stock" instead of an exact count? What happens if the shop orders 10 cartons and only 8 are left when the order is processed?
The answers change the design a lot. "Up to 10 minutes old, shown as In stock or Low" can be a simple copy refreshed every few minutes. "Exact at the moment of ordering" means the order step itself must check and reserve stock in one safe operation (Transactions and Ordering without overselling stock).
The thinking loop
Experienced engineers follow roughly the same steps for every design, from a small feature to a whole platform. The lessons in this track use the same loop, and the case studies near the end, starting with a URL shortener, apply it from start to finish.
What happens at each step
- Requirements. FRs as user actions, NFRs as numbers, and a list of what is out of scope.
- Rough numbers. Users, requests per second, data per day and per year. You only need to know if the answer is 5 requests per second or 5,000 (Speed numbers and estimation).
- API and data. The main endpoints and tables, so everyone agrees on what the system stores and returns (APIs done right, Modeling data in tables).
- Big picture. Boxes and arrows, with every arrow labelled by what travels along it.
- Deep dive. Design the hardest part in detail. For stock, that is usually "two people buy the last unit at once".
- Bottlenecks and failures. Ask what breaks first when load doubles, and what users see when each box fails. A bottleneck is the part that limits everything else. A single point of failure is a part with no backup.
- Trade-offs. Say what the design gives up and why that is acceptable for these numbers. Write it down.
- Iterate. New information changes earlier answers, so you go around again.
Running a design review turns this loop into a timed agenda for a real meeting.
Quick check
In which step do you find out whether the system must handle 5 or 5,000 requests per second?
Rough numbers turn users and busy hours into requests per second and gigabytes, before anyone draws boxes. The big picture and the deep dive depend on that answer, so they come after it.
The starting box
Almost every system you will design starts as three boxes. A client (the app or browser) sends a request over the network. An API server receives it, checks who is asking, applies the business rules and asks the database for data. The database stores everything permanently and finds rows quickly.
If you have built an app, you have built this. Mangrove Market's version is a React Native app, a NestJS API in TypeScript and a PostgreSQL database, and most code in this track uses that same stack. Hold on to this picture. Every later lesson either explains one of these arrows in more depth, or adds a new box to solve a specific problem that appears when the numbers grow.
Two facts about this picture matter for everything that follows. First, the API server can be made stateless: it remembers nothing between requests, so you can run three or ten identical copies of it (Scaling up vs scaling out). Second, the database holds the state. You cannot copy it as easily, because every copy must agree on the data. That is why so many hard problems end up at the database, and why the next courses in this track, starting with Modeling data in tables, are about databases.
Quick check
Why is it easier to run ten copies of the API server than ten copies of the database?
A stateless server has nothing to keep in sync, so any copy can answer any request. Databases can run on several machines (you will see replicas later), but keeping the copies in agreement is the hard part. Price is not the reason.
Build it
We will write a one-page requirements sheet, then fill it in for a "Store stock lookup API" for Mangrove's Store app.
-
Save the template
Keep this file in your team's docs and copy it for every new feature. Each section forces one kind of decision. The comments tell the writer what a good answer looks like.
Markdown# Requirements: <feature name> Owner: <PM name> Date: <yyyy-mm-dd> Status: draft ## 1. Problem <!-- Who has the problem, and what does it cost them today? --> ## 2. Functional requirements (what it does) <!-- One user action per line: "<role> can <action>" --> - FR1: - FR2: ## 3. Non-functional requirements (how well) <!-- Every line needs a number, a unit and where it is measured --> | Quality | Target | Measured where / how | |--------------|--------|----------------------| | Latency | | | | Availability | | | | Freshness | | | | Load (peak) | | | | Data size | | | | Security | | | ## 4. Rough numbers <!-- Show the arithmetic so others can check it --> ## 5. Constraints <!-- Team, deadline, existing systems, devices, network --> ## 6. Out of scope (this version) ## 7. Open questions -
Write the problem and the FRs
Write FRs as actions a named role can take. If you cannot name the role, you do not understand the requirement yet. The numbers below are for Mangrove Market's roughly 320 stores.
Markdown# Requirements: Store stock lookup API Owner: Product (Store app) Date: 2026-10-06 Status: draft ## 1. Problem Floor staff walk to the storeroom to check if a product is in stock. This takes 3-5 minutes per customer question during the evening rush. ## 2. Functional requirements - FR1: Store staff can scan a barcode and see the stock of that product at their own store. - FR2: Store staff can see stock of the same product at the 5 nearest stores, to send a customer there. - FR3: Area managers can look up any product at any store in their area. -
Put a number on every NFR
Each target says what is measured, the limit, and where you measure it. "Fast" is not a requirement. "p95 under 300 ms, measured at the API" is. The term p95 is explained fully in Measuring speed and reliability, and QPS in Speed numbers and estimation.
Markdown## 3. Non-functional requirements | Quality | Target | Measured where / how | |--------------|------------------------------|-------------------------------| | Latency | p95 < 300 ms, p99 < 800 ms | API server, per request | | Availability | 99.9% per month, 8am-11pm | Health check every 30 s | | Freshness | a sale shows within 2 min | Test sale, then lookup | | Load (peak) | 40/s at festival peak 7-10pm | Request log, busiest hour | | Response | < 5 KB per lookup | Slow store internet | | Data size | ~6.4M stock rows today | count(*) on stock | | Security | staff see own store + FR2 | Role check on every request | ## 4. Rough numbers - 320 stores x ~2,000 lookups/day = 640,000 lookups/day - 640,000 / 86,400 s = ~7.4 lookups/s on average - Festival evening peak ~5x average = ~37, so plan for 40 lookups/s - 320 stores x ~20,000 products each = ~6.4M stock rows -
Write constraints, out of scope and open questions
The out-of-scope list protects the deadline. The open questions list is where you write things you do not know yet, instead of guessing.
Markdown## 5. Constraints - Must use the existing Postgres database and NestJS backend. - Two backend developers, first version in 4 weeks. - Handheld scanners run the existing Store app (React Native, Android). - Store internet is often slow; some stores drop to mobile data. ## 6. Out of scope (this version) - Changing stock (adjustments, transfers): separate project. - Working with no internet at all (see offline-first, later). - Stock forecasts and reorder suggestions. ## 7. Open questions - Is 2 minutes of freshness enough for FR2, or do customers arrive at the other store and find nothing? - Who decides which stores an area manager can see? -
Check your numbers against real data
Guesses are fine to start, but check the ones you can. Run this on a reporting copy of the database, never on the busy primary during opening hours. It assumes a
stocktable with one row per store and product, as built in Modeling data in tables. In Mangrove's database asiteis a store or a DC, and anarticleis a product.SQL-- How many stores, products and stock rows do we really have? SELECT count(DISTINCT site_id) AS stores, count(DISTINCT article_id) AS articles, count(*) AS stock_rows FROM stock;Outputstores | articles | stock_rows --------+----------+------------ 320 | 24318 | 6391204The output above is an example from Mangrove's imagined data. If your real number is 10 times your guess, update the sheet before anyone designs anything. That is step 8 of the loop, already at work. You can try the rough-numbers step in your browser: the Practice section below turns it into an auto-checked JavaScript exercise.
Bring it to your team
Use these at the start of any design discussion, before the first box is drawn.
- "Which user action is this design for, and who is the user?"
- "How many users or devices at the busiest hour? A rough number is fine."
- "How fast must it be, for what percentage of requests, measured where?"
- "What does a store do if this is down for 10 minutes on the busiest evening of the year?"
- "How old can the data be before it causes a wrong decision?"
- "What are we deciding not to build in this version?"
Red flags
- Requirements full of adjectives and no numbers: "fast", "scalable", "real-time".
- The first slide is a technology choice, before any requirement or number.
- Nobody can say what users see when the database or an outside service is down.
- There is no out-of-scope list, so every new idea joins version 1.
Practice
Put numbers on a vague request
Imran's team receives this request: "The Rider app should show riders new orders in real time." Using the template from Build it, write two FRs, four NFRs (each with a number, a unit and where it is measured), one out-of-scope item and one open question. Facts you can use:
- About 600 riders, and about 60,000 app orders a day, with 4 times that at festival peaks.
- Most orders arrive between 6 pm and 10 pm.
- Riders often pass through areas with patchy mobile data.
- If no rider accepts an order within 2 minutes, the 30-minute delivery promise is at risk.
- Each FR names a role and one action ("A rider can…").
- Every NFR has a number, a unit and a place where it is measured.
- At least one NFR is sized for the festival peak, not only a normal day.
- At least one NFR says what happens on weak mobile data.
- Your open question is one the facts above cannot answer.
Show a model answer
FR1: A rider can see new orders offered to them, with the store and the drop-off area. FR2: A rider can accept or decline an offered order.
Freshness: an offer reaches an online rider's phone within 10 s for 95% of offers, measured from order creation to the phone's receipt. Peak load: 240,000 orders a day at festival peak; if half arrive in the 4 evening hours, that is about 30,000 an hour, roughly 8 to 10 new offers a second, measured in the request log. Availability: offers and accepts work 99.9% of the time between 7 am and 11 pm, measured by a health check every 30 s. Weak network: an accept tap is retried safely for up to 60 s without creating a double accept, tested on a throttled 3G connection.
Out of scope: turn-by-turn navigation. Open question: if a rider ignores an offer for 30 seconds, who gets it next, and who decides?
Do the rough numbers in code
Write roughNumbers(stores, lookupsPerStorePerDay, peakFactor). It returns an object with perDay (all lookups in a day), avgPerSecond (spread over the 86,400 seconds of a day) and peakPerSecond (the average times the peak factor). Do not round.
function roughNumbers(stores, lookupsPerStorePerDay, peakFactor) {
const perDay = 0; // stores × lookups per store per day
const avgPerSecond = 0; // spread evenly over one day
const peakPerSecond = 0; // the busiest hour
return { perDay, avgPerSecond, peakPerSecond };
}test('320 stores with 2,000 lookups each make 640,000 a day', () => {
expect(roughNumbers(320, 2000, 5).perDay).toBe(640000);
});
test('the average is about 7.41 lookups a second', () => {
expect(roughNumbers(320, 2000, 5).avgPerSecond).toBeCloseTo(7.41, 2);
});
test('a peak factor of 5 gives about 37.04 a second', () => {
expect(roughNumbers(320, 2000, 5).peakPerSecond).toBeCloseTo(37.04, 2);
});
test('a peak factor of 1 means the peak equals the average', () => {
const r = roughNumbers(100, 864, 1);
expect(r.peakPerSecond).toBe(r.avgPerSecond);
expect(r.avgPerSecond).toBe(1);
});
Show a solution
function roughNumbers(stores, lookupsPerStorePerDay, peakFactor) {
const perDay = stores * lookupsPerStorePerDay;
const avgPerSecond = perDay / 86400;
const peakPerSecond = avgPerSecond * peakFactor;
return { perDay, avgPerSecond, peakPerSecond };
}Check yourself
Which of these is a non-functional requirement?
It describes how well the system works, with a number and a percentage. The other three describe what the system does for a user, so they are functional requirements. The spreadsheet export can feel "technical", but it is still a user action.
In the first five minutes of a review, an engineer says "We should use microservices and Kafka for this." Where should the discussion be at that point?
Tools are answers to numbers. Without knowing messages per second, data size and failure tolerance, nobody can judge if Kafka helps or adds cost. Discussing Kafka's trade-offs is useful later, once the numbers show a queue is needed.
The business confirms that partner shops can see stock numbers that are up to 10 minutes old. What does this mainly allow the team to do?
Freshness and availability are separate qualities. A 10-minute freshness budget lets you trade a little freshness for speed and lower database load. The order step still needs a real check, because the copy can be wrong by the time the order arrives.
Flashcards
Key takeaways
- Functional requirements say what the system does. Non-functional requirements say how well, and they must have numbers.
- Scalability, availability, reliability, latency, consistency, durability, maintainability, cost and security pull against each other. Every design is a set of trade-offs.
- There is no best design, only the best design for a given set of numbers.
- Use the loop: requirements, rough numbers, API and data, big picture, deep dive, bottlenecks, trade-offs, iterate.
- Every system starts as client, API server and database. Later lessons add boxes only when the numbers ask for them.
Sources
- AWS Well-Architected Framework, Amazon Web Services: a long checklist of the same qualities, grouped into pillars
- Embracing Risk, in Site Reliability Engineering, Google, 2016: why 100% is the wrong target