Quality attributes and constraints
Every stakeholder wants the new system to be "fast, reliable and easy to change", and none of those words can be designed or tested. This lesson shows how to run a workshop that turns them into a short list of measurable scenarios and named constraints, so the architecture is chosen for reasons everyone signed.
By the end you can
- Separate business drivers, quality attributes and constraints, and test which is which
- Write a quality attribute scenario in six parts, with a response measure someone can check
- Turn vague words such as "fast", "reliable" and "scalable" into numbers with five questions
- Run a half-day quality attribute workshop: brainstorm, merge, vote and refine
- Build a utility tree and find the few scenarios that should drive the design
The everyday picture
A family books a caterer for a wedding and says: "We want the food to be good, and on time." Nobody can plan a kitchen from that sentence.
An experienced caterer asks questions. How many guests? About 400. How long after people sit down should the first plates arrive? Within 20 minutes. What if the power cuts out during the meal? It often does in that area in the rainy season. Each answer changes the plan. Serving 400 plates in 20 minutes needs two serving lines. A likely power cut needs a gas burner or a menu of dishes that are fine at room temperature.
Some facts are not up for discussion at all. The hall has no gas line, the date is fixed, and the food must follow the family's diet rules. The caterer does not trade those away; she designs around them. An architect does the same job with software: ask until "good and on time" becomes numbers, and write down the facts that cannot move.
Three inputs: drivers, quality attributes and constraints
In What system design really is you met functional and non-functional requirements and nine qualities that trade off against each other. At architect level you sort what stakeholders say into three buckets, because each bucket is handled differently.
- Business drivers say why the work is worth paying for. At Mangrove: release every week instead of every month, never let a report stop goods receiving, and survive four times the normal orders at a festival rush.
- Quality attributes say how well the system must behave, in numbers. They are where you make trade-offs.
- Constraints are decisions somebody already made. The design must fit them.
The test that separates the last two is one question: could we trade this away if the benefit were big enough? "Goods receiving saves in under 2 seconds" could become 3 seconds if that saved a year of work, so it is a quality attribute with a target. "The Partner app is built by Lagoon Digital under its current contract" cannot be changed by anyone in the room, so it is a constraint. Constraints come in three kinds: technical (PostgreSQL and NestJS are the company standard; the 400 store tills are existing hardware), organisational (team size, an outside agency, who owns which budget) and business or legal (a deadline before festival season, how long finance must keep invoices).
Give every constraint an owner, the person who could lift it. A constraint without an owner is often an old habit. Ask once, politely, "who decided this, and does it still hold?" Then accept the answer and design around it.
To make sure nobody forgets a whole area, use a checklist. The ISO/IEC 25010:2023 product quality model names nine characteristics: functional suitability, performance efficiency, compatibility, interaction capability, reliability, security, maintainability, flexibility and safety. (The 2011 edition had eight; usability and portability were renamed and safety was added.) Walk the list at the end of a workshop and ask, for each one, "did anyone say anything about this?" Together, the drivers, the constraints and the few scenarios that shape the structure are the architecturally significant requirements.
Quick check
Which of these is a constraint for the back-office move?
Nobody in the project can change the till hardware, so the design must fit it. The 5-minute target is a quality attribute: it could become 15 minutes if that saved a lot of work. Weekly releases are a business driver, the reason the work is funded.
The quality attribute scenario, in six parts
"The system must be fast" cannot be tested. The Software Engineering Institute (SEI) at Carnegie Mellon, in its architecture methods and the book Software Architecture in Practice (Bass, Clements and Kazman), writes each quality requirement as a quality attribute scenario with six parts:
- Source: who or what causes the event (a store worker, a developer, an attacker, a failing disk).
- Stimulus: the event that arrives (saves a goods received note, changes a price rule, the link drops).
- Environment: the conditions at that moment (normal day, festival peak, during a deploy, one database replica down, month-end report running).
- Artifact: the part of the system that is hit (the receiving API and its database, the pricing module, the POS).
- Response: what the system does (saves and updates stock, deploys without touching other modules, keeps selling and syncs later).
- Response measure: the number that says pass or fail.
Scenarios are not only about speed. A maintainability scenario: a developer on the buying team (source) adds a new promotion type (stimulus) during normal work (environment) to the pricing module (artifact); the change is built, tested and released without editing other modules (response) in 3 developer-days or fewer, with one team's review (measure). An availability scenario: the store's internet link (source) drops for 2 hours (stimulus) on a Saturday evening (environment); the POS (artifact) keeps selling and syncs later (response); no sale is lost and the sync finishes within 15 minutes of reconnecting (measure). Offline-first apps and data sync shows how a design meets that one.
Write the environment with care. Most expensive design choices come from it: "at 4 times the normal load", "while one replica is down", "during a deploy". A scenario without an environment is usually tested on a quiet day and passes, and then fails on the day that matters.
Quick check
"When a customer searches, the app shows results with p95 under 400 ms, measured at the API." Which part is missing that would most change the design?
The source (a customer) and the artifact (search behind the API) are clear enough. What is missing is when: on a normal evening or at a festival peak with 4 times the orders. That one choice decides caching, capacity and cost, so it belongs in the scenario.
Turning adjectives into numbers
Stakeholders speak in adjectives. Your job in the room is to ask until each adjective becomes a measure. Five questions work for almost every word:
- Which action? "Fast" for what: search, checkout, saving a goods received note (GRN), a report?
- For what share, measured where? A percentile, never an average (Measuring speed and reliability: p99, SLOs and nines explains why), measured at the API or on the device.
- Under what conditions? Normal day, festival peak, during a deploy, with a part broken.
- What does a miss cost? Minutes of staff time, lost orders, a fine, a lost partner shop. This tells you how much the target is worth.
- What is today's number? Measure before you promise. A target far from today's number is a big project; one close to it may need no architecture change.
| They say | You ask | The measure |
|---|---|---|
| "Fast" | Which action, which percentile, which load? | GRN save p95 under 2 s at 4 times the morning peak |
| "Always up" | Which hours matter, and what counts as down? | Partner ordering available 99.9% from 6:00 to 23:00, per month |
| "Real-time" | How old may the data be before someone acts wrongly? | A buyer's price change is visible in the app within 5 minutes |
| "Scalable" | What grows, by how much, by when, at what cost? | 240,000 app orders a day with checkout p95 unchanged, cost less than twice today's |
| "Secure" | Which asset, which attacker, what response? | A partner shop user can never read another shop's invoices; every attempt alerts within 10 minutes |
| "Easy to change" | Which change, by whom, how long? | A new promotion type in 3 developer-days, in one module |
A useful habit comes from Tom Gilb's Planguage, a notation for requirements: give each measure more than one level. The must level is the worst the business accepts; below it the release is a failure. The plan level is the target you design for. "GRN save p95: must 3 s, plan 2 s" tells the team where the hard line is and where they have room to trade.
Finally, a good measure is checkable by someone other than you. A person should be able to run a test or open a dashboard and say pass or fail. That is what lets you turn the scenario into an automated fitness function later in the course.
Quick check
Which response measure could a teammate check without asking you what you meant?
The first names the percentiles, the place it is measured and the conditions. "As fast as the old system" needs a baseline nobody wrote down, and "notice no delay" depends on who you ask. Both are good starting points for questions, not measures.
Running the workshop
The SEI's Quality Attribute Workshop (QAW) has eight steps: presentation and introductions, the business or mission presentation, the architectural plan presentation, identification of architectural drivers, scenario brainstorming, scenario consolidation, scenario prioritisation and scenario refinement. The full version can take a day or more. Michael Keeling's book Design It! describes a lighter "mini-QAW" that fits a few hours. For most product teams, half a day is enough.
Who comes. Eight to twelve people who feel the system's pain or pay for it: the sponsor, product managers, someone from the stores, finance, the on-call engineer, a data analyst, security, and the outside agency if one builds part of the system. Leave out people who only want to watch; they make the room quieter.
A half-day agenda.
- 15 minutes: purpose and rules. Show one finished scenario so people see the shape.
- 20 minutes: the sponsor presents the business drivers.
- 15 minutes: an engineer shows today's architecture and its known pains.
- 20 minutes: list the drivers and constraints heard so far, each constraint with an owner.
- 60 minutes: brainstorm in a round robin. Each person offers one scenario or passes, until two rounds in a row produce nothing new. The facilitator checks that every scenario has a stimulus and a response.
- 20 minutes: consolidate. Merge scenarios that say the same thing, so votes are not split between duplicates.
- 15 minutes: prioritise with dot voting. A common rule of thumb is to give each person votes equal to about a third of the scenario count.
- 40 minutes: refine the top four to six into full six-part scenarios with numbers. You finish the rest after the meeting and send them out for sign-off.
Two facilitation moves matter most. First, start from incidents: "tell me about the last time this system hurt you" produces better scenarios than "what qualities do you want?". Second, park solutions. When someone says "we need microservices" or "we need GraphQL", write it on a parking list and ask "what problem would that solve?" The answer is the scenario. The solution comes later, in Choosing an architecture style.
You leave the room with three things: the prioritised scenarios, the constraints with owners, and a list of open questions and risks (for example "nobody knows today's p95 for partner orders").
The utility tree: which scenarios drive the design
The SEI's Architecture Tradeoff Analysis Method (ATAM) adds a tool for the last step, the utility tree. The root is "utility", the overall goodness of the system. The branches are quality attributes, the leaves are concrete scenarios. Each leaf gets two ratings, high, medium or low: how important it is to the business, and how difficult or risky it is to achieve.
Read the ratings as a sorting rule. High importance, high difficulty leaves are the architectural drivers: usually three to six of them, and every big decision should trace back to one. High importance with low difficulty is ordinary good engineering; build it, but it will not decide the style. Low or medium importance with high difficulty deserves a hard question before anyone spends money on it.
The tree also shows where scenarios pull against each other. The POS must sell when the link is down, and the customer app must never sell stock that is not there (Case study: ordering without overselling stock). A design choice that helps one hurts the other; ATAM calls such a choice a trade-off point. Write the conflict down and let the business decide which scenario wins, and by how much.
Build it
We will produce the workshop's three outputs for Mangrove's back office, work out the numbers behind two measures, and turn the top scenario into a load test that passes or fails on its own.
-
Write each scenario on one card
Keep scenarios in a Markdown file next to the code, for example
docs/architecture/quality-scenarios.md, so they are reviewed like code. One card per scenario, with an id other documents can point to:Markdown## PERF-1 Goods receiving during the month-end report - Source: a receiver in any of the 320 stores - Stimulus: saves a GRN with up to 40 lines - Environment: morning peak (7:00 to 10:00) while the month-end stock report runs - Artifact: receiving API and the database it writes to - Response: GRN saved, stock updated, confirmation on the handheld scanner - Response measure: p95 under 2 s (must 3 s), p99 under 5 s, 0 failed saves caused by the report, measured at the API - Today: two freezes of about 40 minutes this year; p95 unknown (open question Q3) - Business importance / difficulty: H / H - Raised by: Ama (store manager); votes: 9 - Owner of the measure: back-office on-call engineer -
Record constraints with owners, and the utility tree
Markdown## Constraints | Id | Constraint | Kind | Owner | |-----|--------------------------------------------------------|----------------|-------| | C-1 | PostgreSQL and NestJS remain the platform | Technical | Ken | | C-2 | Partner app built by Lagoon Digital, current contract | Organisational | Imran | | C-3 | At most 6 engineers on the move this year | Organisational | Ken | | C-4 | The 400 tills are existing hardware | Technical | Ken | ## Utility tree (importance, difficulty) - Performance - PERF-1 GRN save under 2 s during the report (H, H) - Modifiability - MOD-1 Pricing change in a weekly release, no freeze (H, H) - Availability - AVL-1 Partner ordering 99.9% from 6:00 to 23:00 (H, M) - Security - SEC-1 A partner shop sees only its own invoices (H, L) - Scalability - SCL-1 10x orders in five years (M, H): ask the sponsor -
Work out what the numbers mean
Turn each measure into counts people can picture. It shows whether the target is strict or loose, and gives the on-call engineer something to check.
OutputPERF-1, morning peak GRNs per day 320 stores x about 4 deliveries = about 1,280 in the 3-hour peak about 60% (an assumption to measure) = about 770 saves p95 under 2 s at most about 38 of those 770 saves may take longer 4x test load 770 / 180 min = about 4.3 saves a minute; test at 20 a minute AVL-1, partner ordering, 6:00 to 23:00 every day hours that count 17 h x 30 days = 510 h a month 99.9% allows 510 h x 0.001 = 0.51 h = about 31 minutes down a month the last freeze 40 minutes, so one freeze alone breaks the target -
Make the top scenario a test that passes or fails
The measure becomes a threshold in a load test. This sketch uses k6, an open-source load testing tool whose scripts are JavaScript. It runs the stimulus and the environment at the same time against a staging copy, and k6 exits with a non-zero code when a threshold fails, so CI can block a release. The API paths are illustrative.
JavaScript// perf-1.js: scenario PERF-1 as a load test. Run: k6 run perf-1.js import http from 'k6/http'; import { check } from 'k6'; const API = __ENV.API_URL; // staging, never production const headers = { 'Content-Type': 'application/json' }; const grn = JSON.stringify({ siteId: 101, supplierId: 7, lines: Array.from({ length: 40 }, (_, i) => ({ articleId: 5000 + i, qty: 12 })), }); export const options = { scenarios: { receiving: { // stimulus: 20 saves a minute, over 4x the peak executor: 'constant-arrival-rate', rate: 20, timeUnit: '1m', duration: '15m', preAllocatedVUs: 5, maxVUs: 20, exec: 'saveGrn', }, report: { // environment: the month-end report runs meanwhile executor: 'per-vu-iterations', vus: 1, iterations: 1, maxDuration: '15m', exec: 'runReport', }, }, thresholds: { // response measure, for the receiving scenario only 'http_req_duration{scenario:receiving}': ['p(95)<2000', 'p(99)<5000'], 'http_req_failed{scenario:receiving}': ['rate<=0'], }, }; export function saveGrn() { const res = http.post(`${API}/grns`, grn, { headers }); check(res, { 'GRN saved': (r) => r.status === 201 }); } export function runReport() { http.post(`${API}/reports/stock-month-end`, null, { timeout: '15m' }); }k6 tags every request with the name of the scenario that sent it, which is why the thresholds can look at receiving requests only. On today's shared database this test should fail. That failure is the evidence for the architecture change, and the same test proves the change worked.
Bring it to your team
Bring these to the next planning meeting or design review that starts with "it must be fast and reliable".
- "Which action do we mean by fast, at which percentile, and under which load?"
- "What did the last incident cost, in minutes, orders or money? That tells us what the target is worth."
- "Is this a constraint or a target? If it is a constraint, who owns it and could lift it?"
- "What is today's number? Who will measure it before we promise a new one?"
- "Which three to six scenarios are high importance and high difficulty, and does each big decision trace to one of them?"
- "Where do two scenarios conflict, and who decides which one wins?"
Red flags
- Requirements full of "fast", "scalable", "real-time" or "robust" with no numbers
- Response times given as averages, or with no place of measurement
- Scenarios with no environment, so they are only ever tested on a quiet day
- A technology choice ("we need Kafka") written down as a requirement
- Targets copied from another company instead of priced for this business
- Constraints nobody can name an owner for
Practice
Lint a quality attribute scenario
Before the workshop's top scenarios go out for sign-off, a script checks each card. Write lintScenario(s). The scenario object has six string fields: source, stimulus, environment, artifact, response and measure. Return an array of problems, empty when the card is ready:
"missing <part>"for each part that is absent or only spaces, in the order above- if the measure is present:
"measure has no number"when it contains no digit - then
"vague word: <word>"for each offast,quick,real-time,instant,seamless,user-friendlyfound in the measure, in that order, ignoring case - then
"use a percentile, not an average"when the measure contains the wordaverage
The starter only checks the measure, and misses blank parts.
function lintScenario(s) {
// return a list of problems; an empty list means the card is ready
const problems = [];
if (!s.measure) problems.push('missing measure');
if (!/[0-9]/.test(s.measure)) problems.push('measure has no number');
return problems;
}const good = {
source: 'a receiver in any store', stimulus: 'saves a 40-line GRN',
environment: 'morning peak while the month-end report runs',
artifact: 'receiving API and database', response: 'GRN saved and stock updated',
measure: '95% of saves under 2 s, 0 failed saves',
};
test('a complete scenario has no problems', () => expect(lintScenario(good)).toEqual([]));
test('blank and absent parts are listed in order', () => {
expect(lintScenario({ ...good, environment: ' ', artifact: undefined })).toEqual(['missing environment', 'missing artifact']);
});
test('a measure with no number is flagged, with its vague word', () => {
expect(lintScenario({ ...good, measure: 'Saves feel Fast to staff' })).toEqual(['measure has no number', 'vague word: fast']);
});
test('vague words are found in any case', () => {
expect(lintScenario({ ...good, measure: 'Real-time: 99% of updates in under 5 s' })).toEqual(['vague word: real-time']);
});
test('an average is not accepted', () => {
expect(lintScenario({ ...good, measure: 'average save time under 1.2 s' })).toEqual(['use a percentile, not an average']);
});
Show a solution
const PARTS = ['source', 'stimulus', 'environment', 'artifact', 'response', 'measure'];
const VAGUE = ['fast', 'quick', 'real-time', 'instant', 'seamless', 'user-friendly'];
function lintScenario(s) {
const problems = [];
for (const part of PARTS) {
const value = s[part];
if (typeof value !== 'string' || value.trim() === '') problems.push(`missing ${part}`);
}
const measure = typeof s.measure === 'string' ? s.measure.toLowerCase() : '';
if (measure.trim() === '') return problems; // already reported as missing
if (!/[0-9]/.test(measure)) problems.push('measure has no number');
for (const word of VAGUE) {
if (measure.includes(word)) problems.push(`vague word: ${word}`);
}
if (measure.includes('average')) problems.push('use a percentile, not an average');
return problems;
}A script like this cannot judge whether a target is right, only whether it is checkable. It is still worth running: it catches the cards that would pass sign-off and then be impossible to test.
Turn three quotes into scenarios
At the workshop you wrote down three quotes. Ama, a store manager: "The Store app must never freeze when we receive deliveries." The finance lead: "Month-end numbers have to be right." A buyer: "We need to change prices whenever we want." Rewrite each as a six-part scenario with a must and a plan level, say which are constraints if any, and rate each for importance and difficulty.
- Each scenario has all six parts, and each environment names a real condition (peak, report running, deploy)
- Each response measure is a number someone else can check, with a percentile where time is measured
- "Right" is turned into something checkable, such as totals matching the ledger to the cent by a set time
- "Whenever we want" is split into how often a change ships and how fast it reaches customers
- Each scenario has importance and difficulty ratings with a one-line reason
Show a model answer
PERF-1 (Ama)
Source: receiver in a store. Stimulus: saves a GRN of up to 40 lines.
Environment: morning peak while the month-end report runs.
Artifact: receiving API and database. Response: saved, stock updated.
Measure: p95 must 3 s, plan 2 s; 0 failed saves. Rating: H, H
(two store-wide freezes this year; needs reporting off the shared database).
DATA-1 (finance)
Source: finance lead. Stimulus: runs the month-end stock valuation.
Environment: first working day of the month, after the 02:00 close.
Artifact: reporting data in the warehouse. Response: report ready.
Measure: totals match the ledger to the cent; ready by 09:00 (must 11:00).
Rating: H, M (correctness is vital; timing is moderate work).
MOD-1 (buyer), split in two
Source: buying team developer. Stimulus: new price rule.
Environment: normal week. Artifact: pricing module.
Response: released without a back-office freeze.
Measure: weekly release, plan 2 developer-days. Rating: H, H.
LAT-1: a published price change reaches the app and the Partner app
within 5 minutes (must 15). Rating: H, M.
None of the quotes is a constraint: each could be traded at a price.Check yourself
The Partner app lead says "the Partner app must handle festival traffic". Which rewrite is a complete quality attribute scenario?
The third option has a source, a stimulus, an environment, an artifact, a response and a checkable measure. The first is a solution, not a requirement. The second has no number. The fourth uses an average over a month, which hides exactly the festival evening the quote was about.
In the brainstorm, an engineer says "we need an event streaming platform". What should the facilitator do?
A technology is a possible answer, and the workshop is about the questions. Parking it keeps the idea without letting it take over, and "what would it solve?" usually reveals a real scenario, such as a price change reaching every app within 5 minutes. It is not a constraint, because nobody has decided it, and voting on solutions here skips the trade-off analysis that comes later.
Four leaves of a utility tree are rated (importance, difficulty). Which one should most influence the choice of architecture style?
High importance with high difficulty is what drives structure; weekly independent releases push towards clear module boundaries. The export is important but easy, so any style handles it. The ten-times scenario is hard but only medium importance, so it is a question for the sponsor before anyone designs for it.
Flashcards
Key takeaways
- Sort what stakeholders say into business drivers, quality attributes (targets you can trade) and constraints (decisions with an owner that you design around).
- Write each quality requirement as a six-part scenario; the environment and the response measure are the parts that change the design and make it testable.
- Turn adjectives into numbers with five questions, price each target by what a miss costs, and give it a must and a plan level.
- A half-day workshop (brainstorm, merge, vote, refine) and a utility tree find the three to six high-importance, high-difficulty scenarios that every big decision should trace back to.
Sources
- Quality Attribute Workshops (QAWs), Third Edition, Software Engineering Institute, 2003
- ATAM: Method for Architecture Evaluation, Software Engineering Institute, 2000
- ISO/IEC 25010:2023 Product quality model, ISO, 2023
- Discover quality requirements with the mini-QAW, IREB Requirements Engineering Magazine
- Thresholds, Grafana k6 documentation