What testing is for
Testing gives your team facts about risk, so people can decide whether to ship, fix or wait. In this lesson you learn to find the risks in a release, score them, and plan testing that puts the most effort where a failure would hurt most. You will do it for Mangrove Market's festival-rush checkout release.
By the end you can
- Explain the chain from error to defect to failure, and say where testing can catch each link
- Use the seven testing principles to argue for or against a test plan
- Name the quality characteristics a change can hurt, and tell verification from validation
- Score product risks by likelihood × impact and decide what you will test, how deeply, and what you will not test
- Ask "how will we know it works?" in refinement and turn the answers into a one-page test plan
The everyday picture
The day before a long drive to visit family for a festival, you take the car to a mechanic. She has one hour. She cannot check every bolt, so for each part she asks two questions: how likely is it to fail on this trip, and how bad would it be if it did? The brakes and tyres get most of the hour, because on a long drive with a full car a failure there is both likely and dangerous. The radio gets a glance.
At the end she does not promise that the car will never break down. Nobody can. She tells you what she found: "Brakes and tyres are good. The battery is old, so it may not start on a cold morning. I did not look at the air conditioning." Now you can decide: go, replace the battery first, or pack a jump-start cable.
Software testing is that hour. Time is always limited, so you spend it where likelihood and impact are highest, and your report is information for a decision. The rest of this lesson turns each part of the mechanic's hour into a method you can use.
Testing gives information for decisions
The ISTQB Foundation Level syllabus (version 4.0, the most common entry certification for testers) lists several objectives of testing: finding defects, checking that requirements are met, building confidence, reducing the level of risk, and giving people the information they need to make informed decisions. The last objective ties the others together. A product manager deciding whether to release on festival eve does not need "312 tests passed". She needs "payments and discounts were tested deeply and work; the slot list slows down above three times normal load; we did not test the old help pages".
To report clearly, you need clear words for what goes wrong. The syllabus uses a chain of three. A person makes an error. The error leaves a defect in a work product: code, a story, a test, a configuration file. When code with a defect runs in the right conditions, the system shows a failure: it does something it should not, or does not do something it should.
Here is the chain at Mangrove Market's checkout. The rule is that delivery is free from $25.00 (2,500 cents); below that it costs $1.50 (150 cents). One line of the story says "orders over $25 get free delivery". A developer reads "over" (the error) and writes cartCents > 2500 (the defect). Every customer with a cart of exactly $25.00 now pays $1.50 they should not (the failure). Customers at $24.99 or $25.01 see nothing wrong, which is why a defect like this can hide for weeks. Some defects never cause a failure at all, because nobody reaches that code.
Behind the error sits a root cause: here, a story that says "over $25" in one place while the design says "from $25". Fixing the line removes one defect. Fixing the way stories state their rules removes a whole family of them.
Each link in the chain is a place to stop the problem, and the earlier you stop it, the cheaper it is. A tester who reads the story before any code exists and asks "is $25.00 itself free?" stops it at the root. That is static testing: checking work products by reading them. A test with a cart of exactly 2,500 cents (you design tests like this in Designing tests that find bugs) makes the failure happen in a test environment instead of in a customer's pocket. Finding the defect that caused a failure is debugging, a developer's job covered in Errors, debugging and testing. Testing finds and shows failures; debugging finds and fixes their defects.
Quick check
No code has been written yet. A tester notices that the FRESH10 story says "carts over $20" while the campaign brief says "carts of $20 or more". What has she found?
Stories, designs and tests are work products, and they can hold defects too. She found one by reading, which is static testing. It is not a failure, because nothing has run. Catching it now costs a one-line edit; catching it after release costs a hotfix and refunds.
Seven principles that shape every test plan
The syllabus lists seven testing principles. Each one is short, and each one changes a decision you make when you plan. Here they are in ISTQB version 4.0 wording, with what each means for you and how it shows up at the checkout.
| Principle | What it means for you | At the checkout |
|---|---|---|
| 1. Testing shows the presence, not the absence of defects | Passing tests lower the risk. They never prove there are no defects. | "All 312 tests passed" goes in the report next to what was not tested. |
| 2. Exhaustive testing is impossible | Use techniques, priorities and risk to choose a small set of tests. | Every cart from 500 to 50,000 cents, times 20 quantities, 3 payment methods and every slot, is billions of cases. You test partitions and boundaries. |
| 3. Early testing saves time and money | Review stories and designs before code exists. | The "over $25" question costs one minute in refinement and a hotfix after release. |
| 4. Defects cluster together | A few areas hold most of the defects. Plan more testing there. | Last festival, most checkout bugs came from slot booking, so it gets extra sessions this time. |
| 5. Tests wear out | The same tests, run again and again, find fewer new defects. Change them and add new ones. | An unchanged checklist keeps passing while each new discount rule breaks in a new way. |
| 6. Testing is context dependent | No single approach fits every product or change. | Payments get deep testing with evidence; a new banner on the home screen gets a quick look. |
| 7. Absence-of-defects fallacy | Meeting every written requirement does not guarantee a product people want. Validate as well as verify. | FRESH10 can match its story perfectly and still win no new customers if nobody finds the code field. |
People misuse two of these. "Exhaustive testing is impossible" is the reason you need test techniques and risk, and it never means "test less". "Tests wear out" says that a fixed set of tests loses its power to find new kinds of defects. It says nothing against regression testing, which reruns old checks to show that a change broke nothing that used to work. That job stays useful forever, and it is the first thing teams automate. What you add, release by release, are new tests aimed at what changed.
Quick check
Mangrove's 400-case checkout checklist has passed unchanged for six releases. In the same months, customers reported nine new bugs in discount rules. What does this suggest?
Passing tests that find nothing new while bugs reach customers is the "tests wear out" principle. The old cases still guard what used to work, so deleting them throws away regression protection. The fix is to keep them and add tests designed for each new rule.
What quality means: the characteristics
"Does it work?" is too vague to plan with. Quality has several sides, and a change can hurt one side while the others look fine. The international standard that names them is ISO/IEC 25010. Its 2023 edition has nine quality characteristics. The 2011 edition, which many courses and the ISTQB syllabus still use, had eight; where a name changed, the old one is in brackets.
| Characteristic | The question it asks | A checkout example |
|---|---|---|
| Functional suitability | Does it do the right things, completely and correctly? | Is the fee 0 at exactly 2,500 cents? |
| Performance efficiency | Is it fast enough with the resources it has? | Does the slot list meet the team's 2-second target at 4× normal orders? |
| Compatibility | Does it work alongside other systems and exchange data with them? | Does a new order reach the pickers' Store app and the Rider app? |
| Interaction capability (usability) | Can people use it easily, including people with disabilities? | Can a first-time shopper find the code field? Does a screen reader announce the fee? |
| Reliability | Does it keep working, and recover when something fails? | If the wallet provider answers late, is the customer charged exactly once? |
| Security | Is data safe from people who should not see or change it? | Can a customer open someone else's order by changing the number in a link? |
| Maintainability | Can the team change and test it easily? | Is the fee rule in one place, or copied into three screens? |
| Flexibility (portability) | Can it adapt to other devices, environments and growth? | Does checkout work on a cheap Android phone with an old OS version? Can it scale for festival week? |
| Safety (new in 2023) | Does it avoid putting people or property in danger? | Mostly for physical systems; at Mangrove it matters for the Rider app more than for checkout. |
Use the list as a prompt when you read a change. For each change in a release, run down the nine and ask "could this change hurt that?" Most answers will be no. Every yes becomes a line in your risk table. Teams that only read the story test only functional suitability, because that is what stories describe. The failures that reach the news are more often reliability, performance and security.
Two testing objectives sound alike, so keep them apart. Verification checks that the product meets its specified requirements: did we build it right? Validation checks that it meets the needs of its users and other stakeholders in its real environment: did we build the right thing? FRESH10 can pass every verification test: 10% off, only from 2,000 cents, first order only, never with the staff discount. Yet if shoppers in a usability session cannot find where to type the code, the feature fails validation, and the campaign will not win the new customers it was built for. That is principle 7 at work. You validate by putting the product in front of real people in real conditions: a usability session, a pilot in a few stores, or user acceptance testing with store staff (Bugs, reports and acceptance).
Quick check
The new goods-received screen in the Store app does exactly what its acceptance criteria say. In a trial store, receivers wearing work gloves keep pressing the wrong quantity button. What kind of finding is this?
The screen matches its requirements, so verification passes. Real users in their real environment cannot use it well, so validation fails. Dismissing it because the criteria passed is the absence-of-defects fallacy; the fix usually starts with a better requirement, such as a minimum button size.
Risk: likelihood times impact
A risk is something that may happen and would cause harm if it did. It has two factors: likelihood, how probable it is, and impact, how bad the harm would be. Together they set the risk level. A quick, common method is to score each factor from 1 to 5 and multiply them: risk score = likelihood × impact, from 1 to 25. The syllabus calls multiplying the quantitative approach; a risk matrix with words such as "low" and "high" is the qualitative approach. Either works if the team agrees on what the levels mean.
A score is only useful if two people would give the same number, so write anchors down. Likelihood 5 means "new, complex code in an area that has failed before"; 1 means "unchanged code with a long clean record". Impact 5 means "customers lose money, or we break a law or leak personal data"; 1 means "a cosmetic glitch nobody would report". Ask a developer for likelihood, because developers know which code is new and tangled. Ask the product manager or the business owner for impact, because they know what a failure costs. Your job as the tester is to ask both, challenge numbers that look too comfortable, and write the result down.
Two kinds of risk go into a test plan. A product risk is about the product's quality: the fee is wrong, a payment is taken twice, the slot list is slow. Testing reduces these, because it finds them before customers do. A project risk is about the work: the wallet provider's test sandbox arrives late, a tester is on leave in festival week, the test environment is smaller than production. Testing cannot remove these. You manage them with plans, people and dates, and you list them because they limit what testing can do.
Quick check
Risk A has likelihood 2 and impact 5. Risk B has likelihood 4 and impact 2. Which do you test first?
2 × 5 = 10 and 4 × 2 = 8, so A comes first. Likelihood alone is a trap: a likely annoyance can matter less than an unlikely disaster. Multiplying both factors is what keeps the rare but severe risk from being ignored.
Risk-based test planning: what, how deep, and what not
Risk-based testing means the risk analysis decides what you test, in what order, and how deeply. You turn scores into depth with tiers the team agrees on. Mangrove's checkout team uses three:
- Deep (score 15 to 25): test cases designed with several techniques, an exploratory session (Exploratory testing), checks below the screen at the API and in the data (Testing APIs and data), a second person reviews the test cases, and these areas are tested first.
- Standard (score 8 to 14): designed test cases for the main rules and boundaries, plus the regression checks for that area.
- Light (score 1 to 7): a short check on two phones, or nothing new if the code did not change.
Add one override: anything with impact 5 is always at least standard, however unlikely it looks. "Card details written to the app logs" scores only 1 × 5 = 5, but the harm is too serious to leave to chance. Areas that score 12 or more, plus every impact-5 area, form the must-test list: the release does not go out until they are tested and the results are known.
Then write down what you will not test, and why. "The Partner app and the Store app are out of scope: this release does not change them" is a decision other people can check and challenge. Silence is a gap nobody sees. When time runs short, you cut from the bottom of the list, and the report says what was cut. That is the mechanic saying "I did not look at the air conditioning".
Risk also sets the order. Test the highest risks first, so that if the release date moves forward, the riskiest areas are already done. When two risks have the same score, put the one with the higher impact first: a rare disaster outranks a common annoyance. And keep scoring as you test. Defects cluster, so an area where you find three bugs in a morning has just shown you its likelihood is higher than you thought.
How testing fits a modern team
In older projects, testers received a finished build at the end and tested it in a separate phase. Modern teams use a whole-team approach: everyone is responsible for quality, and anyone with the right skills can do any task. Developers write unit tests, the product manager writes clear acceptance criteria, the designer checks the screens on real phones, and the tester brings the risk view, designs the tricky tests and explores.
Shift left means moving testing work earlier: reviewing stories before coding starts, writing test cases before the code exists, running automated checks on every change in CI (GitHub Actions: continuous integration), and testing performance on single components instead of waiting for the whole system. It costs some effort early and saves more later, which is principle 3 in practice.
The best place to shift left is refinement, the meeting where the team discusses upcoming stories. Your question there is "how will we know it works?" For the festival-slot story, that question becomes several: What happens if the slot fills while the customer is paying? Does "up to 3 days ahead" count from now or from midnight? Can a cash-on-delivery order take a festival slot? Each answer becomes an acceptance criterion. The story and its acceptance criteria become the test basis, the material your tests are designed from. A vague test basis gives vague tests, so improving it is testing work too.
Testing also happens at several test levels. ISTQB version 4.0 names five: component testing (one piece in isolation, often called unit testing), component integration testing (how pieces talk to each other), system testing (the whole product), system integration testing (the product with outside systems such as the wallet provider), and acceptance testing (by users or the business). Developers usually own the first two. This course works mostly at the system and acceptance levels; the next course, Test Automation and Quality Engineering, covers the lower levels in code.
Build it
Write a risk table and a one-page test plan for Mangrove's festival-rush checkout release. Use a spreadsheet or a markdown file; the templates work in both. The release ships two weeks before the festival and contains five changes: the FRESH10 code for new customers, festival delivery slots bookable up to 3 days ahead, a new automatic retry when the wallet provider answers slowly, a rewritten cart that recalculates the delivery fee, and a new order-history layout.
-
Collect the test basis and list what could go wrong
Gather the five stories with their acceptance criteria, the release notes, last festival's bug list and the top support tickets. For each change, run down the quality characteristics and write one line per real worry. Keep each line concrete enough that someone could test it.
OutputChange What could go wrong (characteristic) FRESH10 code accepted on a 2nd order or with the staff discount (functional) Festival slots slot list slow or failing at 4x orders (performance) last place in a slot booked twice at the same moment (functional) Wallet retry customer charged twice when the provider answers late (reliability) card details written to the app logs by the new code (security) Cart rewrite fee wrong at exactly 2,500 cents (functional) Order-history layout broken on small screens (interaction capability) -
Agree the scales before anyone scores
Write the anchors where everyone can see them. Without them, one person's 3 is another person's 5.
OutputLikelihood 5 new, complex code in an area that failed before 3 changed code, some tests exist, average history 1 unchanged code, long clean record Impact 5 customers lose money, legal or personal-data harm 3 customers are blocked or overcharged a little; workaround exists 1 cosmetic, nobody would report it Score likelihood x impact (1 to 25) Depth 15-25 deep | 8-14 standard | 1-7 light | impact 5 = at least standard Must-test score 12 or more, or impact 5 -
Score, sort and choose the depth
Ask a developer for each likelihood and the product manager for each impact. Sort by score, highest first; on a tie, the higher impact goes first. Then set the depth from the tiers.
Markdown| ID | Risk | Characteristic | L | I | Score | Depth | Must-test | |----|-----------------------------------------------|----------------|---|---|-------|----------|-----------| | R1 | Wallet charged twice on a late reply + retry | Reliability | 4 | 5 | 20 | Deep | yes | | R2 | FRESH10 accepted on 2nd order or with staff | Functional | 4 | 4 | 16 | Deep | yes | | R3 | Slot list slow or failing at 4x orders | Performance | 3 | 4 | 12 | Standard | yes | | R4 | Delivery fee wrong at exactly 2,500 cents | Functional | 4 | 3 | 12 | Standard | yes | | R5 | Last place in a slot booked twice | Functional | 3 | 3 | 9 | Standard | no | | R6 | Order-history layout broken on small screens | Interaction | 3 | 2 | 6 | Light | no | | R7 | Card details written to app logs | Security | 1 | 5 | 5 | Standard | yes (I=5) | -
List the project risks and what you will do about them
These do not go into the score table, because testing cannot find them. They go into the plan because they limit what testing can do.
Markdown## Project risks - The wallet provider's test sandbox is ready only 5 days before release. Action: ask now for a fake wallet service that can answer slowly; test R1 against it first. - One of the two QA engineers is on leave in festival week. Action: finish all must-test areas a week early; a developer pairs on R6. - The load-test environment is a quarter of production's size. Action: test R3 at 4x of that environment's normal load and say so in the report. -
Write the one-page test plan
One page is enough when the risk table carries the detail. Each heading answers a question someone will ask on release day.
Markdown# Test plan: festival checkout release ## Scope In: FRESH10, festival slots, wallet retry, cart fee rewrite, order-history layout. Out: Partner app, Store app, POS, Rider app (no changes in this release). ## Approach (from the risk table) - Deep, tested first: R1 wallet retry, R2 FRESH10. - Standard: R3 slot load, R4 fee at 2,500 cents, R5 last slot, R7 card data in logs. - Light: R6 order history on two small phones. ## Test basis The five stories and their acceptance criteria, last festival's bug list. ## Environments and data Staging with a fake wallet that can answer late; first-time and repeat test customers. ## Entry and exit criteria Start: builds pass CI; test data loaded. Exit: every must-test risk tested; no open bug that charges customers wrongly. ## Project risks See the list above. ## Reporting Daily note in the team channel; one-page summary for the go/no-go meeting: tested, not tested, open bugs, risks that remain. -
Review it with the team and keep it alive
Walk through the plan in refinement. Ask the developer "which code are you least sure about?" and the product manager "which failure would hurt most on festival eve?". If an answer moves a score, change the table. During testing, raise the likelihood of any area where bugs cluster, and add the scores of new risks you discover. The plan is a live document until release day.
Bring it to your team
Ask these in refinement, release planning and go/no-go meetings.
- How will we know this story works? What would make us say it does not?
- Which code in this release are you least sure about, and why?
- If this failed on festival eve, what would it cost us: money, customers, a legal problem?
- What are we choosing not to test, and does everyone agree with that choice?
- Which of our risks are about the product, and which are about the project, and who owns each?
- Did we check how real users will use this, or only that it matches the story?
Red flags
- "QA will catch it" said in refinement: quality has become one person's job.
- A test plan with no "out of scope" line.
- Every area tested to the same depth, whatever its risk.
- Risk scores that nobody outside the test team helped set.
- A release report that says only how many tests passed.
Practice
Score and sort the festival risks
Each risk area is an object like { name: 'FRESH10 rules', likelihood: 4, impact: 4 }. In JavaScript, area.likelihood reads one field of the object and * multiplies two numbers. Complete riskScore(area) so it returns likelihood × impact. Then finish the tie rule in prioritise(areas), which sorts the areas highest score first. The sort function compares two areas, a and b, and needs a number back: below 0 puts a first, above 0 puts b first. So b.impact - a.impact is above 0 when b has the higher impact, which puts b first.
// A risk area looks like: { name: 'FRESH10 rules', likelihood: 4, impact: 4 }
function riskScore(area) {
// Return likelihood × impact. Each is 1 to 5, so the score is 1 to 25.
return 0;
}
function prioritise(areas) {
// [...areas] makes a copy, so the original list stays as it was.
return [...areas].sort((a, b) => {
const byScore = riskScore(b) - riskScore(a); // higher score first
if (byScore !== 0) return byScore;
return 0; // same score: put the area with the higher impact first
});
}const festival = [
{ name: 'Order history layout', likelihood: 3, impact: 2 },
{ name: 'Delivery fee at 2,500 cents', likelihood: 4, impact: 3 },
{ name: 'Card details in logs', likelihood: 1, impact: 5 },
{ name: 'Wallet charged twice', likelihood: 4, impact: 5 },
{ name: 'Slot list at 4x orders', likelihood: 3, impact: 4 },
{ name: 'Last slot booked twice', likelihood: 3, impact: 3 },
{ name: 'FRESH10 rules', likelihood: 4, impact: 4 },
];
test('riskScore multiplies likelihood by impact: 4 x 5 is 20, 1 x 1 is 1', () => {
expect(riskScore({ name: 'x', likelihood: 4, impact: 5 })).toBe(20);
expect(riskScore({ name: 'y', likelihood: 1, impact: 1 })).toBe(1);
});
test('prioritise puts the highest scores first: wallet (20), then FRESH10 (16)', () => {
const names = prioritise(festival).map((a) => a.name);
expect(names.slice(0, 2)).toEqual(['Wallet charged twice', 'FRESH10 rules']);
});
test('two areas score 12: the slot list (impact 4) goes before the delivery fee (impact 3)', () => {
const names = prioritise(festival).map((a) => a.name);
expect(names.indexOf('Slot list at 4x orders')).toBeLessThan(names.indexOf('Delivery fee at 2,500 cents'));
});
test('the whole festival list comes out in risk order', () => {
expect(prioritise(festival).map((a) => a.name)).toEqual([
'Wallet charged twice', 'FRESH10 rules', 'Slot list at 4x orders', 'Delivery fee at 2,500 cents',
'Last slot booked twice', 'Order history layout', 'Card details in logs',
]);
});
test('prioritise leaves the original list unchanged', () => {
const copy = festival.map((a) => a.name);
prioritise(festival);
expect(festival.map((a) => a.name)).toEqual(copy);
});
Show a solution
function riskScore(area) {
return area.likelihood * area.impact;
}
function prioritise(areas) {
return [...areas].sort((a, b) => {
const byScore = riskScore(b) - riskScore(a); // higher score first
if (byScore !== 0) return byScore;
return b.impact - a.impact; // tie: higher impact first
});
}Build the must-test list
The must-test list holds every area that scores 12 or more, plus every area with impact 5, whatever its score. The starter keeps only the first rule. In JavaScript, filter keeps the items for which the rule is true, map turns each kept area into its name, and || means "or". Add the impact-5 rule, and keep the areas in the order they arrive.
function mustTest(areas) {
return areas
.filter((area) => area.likelihood * area.impact >= 12) // keep areas that meet the rule
.map((area) => area.name); // keep only their names
}const festival = [
{ name: 'Wallet charged twice', likelihood: 4, impact: 5 },
{ name: 'FRESH10 rules', likelihood: 4, impact: 4 },
{ name: 'Slot list at 4x orders', likelihood: 3, impact: 4 },
{ name: 'Delivery fee at 2,500 cents', likelihood: 4, impact: 3 },
{ name: 'Last slot booked twice', likelihood: 3, impact: 3 },
{ name: 'Order history layout', likelihood: 3, impact: 2 },
{ name: 'Card details in logs', likelihood: 1, impact: 5 },
];
test('areas scoring 12 or more are on the list, including exactly 12', () => {
const list = mustTest(festival);
expect(list).toContain('Wallet charged twice');
expect(list).toContain('Delivery fee at 2,500 cents');
});
test('an unlikely area with impact 5 is still must-test: card details in logs scores 1 x 5', () => {
expect(mustTest(festival)).toContain('Card details in logs');
});
test('low-risk areas stay off the list: last slot (9) and order history (6)', () => {
const list = mustTest(festival);
expect(list).not.toContain('Last slot booked twice');
expect(list).not.toContain('Order history layout');
});
test('the festival must-test list has exactly five areas, in their original order', () => {
expect(mustTest(festival)).toEqual([
'Wallet charged twice', 'FRESH10 rules', 'Slot list at 4x orders',
'Delivery fee at 2,500 cents', 'Card details in logs',
]);
});
Show a solution
function mustTest(areas) {
return areas
.filter((area) => area.likelihood * area.impact >= 12 || area.impact === 5)
.map((area) => area.name);
}Ask "how will we know it works?" in refinement
The next story in refinement is: "As a customer, I can book a festival delivery slot up to 3 days ahead, so I get my groceries before the festival." Its only acceptance criterion is "Customers can see and book festival slots." Write four questions you would ask in refinement, each naming the risk or quality characteristic behind it, and rewrite the acceptance criterion as two or three checkable lines.
- At least one question is about a boundary or a time rule, such as what "3 days ahead" means at 11:55 pm
- At least one question is about something other than functional suitability: load at 4× orders, a full slot during payment, or a screen reader
- Each question names the risk it protects against
- The rewritten criteria are specific enough that two testers would agree whether they pass
Show a model answer
Questions: (1) Does "3 days ahead" count from now or from midnight today? A slot at 11:55 pm three days out is the boundary (functional suitability). (2) What happens if the last place in a slot is taken while I am paying? Do I lose my money or my slot (reliability, risk of a double booking)? (3) How fast must the slot list load at 4× normal orders on festival eve (performance efficiency)? (4) Can a cash-on-delivery order book a festival slot, and if not, what does the customer see (functional suitability, a blocked customer)?
Rewritten criteria: Given it is 11:00 on day 1, the slot list shows festival slots up to 23:59 on day 4 and none later. Given a slot has one place left and two customers pay for it, exactly one gets the slot and the other is offered the next slot before any payment is taken. The slot list appears within 2 seconds at 4× normal orders.
Check yourself
The festival release passes all 312 checkout tests. The product manager asks, "So there are no bugs left?" What is the best answer?
Testing shows the presence of defects, never their absence, so "defect-free" overclaims. Testing every cart is impossible, so waiting for that blocks every release. The useful answer gives the information a decision needs: depth per area and what was left out.
Which of these belongs in the test plan's project risks, not in the product risk table?
A late sandbox threatens the work, so it is a project risk you manage with dates and a fake wallet service. The other three are about the product's quality (functional suitability, performance efficiency, security), so testing can find them and they go into the scored table.
Two days before release, a broken test environment costs you a full day. Your must-test areas are half done. Following risk-based testing, what do you do?
The risk order exists for exactly this moment: the highest risks are covered whatever happens. Spreading time evenly throws that order away, and delaying may be right, but that is the product manager's decision, made from your report of what was cut.
Flashcards
Key takeaways
- Testing gives information about risk for decisions; it cannot prove software has no defects.
- Errors leave defects that cause failures; catch them as early in the chain as you can, ideally in refinement.
- Quality has nine sides in ISO/IEC 25010:2023; check each change against them, and validate with real users as well as verifying against the story.
- Score risk as likelihood × impact with the developer and the product manager, test the highest first and most deeply, and always test impact-5 risks.
- A good test plan states what will not be tested and why, and fits on one page.
Sources
- Certified Tester Foundation Level (CTFL) v4.0, ISTQB, syllabus version 4.0.1, 2024
- ISTQB CTFL Syllabus v4.0.1 (PDF): sections 1.2 and 1.3 on defects and the seven principles, 5.2 on risk management
- ISTQB Glossary, ISTQB
- ISO/IEC 25010:2023 Product quality model, ISO, 2023
- The ISO 25010 update of 2023, arc42 Quality Model