1. Courses
  2. Engineering Roles
  3. Software Testing and QA
  4. Stop 2

40 minBeginner

Designing tests that find bugs

You cannot test every possible cart, so you need a small set of tests that is very likely to find the bugs that are there. This lesson gives you four black-box techniques that do exactly that, and you will use them on Mangrove Market's checkout: partitions, boundaries, decision tables and state transitions.

By the end you can

  • Split an input into valid and invalid equivalence partitions and pick one value per partition
  • Find the boundary values of a rule and choose 2-value or 3-value boundary tests
  • Turn combined business rules into a decision table and collapse it to the columns worth testing
  • Draw a state model of an order and test valid transitions, invalid transitions and 0-switch coverage
  • Say how much each technique covers, and use pairwise testing when combinations explode

The everyday picture

A fairground ride has a sign: "You must be at least 120 cm tall." The person checking the line does not measure a hundred children to know the rule works. One child who is clearly too short, one who is clearly tall enough: that settles the two groups. The arguments happen at the line itself. A child who is 119 cm, a child who is exactly 120 cm. Is "at least 120" really letting the 120 cm child on?

Test design works the same way. You group inputs that the software should treat the same, test one from each group, and spend your extra effort right at the edges, where people and code most often get the rule slightly wrong. The rest of this lesson adds two more tools for rules that combine (the ride also says "under 140 cm rides with an adult") and for things that move through steps (waiting, riding, stopped).

Equivalence partitioning: one value speaks for many

Every technique starts from the , here the checkout rules the team agreed on. In this course, all money is in integer cents, so $5.00 is 500. Integers avoid rounding errors that floating-point numbers bring.

Output
Checkout rules (customer app)
Cart total allowed:  500 to 50000 cents inclusive
  below 500  -> "Add $5.00 to check out"
  above 50000 -> blocked
Delivery fee:        150 cents when the cart total is below 2500, free from 2500
Quantity per line:   1 to 20, whole numbers only

An is a group of inputs the software should handle the same way. If the checkout charges the fee correctly for a cart of 1,200 cents, it will almost certainly charge it correctly for 1,300 too, because the same line of code decides both. So you test one value per partition and move on.

For the cart total, read the rules and draw the groups. There are two valid partitions: 500 to 2499 (allowed, fee 150) and 2500 to 50000 (allowed, free delivery). There are two invalid partitions: below 500 and above 50000. Four partitions, so four tests, for example 300, 1200, 3000 and 60000. Each of those is a : an input plus the result you expect.

Do the same for quantity per line. Valid: 1 to 20. Invalid: 0 and below, 21 and above, and a value that is not a whole number, such as 2.5. That last one is easy to forget. The rule says "whole numbers", so "not whole" is its own partition, and someone has to decide what the app does with it.

Two habits make partitions useful. First, test invalid partitions one at a time. If a test sends quantity 0 and a cart of 300 together and gets an error, you cannot tell which rule produced it; the second problem hides the first. Second, partitions come from the rules, not from the screen. A quantity stepper that only shows 1 to 20 does not stop the API from receiving 25, so the invalid partitions still need tests (Testing APIs and data does that below the UI).

Quick check

How many tests does equivalence partitioning need for the delivery fee, if you count only the two valid partitions of the cart total?

under 500 blocked, e.g. 300 500 to 2499 fee 150, e.g. 1200 2500 to 50000 free, e.g. 3000 over 50000 blocked, e.g. 60000 max 499 min 500 max 2499 min 2500 max 50000 min 50001
Four partitions of the cart total. Partitioning tests one value inside each box. Boundary value analysis tests the values on both sides of each dashed edge: 499 and 500, 2499 and 2500, 50000 and 50001.

Boundary value analysis: bugs live at the edges

Partitioning picks values from the middle of each group. Most bugs are not in the middle. A developer reads "free from $25.00" and types cart > 2500 instead of cart >= 2500. The middle values 1200 and 3000 still pass. Only a cart of exactly 2500 shows the bug. (BVA) targets these mistakes. A boundary value is the smallest or largest value of an ordered partition, so every edge between two partitions has two boundary values, one on each side.

The ISTQB Foundation syllabus (version 4.0) describes two forms. In 2-value BVA, for each boundary value you test the value itself and its closest neighbour in the next partition. At the free-delivery edge, that means 2499 and 2500. In 3-value BVA, for each boundary value you test the value and both its neighbours. Around the same edge, that gives 2498, 2499, 2500 and 2501. Here is the full set for the checkout rules:

RuleEdge2-value BVA3-value BVA
Cart minimum $5.00499 | 500499, 500498, 499, 500, 501
Free delivery from $25.002499 | 25002499, 25002498, 2499, 2500, 2501
Cart maximum $500.0050000 | 5000150000, 5000149999, 50000, 50001, 50002
Quantity 1 to 200 | 1 and 20 | 210, 1, 20, 21-1, 0, 1, 2, 19, 20, 21, 22

Why would you ever pay for the extra values of 3-value BVA? Because some bugs pass both 2-value tests. Suppose a refactor turns "free from 2500" into "free when the cart equals 2500". At this edge, 2499 still pays the fee and 2500 is still free, so both 2-value tests pass. The 3-value test at 2501 expects free delivery, gets a fee, and catches it. Use 2-value BVA by default and 3-value BVA on the rules where a mistake costs the most, such as money and limits.

Boundaries hide in places that are not obviously numbers too. Dates have them (a slot that closes at 18:00, a code that expires at midnight in which time zone?), lists have them (an empty cart, the 20th line), and text has them (a promo code field that accepts 12 characters). Wherever a rule says "at least", "up to", "from", "before" or "maximum", write down the edge and the values on each side.

Quick check

The rule says the cart maximum is $500.00 inclusive. Which pair is the 2-value boundary test for that edge?

Decision tables: when rules combine

Partitions and boundaries work on one input at a time. Some rules depend on several conditions together. The discount code FRESH10 gives 10% off when the cart total is at least 2000 cents, the customer has never ordered before, and the order does not already have the staff discount. A lists every combination so none is forgotten.

Write the conditions as rows at the top and the actions as rows at the bottom. Each column is one decision rule. With three true-or-false conditions there are 2 × 2 × 2 = 8 columns. Now look for columns where one condition already decides the result. If the cart is under 2000, the code is refused whatever the other two conditions say, so four columns collapse into one, marked with "–" for "does not matter". If the cart is big enough but this is not the first order, the staff discount does not matter either, so two more columns collapse. Eight columns become four:

FRESH10R1R2R3R4
Cart total 2000 or moreTTTF
First orderTTF–
Staff discount on the orderFT––
Apply 10% offX
Message shown"Can't combine with staff discount""For your first order only""Spend $20.00 to use FRESH10"

Each column is a test case, so full decision table coverage here is four tests. The table also does something more valuable: it forces questions. Column R4 says a returning staff member with a $15.00 cart sees the "Spend $20.00" message. Is that the message the product manager wants, or should it say "first order only"? Collapsing a column is a claim that the other conditions do not matter, so check that claim with the team before you rely on it. Other questions fall out too. Is the 2000 checked before or after the delivery fee? If a customer applies the code and then removes items until the cart is 1800, does the discount disappear? Those questions are cheap in refinement and expensive after release.

Techniques combine. The condition "cart total 2000 or more" has its own edge, so add two runs: column R1 at exactly 2000 and column R4 at 1999. One design, two techniques, six tests, each one aimed at a specific way the code could be wrong.

Quick check

A decision table has four true-or-false conditions. How many columns does the full table have before you collapse anything?

State transition testing: what can happen next

An order is not one input. It moves through states, and events move it from one state to the next. The checkout rules for Mangrove orders are:

Output
created         --pay-->        paid             (card or wallet)
created         --chooseCod-->  confirmed-cod    (cash on delivery)
paid            --pick-->       picking
confirmed-cod   --pick-->       picking
picking         --dispatch-->   out-for-delivery
out-for-delivery --deliver-->   delivered
created, paid, confirmed-cod, picking  --cancel-->  cancelled
paid            --refund-->     refunded         (refunds only from paid)

That is 11 valid transitions: three from created, three from paid, two from confirmed-cod, two from picking and one from out-for-delivery. Every other pair of state and event is an invalid transition, and the system must refuse it: cancel an order that is already out for delivery, refund a cash-on-delivery order that was never paid, pay twice for the same order. Invalid transitions are where the expensive bugs live. A double payment or a refund of money you never received costs real money, and both look fine in every screen until someone tries the wrong event in the wrong state.

A lets you measure what you tested. The syllabus names three levels. All states coverage visits every state at least once. Valid transitions coverage, also called , exercises every valid transition at least once; here, all 11. All transitions coverage adds an attempt at every invalid transition too. When you try invalid transitions, try only one per test case: if the first wrong event is wrongly accepted, it changes the state and can hide what the second one would have shown.

A useful way to find invalid transitions is a state table: states down the side, events across the top, the resulting state in each cell. Every empty cell is an invalid transition you could try. Eight states and seven events give 56 cells, and only 11 are valid, so you cannot try all 45 invalid ones in every release. Pick by risk: anything that moves money (pay, refund), anything the customer can trigger from the app (cancel), and anything a rider or picker can trigger at the wrong moment (deliver before dispatch).

Quick check

Your tests take one card order through created, paid, picking, out-for-delivery and delivered. Which coverage is still missing?

cancel allowed created paid confirmed-cod picking out-for-delivery delivered refunded cancelled pay chooseCod pick pick dispatch deliver refund cancel invalid
The order state model. The solid arrows show the 11 valid transitions: the one cancel arrow stands for four, one from each state inside the dashed zone. The red dashed arrow is one invalid transition worth testing: cancel after the order left the store must be refused.

How much is enough: coverage, mutants and pairs

Each technique comes with its own measure of : partitions exercised out of partitions found, boundary values tested out of boundary values found, decision table columns run out of feasible columns, transitions exercised out of transitions in the model. These numbers are useful because they are honest about scope. "100% 2-value BVA coverage of the cart rules" tells a reviewer exactly what you tried and what you did not. They are not a promise of no bugs: 100% coverage of a model with a missing rule still misses the rule.

A second way to judge a set of tests is to ask which bugs it would catch. A is a copy of the code with one small planted mistake, such as > where the rule needs >=. If at least one test fails on the mutant, the tests "kill" it. Mutation testing tools generate mutants automatically from real code; in this lesson's practice, the checker plants typical edge mistakes and tells you which ones your test cases miss. It is a direct way to see why 1200 and 3000 are weak tests for the fee rule, and 2499 and 2500 are strong ones.

Finally, some features are a combination problem. Checkout must work for 3 payment methods (card, wallet, cash on delivery) × 2 slot types (Now, Later) × 3 device groups (a cheap Android phone, a newer Android phone, an iPhone). That is 18 combinations. picks a smaller set in which every pair of values (card with Later, wallet with the cheap phone, and so on) appears at least once. For these three parameters, 9 tests cover every pair. It works because many combination bugs need only two things to line up. Use a pairwise tool or a spreadsheet to build the set, then add the few full combinations that matter most by risk, such as cash on delivery at the festival rush on the cheap phone.

Think first: you have time for only three tests of the delivery fee. Which three?

2499 and 2500 for the free-delivery edge, where the money bug is most likely and most visible, and 499 for the cart minimum, which also tells you whether small carts are blocked. You lose the maximum edge, so write that down as a known gap in your test notes instead of pretending you covered it.

Build it

Write a checkout test-case table where every row names the technique that produced it. A spreadsheet or a markdown file in the team's repo both work.

  1. Copy the test basis and number the rules

    Paste the rules at the top, each with an id, so every test can point to the rule it checks. If a rule is unclear, write the question next to it now.

    Markdown
    ## Test basis: checkout rules (story CHK-214)
    - R-CART: cart total 500 to 50000 cents inclusive; below 500 shows "Add $5.00 to check out"; above 50000 blocked
    - R-FEE: delivery fee 150 cents below 2500; free from 2500
    - R-QTY: quantity per line 1 to 20, whole numbers
    - R-F10: FRESH10 = 10% off, cart 2000 or more, first order only, not with staff discount
    - R-ORDER: order states as in the state model (11 valid transitions)
    Questions: is the FRESH10 minimum checked before or after the fee? What happens to 2.5 in quantity?
  2. List partitions and boundaries per rule

    One small table per rule. This is your design work; the test cases come from it.

    Markdown
    | Rule  | Partition           | Valid? | Partition value | Boundary values |
    |-------|---------------------|--------|-----------------|-----------------|
    | R-FEE | 500 to 2499         | yes    | 1200            | 500, 2499       |
    | R-FEE | 2500 to 50000       | yes    | 3000            | 2500, 50000     |
    | R-CART| below 500           | no     | 300             | 499             |
    | R-CART| above 50000         | no     | 60000           | 50001           |
    | R-QTY | 1 to 20             | yes    | 5               | 1, 20           |
    | R-QTY | 0 and below         | no     | -3              | 0               |
    | R-QTY | 21 and above        | no     | 40              | 21              |
    | R-QTY | not a whole number  | no     | 2.5             | none            |
  3. Add the decision table columns and the transitions

    Copy the four FRESH10 columns as four rows. Then list the 11 valid transitions and the invalid ones you chose by risk, one invalid event per test.

  4. Write the test-case table

    One row per test. Precondition, input and expected result must be exact enough that a teammate gets the same pass or fail. Merge rows when one test covers two items (500 is both a partition value and a boundary value), and say so in the technique column. Below is the start of the table; yours gets a row for every item from steps 2 and 3.

    Markdown
    | ID     | Rule    | Technique              | Precondition             | Input / action          | Expected result                       |
    |--------|---------|------------------------|--------------------------|-------------------------|---------------------------------------|
    | TC-01  | R-CART  | BVA 2-value            | logged-in customer       | cart 499                | "Add $5.00 to check out", no checkout |
    | TC-02  | R-CART  | BVA 2-value            | logged-in customer       | cart 500                | checkout allowed, fee 150             |
    | TC-03  | R-FEE   | BVA 2-value            | logged-in customer       | cart 2499               | fee 150                               |
    | TC-04  | R-FEE   | BVA 2-value            | logged-in customer       | cart 2500               | fee 0                                 |
    | TC-05  | R-FEE   | BVA 3-value            | logged-in customer       | cart 2501               | fee 0                                 |
    | TC-06  | R-CART  | BVA 2-value            | logged-in customer       | cart 50001              | checkout blocked                      |
    | TC-07  | R-QTY   | EP invalid (not whole) | cart with one line       | qty 2.5 via API         | rejected with a clear error           |
    | TC-08  | R-F10   | Decision table R2      | first order, staff disc. | cart 2400, code FRESH10 | "Can't combine with staff discount"   |
    | TC-09  | R-F10   | Decision table R4 + BVA| first order              | cart 1999, code FRESH10 | "Spend $20.00 to use FRESH10"         |
    | TC-10  | R-ORDER | State, invalid         | order out-for-delivery   | cancel                  | refused, state unchanged              |
    | TC-11  | R-ORDER | State, invalid         | order confirmed-cod      | refund                  | refused, no money moves               |
  5. Measure and state your coverage

    Under the table, write one line per technique with the numbers, and list the gaps. A reviewer should see in ten seconds what you tested and what you chose not to.

    Markdown
    Coverage: EP 8/8 partitions | BVA 2-value 10/10 values (3-value only on R-FEE)
    Decision table 4/4 columns | State: valid transitions 11/11 (0-switch), invalid 6 of 45 chosen by risk
    Not covered: pairwise device x payment x slot (planned for the device run), time-zone edges of slots

Bring it to your team

Ask these in refinement, before code exists, and in test reviews.

  • For each "at least", "from" or "up to" in this story, is the edge value inside or outside? Can we write it as a number?
  • Which inputs are invalid, and what exactly should the app say or do for each one?
  • When two discount rules clash, which wins, and what message does the customer see?
  • Which events must the order refuse in each state? Is that enforced by the API or only by hidden buttons?
  • Which technique produced each test, and what coverage do we have per technique?

Red flags

  • Test lists full of middle values (1200, 3000, 5 items) and no edge values.
  • Acceptance criteria that say "a reasonable amount" or "small carts" instead of a number.
  • Only the happy path through the order states, with no attempt at an invalid event.
  • Several invalid inputs in one test, so a failure cannot be traced to one rule.
  • Coverage percentages quoted without saying which technique and which model they measure.

Practice

Write cases that catch the delivery-fee bugs

You write test cases as data, and the checker runs them. checkoutFee(cart) takes a cart total in cents and returns 'blocked' (below 500 or above 50000), 150 (the fee, for 500 to 2499) or 0 (free delivery, from 2500 to 50000). Fill the cases list so that every case expects the right result and, together, the cases catch every planted bug: the checker runs your cases against the correct function and against six buggy versions with typical mistakes at the three edges. Keep it to 12 cases or fewer.

The JavaScript you need: [ ] is a list and its items are separated by commas; { cart: 499, expected: 'blocked' } is one item with two named fields; text goes in quotes and numbers do not.

JavaScript exerciseRuns in your browser with tests
// Each case: a cart total in cents and what checkoutFee should return:
// 'blocked', 150 (the delivery fee in cents) or 0 (free delivery).
const cases = [
  { cart: 499, expected: 'blocked' },
  { cart: 1200, expected: 150 },
];
Ctrl + Enter runs
Show a solution
JavaScript
// 2-value BVA: both sides of each of the three edges
const cases = [
  { cart: 499, expected: 'blocked' },   // just below the minimum
  { cart: 500, expected: 150 },         // the minimum itself
  { cart: 2499, expected: 150 },        // last cart that pays the fee
  { cart: 2500, expected: 0 },          // first cart with free delivery
  { cart: 50000, expected: 0 },         // the maximum itself
  { cart: 50001, expected: 'blocked' }, // just above the maximum
];

Six boundary values kill all six bugs. The partition values 1200 and 3000 kill none of them, which is the whole point of boundary value analysis.

Pass the exercise to finish this task.

Test the order states, valid and invalid

Each item in tries starts an order in a state (from), sends one event, and says what should happen: the new state, or 'rejected' when the event is not allowed. Use the rules and event names from the state model in this lesson (pay, chooseCod, pick, dispatch, deliver, cancel, refund). Reach valid transitions (0-switch) coverage, all 11, and add invalid tries that catch three planted bugs: one lets an order be cancelled after it leaves the store, one allows a refund on a cash-on-delivery order, and one lets a paid order be paid again.

The JavaScript you need is the same as before: a list in [ ] of items in { }, with state and event names in quotes, written exactly as in the state model.

JavaScript exerciseRuns in your browser with tests
// from: the state the order starts in; event: what happens; expected: new state or 'rejected'
const tries = [
  { from: 'created', event: 'pay', expected: 'paid' },
  { from: 'delivered', event: 'cancel', expected: 'rejected' },
];
Ctrl + Enter runs
Show a solution
JavaScript
const tries = [
  // the 11 valid transitions (0-switch coverage)
  { from: 'created', event: 'pay', expected: 'paid' },
  { from: 'created', event: 'chooseCod', expected: 'confirmed-cod' },
  { from: 'created', event: 'cancel', expected: 'cancelled' },
  { from: 'paid', event: 'pick', expected: 'picking' },
  { from: 'paid', event: 'cancel', expected: 'cancelled' },
  { from: 'paid', event: 'refund', expected: 'refunded' },
  { from: 'confirmed-cod', event: 'pick', expected: 'picking' },
  { from: 'confirmed-cod', event: 'cancel', expected: 'cancelled' },
  { from: 'picking', event: 'dispatch', expected: 'out-for-delivery' },
  { from: 'picking', event: 'cancel', expected: 'cancelled' },
  { from: 'out-for-delivery', event: 'deliver', expected: 'delivered' },
  // invalid transitions chosen by risk, one per try
  { from: 'out-for-delivery', event: 'cancel', expected: 'rejected' },
  { from: 'confirmed-cod', event: 'refund', expected: 'rejected' },
  { from: 'paid', event: 'pay', expected: 'rejected' },
];

The first 11 items are the valid transitions; the last three are the invalid ones that move money or break a promise to the customer. Each invalid try is its own item, so a failure points at one rule.

Pass the exercise to finish this task.

Check yourself

A new rule says "Later" delivery slots can be booked from 1 to 3 days ahead. Which set of tests applies 2-value boundary value analysis to that rule?

A tester tries "cancel" and then "refund" on a delivered order in the same test, and the order ends up refunded. What is the main problem with this test?

The festival bundle team adds a fourth condition to FRESH10: "not during the flash sale hour". How does that change the decision table work?

Flashcards

Open my review

Key takeaways

  • Partition each input into valid and invalid groups from the rules, test one value per group, and invalid ones one at a time.
  • Bugs gather at edges: test both sides of every boundary (2-value), and both neighbours (3-value) where mistakes are costly.
  • Decision tables list every combination of conditions; collapsing them shows the tests you need and the questions the rules have not answered.
  • State models show valid transitions to cover (0-switch) and invalid ones to try, chosen by risk: money, cancels and skipped steps.
  • Report coverage per technique, judge test strength by the bugs (mutants) it would catch, and use pairwise for combinations.

Sources

Episode transcriptWhy the bug sits at exactly twenty-five dollars · about 11 minutes · tap any line to play from there
  1. NadiaImran, quick puzzle. The checkout gives free delivery from twenty-five dollars. You can run three tests. Which carts do you pick?
  2. ImranSomething small, like twelve dollars, something big, like thirty, and maybe one around twenty-five.
  3. NadiaThat's what most people say. And the twelve and the thirty will almost never find a bug. The one "around twenty-five" might, if you pick exactly the right cart. Today is about picking the right carts on purpose.
  4. ImranSo this is test design. Making a small set of tests that's very likely to find what's broken.
  5. NadiaExactly. Four techniques: partitions, boundaries, decision tables and state transitions. All on Mangrove's checkout.
  6. NadiaStart with a fairground ride. The sign says you must be at least a hundred and twenty centimetres tall. The person at the gate doesn't measure a hundred kids to know the rule works.
  7. ImranOne kid who's clearly too short, one who's clearly tall enough. Done.
  8. NadiaRight. And where do the arguments happen?
  9. ImranAt the line. The kid who's a hundred and nineteen, and the kid who's exactly a hundred and twenty.
  10. NadiaThat's the whole lesson in one picture. Group the inputs the system should treat the same, test one from each group, and spend your extra effort right at the edges.
  11. ImranLet's do the groups first. What's the official name?
  12. NadiaEquivalence partitioning. A partition is a group of inputs the software should handle the same way. If the fee is right for a twelve dollar cart, it's almost certainly right for thirteen, because the same line of code decides both.
  13. ImranSo for the cart total, what are the groups?
  14. NadiaIf you can see your screen, the first diagram shows four boxes stacked top to bottom. If not, picture four groups. Under five dollars is blocked. Five dollars up to twenty-four ninety-nine pays the dollar fifty fee. Twenty-five up to five hundred is free. Over five hundred is blocked.
  15. ImranTwo valid groups and two invalid ones. So four tests, one from each.
  16. NadiaYes. And for quantity per line, the rule is one to twenty, whole numbers. That gives one valid group and three invalid ones: zero and below, twenty-one and above, and not a whole number, like two and a half.
  17. ImranI'd never have thought of two and a half.
  18. NadiaMost people don't. But the rule says whole numbers, so "not whole" is its own group, and someone has to decide what happens. One more habit: test invalid inputs one at a time. If you send quantity zero and a three dollar cart together and get an error, which rule fired?
  19. ImranNo idea. One problem hides the other. Okay, now the edges.
  20. NadiaBoundary value analysis. A developer reads "free from twenty-five dollars" and types greater than, where the rule needs greater than or equal. Your twelve dollar and thirty dollar tests still pass. Only a cart of exactly twenty-five shows the bug.
  21. ImranSo that's why my middle values were weak.
  22. NadiaYes. On that first diagram, the dashed lines are the edges. The ISTQB Foundation syllabus, version four, describes two ways to test them. Two-value boundary analysis tests the boundary value and its closest neighbour on the other side. At free delivery, that's twenty-four ninety-nine and twenty-five dollars exactly.
  23. ImranAnd three-value?
  24. NadiaEach boundary value and both its neighbours. Around the same edge, that's twenty-four ninety-eight, twenty-four ninety-nine, twenty-five, and twenty-five oh one.
  25. ImranWhy pay for the extra two? The edge is the edge.
  26. NadiaBecause some bugs pass both edge tests. Suppose a refactor makes delivery free only when the cart equals exactly twenty-five dollars. Twenty-four ninety-nine still pays, twenty-five is still free. Both two-value tests pass. Twenty-five oh one gets charged, and the three-value test catches it.
  27. ImranSo two-value by default, three-value where mistakes cost real money.
  28. NadiaThat's it. And here's a story from the lesson. For the festival rush, marketing planned a twenty-five dollar festival basket. Rice, oil, lentils, sugar, priced at exactly twenty-five, so it would just reach free delivery.
  29. ImranOh no. I can guess.
  30. NadiaA QA engineer on the checkout team wrote the boundary cases before the release. The case at exactly twenty-five failed. The code said greater than. Every basket of exactly twenty-five dollars would have paid the dollar fifty fee.
  31. ImranAnd businesses put prices right on the threshold on purpose. That's the whole point of the basket.
  32. NadiaExactly. Tomás estimated roughly four thousand baskets a day on that line in festival week. The fix was one character. The test stayed in the regression set, labelled with its technique.
  33. ImranOkay, next technique. What about rules that combine? Like the discount code, FRESH ten.
  34. NadiaThat's a decision table. FRESH ten gives ten percent off when the cart is at least twenty dollars, it's the customer's first order, and there's no staff discount. Three yes or no conditions, so two times two times two, eight combinations.
  35. ImranEight tests for one discount code? That feels like a lot.
  36. NadiaYou collapse it. If the cart is under twenty dollars, the code is refused whatever the other two say. Four columns become one. If it's not the first order, the staff discount doesn't matter either. Eight columns become four, and each column is one test.
  37. ImranI like that. And I bet the table raises questions too.
  38. NadiaThat's the best part. A returning staff member with a fifteen dollar cart. Which message do they see? "Spend twenty dollars", or "first order only"? Collapsing a column claims the other conditions don't matter. You check that claim with the product manager before you rely on it.
  39. ImranThat's my job in refinement, then. Asking which message wins.
  40. NadiaRight. And the techniques combine. The twenty dollar condition has its own edge, so you run one column at exactly twenty dollars and another at nineteen ninety-nine.
  41. ImranLast one. The order states.
  42. NadiaIf you can see the screen, the second diagram is the order's life. If not, picture this. An order is created. Card or wallet makes it paid, cash on delivery makes it confirmed COD. Both go to picking, then out for delivery, then delivered.
  43. NadiaCancel is allowed from created, paid, confirmed COD and picking. Refund only from paid. Count the arrows and you get eleven valid transitions.
  44. ImranSo I test those eleven and I'm done?
  45. NadiaThat's called valid transitions coverage, or zero-switch coverage. It's a good start, and it's not where the expensive bugs are. They live in the invalid transitions.
  46. ImranLike what?
  47. NadiaCancel after the rider has left the store. Refund a cash on delivery order that was never paid. Pay twice for the same order. Each one moves money or breaks a promise, and each looks fine on every screen until someone tries the wrong event in the wrong state.
  48. ImranHow many invalid ones are there?
  49. NadiaEight states, seven events, fifty-six cells in a state table. Eleven are valid, so forty-five invalid. You pick by risk: money, cancels, and skipped steps. And you try one invalid event per test, so one wrong acceptance can't hide another.
  50. ImranHow do I know when I've done enough? My manager always asks for a coverage number.
  51. NadiaGive one per technique. Partitions tested out of partitions found, boundary values out of boundary values, columns out of columns, transitions out of transitions. It tells a reviewer exactly what you tried. It doesn't promise there are no bugs.
  52. ImranAnd the practice section talks about mutants. That sounds scary.
  53. NadiaIt's friendly, I promise. A mutant is a copy of the code with one small planted mistake, like greater than instead of greater than or equal. If one of your tests fails on it, you've killed the mutant. In the practice, you write the cases and the checker tells you which planted bugs you missed.
  54. ImranAnd I bet twelve dollars and thirty dollars kill nothing.
  55. NadiaNothing at all. Six boundary values kill all six.
  56. ImranWhat about combinations? Card, wallet or cash, Now or Later slots, cheap Android, newer Android, iPhone.
  57. NadiaEighteen combinations. Pairwise testing picks a set where every pair of values appears at least once. For those three, nine tests cover every pair. Then you add the few full combinations that matter most, like cash on delivery on the cheap phone at festival time.
  58. ImranSo what do I actually build this week?
  59. NadiaA test case table. First, paste the rules at the top and give each one an id. Then a small table per rule with its partitions and boundary values. Then the decision table columns and the transitions.
  60. NadiaThen one row per test, with precondition, input, the exact expected result, and a column that names the technique. At the bottom, one coverage line per technique and a list of what you chose not to test.
  61. ImranI'd want that last part in every test plan I review. What we didn't test, written down.
  62. NadiaThat's honest scope. Here's a question for you. Take a second. You only have time for three delivery fee tests. Which three?
  63. ImranHmm. Twenty-four ninety-nine and twenty-five, for the free delivery edge. And four ninety-nine, for the minimum.
  64. NadiaGood. And write down that the five hundred dollar maximum is a known gap, instead of pretending you covered it.
  65. ImranLet me try the takeaways. One: group the inputs, test one value per group, and invalid ones one at a time.
  66. ImranTwo: bugs live at the edges, so test both sides of every boundary, and use three-value where money is on the line.
  67. ImranThree: decision tables and state models show you the combinations and the wrong moves nobody wrote down.
  68. NadiaOne nuance on the third. The table's real value is the questions it forces in refinement. Ask them before the code exists, when answers are cheap.
  69. ImranAnd next time?
  70. NadiaNext time we put the tables down and explore. The Store app's receiving flow, a charter, a timer, and a notebook full of surprises.
Finished reading?Mark it complete to fill in its stop on your course line.

Use it at work

Checkout boundary bugs

Off-by-one at the edges of checkout.

Open in the QA Lab
Next stopExploratory testing
Was this lesson helpful?
Questions and discussion

Loading the discussion…