Test design techniques

Each technique finds a different kind of bug with fewer test cases than guessing. The examples use Mangrove's app and Store app (Test design).

What to test at which layer

Most checks belong low in the pyramid, where they are fast and pin the bug to one place. Keep end-to-end tests for the few journeys that must never break (Test strategy).

Bug report template

One bug per report. A developer should be able to reproduce it without asking you anything (Bugs and UAT).

Title: Checkout total ignores coupon when cart has a promo item (Android)

Environment: Mangrove app 4.12.0 (build 812), Android 14, Pixel 7, staging API
Severity: Major   Priority: High   Found by: exploratory session, checkout charter

Steps to reproduce
1. Sign in as test user qa.member@example.test (member)
2. Add "Basmati rice 5 kg" (promo item) and "Milk 1 L" to the cart
3. Apply coupon SAVE10
4. Open the checkout screen

Expected: discount line -$0.12 (10% of Milk 1 L at $1.20); total $16.38
Actual:   total $16.50; coupon shows "applied" but no discount line

Frequency: 5 of 5 tries. Does not happen without the promo item.
Evidence: screen recording, request id 7f3c2a in API logs
Notes: iOS 17 on the same build shows the right total

Playwright: "I want to..."

Commands for @playwright/test run from the project folder (Playwright E2E).

Locators and assertions

Find elements the way a user does. Role and label locators survive redesigns; CSS paths tied to layout break every sprint. Web-first assertions such as toBeVisible retry until a timeout, so you never need a fixed sleep (Playwright E2E, Accessibility).

import { test, expect } from '@playwright/test';

test('member coupon lowers the total', async ({ page }) => {
  await page.goto('/cart');
  await page.getByLabel('Coupon code').fill('SAVE10');
  await page.getByRole('button', { name: 'Apply' }).click();
  await expect(page.getByRole('status')).toHaveText('Coupon applied');
  await expect(page.getByTestId('cart-total')).toHaveText('$14.85');
});

Jest and Vitest side by side

The APIs are nearly the same; Vitest uses vi where Jest uses jest. Vitest watches by default in a terminal and runs once in CI (Unit and API tests, Testing React).

API testing checklist

Run each row against every new endpoint. curl -i shows the status line and headers; add -H 'Content-Type: application/json' -d '{"qty":2}' to send a body (API testing).

Flaky tests: if you see X, reach for Y

A flaky test is a bug in the test, the app or the environment. Quarantine it with an owner and a date; do not just add retries and forget it (Stable CI).

Accessibility quick checks

Automated scanners such as axe (@axe-core/playwright) catch only part of the problems. The keyboard and a screen reader catch most of the rest. Targets are WCAG 2.2 level AA (Accessibility, Accessible and fast).

Load testing in one screen

Test against staging sized like production, never against production without an agreed window. Decide the pass mark before you run: for example p95 under 500 ms and under 1% errors at 4 times normal traffic, Mangrove's festival peak (Load testing).

// k6 script: run with  k6 run browse.js
import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  vus: 50,
  duration: '5m',
  thresholds: {
    http_req_duration: ['p(95)<500'],
    http_req_failed: ['rate<0.01'],
  },
};

export default function () {
  const res = http.get('https://staging.example.test/api/products?store=S-014');
  check(res, { 'status is 200': (r) => r.status === 200 });
  sleep(1);
}

UAT and release sign-off

User acceptance testing asks whether the change fits the real job, done by the people who do it, such as Ama's store team on real handheld scanners. Agree entry and exit criteria before it starts (Bugs and UAT, Change and UAT).

• Entry: build deployed to the UAT environment, test data loaded, known issues listed, testers briefed with scenarios written in business words.
• During: testers log issues in one place; QA triages daily with the product owner.
• Exit: every scenario run; no open critical or major bugs; minor ones accepted in writing with a fix date.
• Sign-off note: build number, scenarios passed, open issues accepted, who signed and when, and the rollback plan.

Glossary

Plain definitions of the testing terms used across the QA courses.

Acceptance criteria

The conditions a story must meet to be accepted, often written as Given, When, Then. Stories and criteria

Boundary value

A value at the edge of a rule, such as the maximum quantity, where off-by-one bugs live. Test design

Charter

A one-line mission for an exploratory session: explore a target, with a resource, to find a kind of risk. Exploration

Code coverage

The share of code lines or branches that tests ran; it shows what is untested, not that tested code is right. Test strategy

Contract test

A test that checks a consumer and a provider agree on the shape of requests and responses. Contract tests

End-to-end (E2E) test

A test that drives the whole system through the user interface, as a user would. Playwright E2E

Exploratory testing

Designing and running tests at the same time, learning as you go, inside a time box. Exploration

Fixture

Setup a test needs, such as a signed-in page or a seeded store, created fresh for each test. Playwright E2E

Flaky test

A test that sometimes passes and sometimes fails on the same code. Stable CI

Mock, stub, fake

Stand-ins for real dependencies: a stub returns fixed answers, a mock also checks how it was called, a fake is a simple working version. Unit and API tests

Regression

Something that used to work and broke after a change. Why we test

Severity and priority

Severity is how bad the bug's effect is; priority is how soon it should be fixed. Bugs and UAT

Smoke test

A few quick checks that the most important things work, run right after a deploy. Test strategy

Test oracle

How you know the right answer: a spec, a calculation, an older version or a person's judgement. Test design

Test pyramid

Many fast low-level tests, fewer integration tests and only a few end-to-end tests. Test strategy

Trace

Playwright's recording of a run: actions, DOM snapshots, network and console, for debugging failures. Stable CI

UAT

User acceptance testing: the people who will use the change confirm it fits their work. Bugs and UAT

WCAG

Web Content Accessibility Guidelines; level AA of version 2.2 is the usual target. Accessibility

Web-first assertion

An assertion that retries until the page reaches the expected state or a timeout passes. Playwright E2E

p95 latency

The response time that 95 out of 100 requests are faster than; the usual load-test target. Load testing

Courses on Kolom

• Software Testing and QA: test design, exploratory testing, API and accessibility testing, bug reports and UAT.
• Test Automation and Quality Engineering: strategy, unit and API tests, Playwright, contract and load tests, and a stable CI.
• React for the Web: ends with testing React components the way users use them.
• Mobile Apps with React Native: testing and releasing a mobile app to the stores.

What to read next

• Playwright documentation at playwright.dev. The guides on locators, assertions and the trace viewer are short and exact.
• The Practical Test Pyramid by Ham Vocke, at martinfowler.com. What to test at each layer, with examples.
• WCAG 2.2 quick reference at w3.org/WAI. Every success criterion with techniques, filterable by level.
• Grafana k6 documentation at grafana.com/docs/k6. Test types, thresholds and scenarios.
• Explore It! by Elisabeth Hendrickson. The best short book on charters and exploratory testing.