1. Courses
  2. Product Management
  3. Product Discovery
  4. Stop 4

40 minBeginner

Testing ideas cheaply

Building the real thing is the most expensive way to learn that an idea doesn't work. This lesson gives you four cheaper tests (fake doors, concierge tests, Wizard of Oz tests and usability tests), shows you how to set the pass mark before you start, and how to run each one fairly with shop owners and frontline staff.

By the end you can

  • Write a test card with a measurable pass mark before you run anything
  • Pick the cheapest test that answers a given assumption
  • Run a fake door, a concierge test and a Wizard of Oz test without misleading anyone
  • Run a five-person usability test on a phone or a handheld scanner, and rank what you find
  • Read results without fooling yourself, including results that land in the grey zone

The everyday picture

A street vendor wants to add fresh orange juice. A juicing machine costs a month's profit. So for one week she squeezes oranges by hand behind the counter and writes on her chalkboard: "Fresh juice, $1". She decides in advance that if she sells at least 30 cups a day, she'll buy the machine. By Friday she has her answer, and it cost her a sore wrist and a few bags of oranges.

She also learned things a machine would have hidden. Customers want it cold. Mornings sell three times more than afternoons. Nobody buys juice when it rains. A cheap test answers the question you asked, and it teaches you things you didn't know to ask.

Decide what would change your mind, before you test

An is the smallest piece of work that tells you whether a belief from your assumptions map (From research to opportunities) is true. Its most important part happens before any work starts: writing down what result counts as a pass. Decide afterwards and you will find a way to call any result a success.

Strategyzer's , from the work of David Bland and Alexander Osterwalder, has four lines: the hypothesis, the test, the measure and the success criteria. Alberto Savoia's , from The Right It (2019), makes the success line concrete: at least X% of Y will do Z. Write a broad version for the market, then a narrow version you can test this month.

Output
Market: at least 30% of phone-ordering shops will place orders in the app
        within a month of seeing their balance there.
Test:   at least 6 of 20 phone-ordering shops in two areas will confirm an order
        from a weekly balance message within 2 days, before the rep calls.

Decide what each result will mean before you see it. Above the pass mark, continue. Well below it, drop the idea or rethink it. In between, change one thing and test again. Set a time limit too, so the test can't drift on for months.

Evidence also comes in strengths. What people say is weakest. What they do is stronger. What they give up, such as money, time, effort or their standing with a colleague, is strongest. Savoia calls this having skin in the game. A tap on a button is a small action. Confirming a real order with real money is a big one.

Quick check

Which pass mark gives the strongest evidence for "phone-ordering owners will order in the app without waiting for the rep"?

Loading the rest of the lesson…