Why polite people give you bad data

In discovery you try to learn what is true about people's work and lives before you decide what to build. Interviews are the cheapest way to start, and also the easiest to get wrong. Three forces bend the answers you hear.

• Kindness. Once you describe your idea, people protect your feelings. "Sounds useful" costs them nothing.
• Bad forecasting. People are poor at predicting what they will do. Everyone plans to exercise more, save more and switch to the better app. Few do.
• Tidy summaries. Asked what they "usually" do, people describe the official process or their best day. The workarounds and the bad days drop out.

Fitzpatrick's three rules push back against these forces. Talk about their life instead of your idea. Ask about specifics in the past instead of generics or opinions about the future. Talk less and listen more. He also names the kinds of bad data to watch for in your own notes:

• Compliments such as "Great idea!" Note them, then ignore them.
• Fluff: generic claims ("I always check the app"), future promises ("I would definitely use that") and hypotheticals ("If it had a voice button, I might").
• Feature requests such as "Add a reorder button." Treat each one as a clue. Ask what happened that made them want it.

Look at what the right-hand column has in common. Each question points at one real event the person lived through. You can follow it up ("and then?"), check it against what you see in the shop, and count how many people tell a similar story.

The story-based interview

Teresa Torres, in Continuous Discovery Habits (2021), builds her interviews around one prompt: "Tell me about the last time you…". The aim is a story-based interview: one specific event, told in order, with enough detail that you could act it out. The first answer is always thin. "I ordered on Tuesday" is a headline. Your job, in Torres's words, is to excavate the rest.

Use the story as a spine and move along it with neutral probes:

• Set the scene. "What day was it? Where were you? What else was going on?"
• Walk forward. "What happened next?" "And then?" This one question does most of the work.
• Find the decisions. "How did you decide how much to order?" "Why then, and not the day before?"
• Find the workarounds. "What did you do about that?" "Who did you ask?" "Can you show me?"
• Find the cost. "How long did that take?" "What did it cost you when that item ran out?"

A few rules keep the story honest. Ask one question at a time. Don't suggest reasons: "Was it because the app is slow?" is a leading question, and most people will politely agree. Let silence sit for a few seconds; people fill it with the detail you need. Keep your idea to yourself until the end, if you show it at all, because once you pitch, every answer bends toward it. If you are in their shop or store, ask to see the thing they mention: the notebook, the chat thread, the crumpled invoice. Artefacts don't flatter.

A 30-minute interview has a simple shape. Spend two minutes on who you are and on consent to take notes or record. Spend about twenty minutes on one or two stories. Use a few minutes on anything that surprised you. Close with a thank-you that says how you will use what they told you. If the first story ends early, ask for a contrasting one: "Tell me about a time ordering went badly." Two stories side by side show you what varies and why.

Recruiting store staff and shop owners

The best questions are useless if you ask the wrong people. Recruit by behaviour, not by job title or city. For Mangrove's partner question, "shops that placed most of their orders by phone last month" is a far better filter than "small shops". Add a few people who do the opposite, such as shops that moved to the app this year, because contrast explains more than agreement does.

Getting time from people who are paid by the hour, or who run a shop alone, takes more care than booking a video call with an office worker.

Store staff on shift. Agree it with the store manager first and ask her which hour is quiet. In most grocery stores the morning delivery window and the evening rush are the worst times; a weekday mid-afternoon is often the best. Keep it to 15 to 20 minutes and make sure it counts as paid work time. Interview where the work happens, such as the receiving bay, so people can show you. Ask the manager to step away. Staff will not criticise a system, or admit to a workaround that bends a rule, with their boss listening. Say plainly that this is not a test, that there are no wrong answers, and that you will not tell their manager what any one person said.

Shop owners. A corner shop owner is the cashier, the buyer and the cleaner at once. Go at a time they pick, expect interruptions, and stop talking whenever a customer walks in. A sales rep's introduction can open the door, but don't let the rep choose the shops: reps pick their friendliest customers. Pull the list from order data instead.

Thank-yous. Torres suggests trying without an incentive first, because many people like being asked. Company rules may forbid gifts to staff; a thank-you from their manager and a short note on what you learned often matter more. For shop owners a small, standard thank-you is fair. Avoid paying with a discount on the very thing you are researching, which turns the interview into a sales visit.

How many? There is no magic number. Nielsen Norman Group suggests starting with a small sample of around five or six, analysing as you go, and stopping when new interviews stop adding new themes. That point is called saturation. A narrow question with similar people saturates quickly. A broad question across very different people needs many more. Either way, you will contact far more people than you end up interviewing, as the next diagram shows.

Interviewing across languages

Mangrove works in six cities, and many store staff and shop owners are most comfortable in a language that isn't the one head office uses. Interview them in your language and you get short, careful answers and lose the detail that matters. Interview in theirs.

• Use a neutral interpreter. A professional, or a trained colleague from another team, works well. The person's sales rep or supervisor does not. They have a stake in the answer and will smooth it, often without meaning to.
• Brief the interpreter. Explain the goal, the story prompt and the probes. Ask for word-for-word interpretation in the first person ("I called the rep…") rather than summaries ("He says it's fine").
• Speak to the participant, not the interpreter. Look at them, use short sentences, ask one question at a time, and pause so each one can be interpreted.
• Translate the guide in advance, then have someone translate it back. This back-translation catches questions whose meaning shifted, such as a word for "order" that also means "command".
• Collect the words that don't travel. Product names, "credit", "app" and "delivery slot" may have local words, slang or borrowed English. Write down the words people use. They belong in your product's labels and search.
• Plan more time. Everything is said twice, so a 30-minute interview needs roughly 45 minutes.

If you speak some of the language but not well, say so, and invite people to correct you. People are generous with someone who tries, and the corrections are often where the most useful words appear. A 2023 post on the UK government's user research blog describes sessions run across a language barrier: agree everyone's role before the session, let the translator welcome the participant first, and lean on watching what people do.