Context

Why this matters

Teresa Torres (Continuous Discovery Habits) and Petra Wille (Strong Product People) co-host this show, and this episode grew out of a hallway question Petra kept hearing from clients: with behavioral analytics, sales notes, and support tickets already flowing in, do product teams still need to run user interviews. Teresa answers by drawing on real interview transcripts she reviews through her AI product Vistaly, and a framework she built years ago for grading evidence quality.

The Big Idea

Most of what teams call product evidence, support tickets, app store reviews, sales notes, even most "interviews," is a weak signal that flags something is wrong without telling you what to build, and the fix isn't demanding perfection, it's climbing one rung up the ladder of evidence at a time.

Teresa's central example: a team watched a customer demonstrate a usability bug but never asked why the customer needed that behavior in the first place, so when they built a fix, they were guessing.

Key Insights

Weak signals hide their own context

Support tickets, app store reviews, and sales notes rarely include why the customer needed something, only the symptom, like "this feature looks broken." When product experts fill that missing context in themselves, they substitute their own experience for the customer's real, sometimes completely different, intent. That's exactly what led one team astray in Teresa's example.

Most "interviews" aren't customer interviews

Teresa sees this constantly in transcripts submitted through Vistaly: teams label product demos, stakeholder meetings, and usability-preference Q&As as customer interviews. A real story-based interview asks what the customer was doing, when the need arose, and what problem they were actually solving, not just what they currently prefer.

A rotation bug shows the pattern

A team's interview surfaced a real usability issue: a customer struggling to rotate a device from portrait to landscape. The interviewer never asked why landscape mattered to that customer or what task it supported, so when the team built a fix, they were guessing at "landscape works better," with no way to verify it actually solved the underlying problem.

Even weak signals still carry value

A low-quality signal isn't worthless, it's just not strong enough to justify a confident decision on its own. Teresa's framing: risk is lowest with a rich, story-based interview and highest with an unverified guess, and everything else, a support ticket, a sales note, a general interview, sits somewhere on that spectrum in between.

Rejecting weak evidence can backfire

Vistaly could restrict itself to only accepting strict story-based interviews as valid signal, but Teresa says that would shrink her usable customer base to about 12 people. The actual choice the team made: accept weaker "general interviews" too, then coach users toward better interviewing over time instead of blocking them outright.

Mental Models & Frameworks

Ladder of evidence

Teresa's own framework, built roughly ten to fifteen years ago: as you climb toward richer, story-based evidence, both the effort required and the value of what you learn go up together. Low-value signals, a support ticket, an app store review, sit at the bottom; a full story-based interview sits at the top.

Kniberg's triangle for validating an idea

Petra's coaching shortcut, drawn from Henrik Kniberg: before acting on an idea, check for three things at once, a quantitative signal such as an app store review pattern, an internal expert or colleague who backs it, and a piece of real qualitative evidence. If all three line up, the idea is worth pursuing.

Trade-offs & Nuance

Perfection versus shipping a decision

Waiting for a perfectly conducted, story-based interview before making any product decision isn't realistic, since many teams are already deciding with no interviews at all. Teresa's resolution: figure out the actual signal strength of whatever evidence exists, even a mediocre interview, and decide with that, rather than discarding usable but imperfect evidence entirely.

Discouraging bad interviews backfires further

Telling a team their interview "doesn't count" because it wasn't story-based risks teaching them not to talk to customers at all, which is worse than an imperfect interview. Teresa and Petra's approach is to accept the weaker evidence, then coach specific, better questions for next time, rather than rejecting the format outright.

Common Mistakes

Mistaking a demo for an interview

Teams submit product demonstrations, walking a customer through the product and asking "how do you like this," as if they were customer research. A demo tests reactions to something already built; it doesn't surface the customer's actual problem or story, which is what a real interview is for.

Asking preferences instead of stories

An interview that asks "tell me about your experience" and follows up with "what don't you like about it" feels open-ended but isn't grounded in a specific instance. It collects idiosyncratic opinions with no surrounding context, not the kind of evidence that reliably predicts what to build.

Practical Application

Grade evidence before acting on it

Before treating a support ticket, sales note, or interview as grounds for a product decision, ask how far up the ladder of evidence it actually sits, and how much confidence that level deserves, rather than treating every input as equally actionable.

Ask for the story, not preferences

When running an interview, push past "what do you think" to the specific moment: what were you doing, when did the need come up, what were you trying to accomplish. A preference without a grounding story is a much weaker signal than it feels like in the room.

Coach interviewers with side-by-side nudges

Rather than rejecting a weak interview outright, show the interviewer the specific gap in real time, "next time, try asking this instead," using their own transcript as the coaching material. This teaches better interviewing without discouraging teams from talking to customers in the first place.

Don't wait for perfect evidence

If a team hasn't run a single interview yet, an imperfect one is still better than none. Use whatever signal exists, weighted by how much confidence that signal actually deserves, rather than blocking a decision until a flawless, story-based interview is available.

Questions to Consider

• When a support ticket or app store review lands on our team, are we filling in the missing "why" with the customer's real intent, or with our own assumption as product experts?
• How many of the sessions we're currently counting as "customer interviews" are actually product demos, stakeholder meetings, or preference surveys in disguise?
• If we only accepted our strictest, most rigorous form of customer evidence, how much of our current research volume would we actually be throwing away?
• Are we coaching teams toward better interviews with specific next-step nudges, or are we discouraging them from talking to customers at all by rejecting what they already collected?

Bottom Line

Not all evidence is equal, but almost all of it carries some signal, and the goal isn't demanding perfect story-based interviews before every decision, it's learning to read how much confidence each rung of the evidence ladder actually deserves.

Grade your evidence by where it sits on the ladder, and coach interviewers toward better questions instead of discarding what they already collected.

Concepts to Explore

Story-based interviewing

An interview technique focused on collecting a full, specific story, what the customer was doing, what triggered the need, what they tried, whether it actually solved their problem, rather than direct questions about preferences. Teresa treats it as a learnable skill that takes real practice to build and maintain.

Tools & Products

Notable Quotes

"It's really important that we collect that context. If you go and talk to that customer, you learn, oh no, they're trying to do something like a total oddball outlier with your product that you never thought they would do with it." (Teresa Torres)

"The worst thing you could do is never talk to a customer. The best thing you can do is collect a really rich story about their experience. And then there's this whole spectrum in between." (Teresa Torres)