Metrics question
Users can create Skills in Computer, but many run a Skill once and never use it again. How would you determine whether the main issue is setup friction, weak discoverability, inconsistent results, narrow applicability, or lack of trust, and what product changes would you test first to improve repeat use and sharing?
- Perplexity
- Metrics
- Hard
Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.
Start a mock interview on this question · Mock interview from a job description
What this question tests
Tests diagnosing low repeat usage of a created artifact across five plausible causes, and designing targeted tests instead of a single broad fix.
How to approach it
- Instrument the funnel: Skill created, Skill run once, Skill run again within a week, Skill shared with someone else.
- Check setup friction: is time-to-first-successful-run high, and do many Skills fail or need edits on the first run.
- Check discoverability: do users who created a Skill know how to find and re-trigger it later, or does it disappear into a list.
- Check result consistency: does the same Skill produce noticeably different quality results across runs, undermining trust in reuse.
- Check applicability and trust: interview one-time users to see if the Skill solved a genuinely recurring need or a one-off task that has no reason to repeat.
- Design small tests per hypothesis, for example a persistent shortcut to re-run a Skill for discoverability, and a consistency-focused evaluation pass for reliability, and run them in parallel where feasible.
What a strong answer includes
- Breaks the funnel into created, run once, run again, shared, isolating exactly where the drop-off happens instead of assuming one cause.
- Distinguishes a one-off-task problem, no reason to repeat, from a trust or discoverability problem, which need very different fixes.
- Proposes a concrete, targeted test per hypothesis rather than one broad redesign.
- Uses direct user interviews with one-time users to separate applicability from a product usability gap.
Common mistakes
- Assuming setup friction is the cause without checking whether the task itself was simply one-off.
- Redesigning the whole Skills feature before isolating which specific stage of the funnel is actually broken.
- Ignoring result consistency as a possible driver of low trust and repeat use.
Likely follow-up questions
- How would you tell a one-off task from a discoverability failure in the data?
- Which hypothesis would you test first with the least engineering effort?
More metrics questions
- What metrics would tell you whether Perplexity is winning against Google Search?Perplexity · Metrics · Hard
- Queries per user dropped 20% after a UI change. How do you investigate?Perplexity · Metrics · Medium
- You have one team and can materially advance only one of these areas in the next 6-12 months: Skills, collaboration, artifact generation, or Projects. How would you choose where to invest, and what leading indicators and longer-term outcome metrics would you use to defend that roadmap to leadership?Perplexity · Metrics · Hard
- Suppose Perplexity launches a new 'resume previous work' feature for research and productivity workflows. What metrics would you use to determine whether it improves retention, and how would you separate durable user value from short-term engagement spikes caused by novelty or accidental usage?Perplexity · Metrics · Hard
- Pick one Perplexity use case, research, investing, or shopping, and design a first prototype in Computer for a domain expert. What exact job-to-be-done would it solve, what would the user see and control step by step, and which early metrics would tell you the prototype is genuinely improving productivity?Perplexity · Metrics · Medium
- Perplexity cares about product-led growth and data-driven flywheels. For a new AI productivity feature inside Search or Computer, what user actions would you optimize first to drive activation, retention, and learning effects, and how would you know the feature is creating a real flywheel rather than a one-time novelty spike?Perplexity · Metrics · Hard
More questions from Perplexity
Learn the skill behind it
Chapters of the AI PM course that teach what this question tests.
- Chapter 9: Prove it paid off: outcomes, economics, and pricing
- Chapter 2: Data fluency: SQL, logs, and reading the truth yourself
- Chapter 14: Get the job: the AI PM interview loop