Metrics question
Suppose Perplexity launches a new 'resume previous work' feature for research and productivity workflows. What metrics would you use to determine whether it improves retention, and how would you separate durable user value from short-term engagement spikes caused by novelty or accidental usage?
- Perplexity
- Metrics
- Hard
Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.
Start a mock interview on this question · Mock interview from a job description
What this question tests
Tests defining retention metrics for a new feature and separating durable value from novelty-driven or accidental engagement spikes.
How to approach it
- Define the core retention metric: percent of users who use 'resume previous work' again within a set window after their first use, not just first-use adoption.
- Define a durability check: whether repeat usage persists at week 4 and week 8, not just week 1, since novelty spikes typically fade by then.
- Segment users by whether resuming actually changed their session behavior, for example completing a task they had previously abandoned, versus just reopening and leaving.
- Check for accidental usage by looking at time-on-resumed-session; a very short session after resuming suggests an accidental click, not genuine continuation.
- Compare retention for users of this feature against a matched cohort of similar users without it, to isolate the feature's incremental effect.
- Treat a spike that fades by week 4 to 8 as novelty, and a sustained pattern with real task completion as durable value.
What a strong answer includes
- Uses week 4 and week 8 persistence, not week 1, as the real test for durable value versus novelty.
- Checks session behavior after resuming, like task completion, not just the act of resuming, to rule out accidental clicks.
- Uses a matched cohort comparison to isolate the feature's incremental effect from general product engagement trends.
Common mistakes
- Declaring success based on first-week adoption alone.
- Not checking whether resumed sessions actually led to real task completion.
Likely follow-up questions
- How would you build a matched cohort if the feature rolled out to everyone at once?
- What would you do if retention is strong but time-on-resumed-session stays very short?
More metrics questions
- What metrics would tell you whether Perplexity is winning against Google Search?Perplexity · Metrics · Hard
- Queries per user dropped 20% after a UI change. How do you investigate?Perplexity · Metrics · Medium
- You have one team and can materially advance only one of these areas in the next 6-12 months: Skills, collaboration, artifact generation, or Projects. How would you choose where to invest, and what leading indicators and longer-term outcome metrics would you use to defend that roadmap to leadership?Perplexity · Metrics · Hard
- Users can create Skills in Computer, but many run a Skill once and never use it again. How would you determine whether the main issue is setup friction, weak discoverability, inconsistent results, narrow applicability, or lack of trust, and what product changes would you test first to improve repeat use and sharing?Perplexity · Metrics · Hard
- Pick one Perplexity use case, research, investing, or shopping, and design a first prototype in Computer for a domain expert. What exact job-to-be-done would it solve, what would the user see and control step by step, and which early metrics would tell you the prototype is genuinely improving productivity?Perplexity · Metrics · Medium
- Perplexity cares about product-led growth and data-driven flywheels. For a new AI productivity feature inside Search or Computer, what user actions would you optimize first to drive activation, retention, and learning effects, and how would you know the feature is creating a real flywheel rather than a one-time novelty spike?Perplexity · Metrics · Hard
More questions from Perplexity
Learn the skill behind it
Chapters of the AI PM course that teach what this question tests.
- Chapter 9: Measuring AI Product Economics
- Chapter 2: Using SQL and Logs
- Chapter 14: Interviewing for AI PM Roles