Metrics question
After a cross-device prompt campaign, overall paid conversion increases, but retention falls for a high-value segment. How would you diagnose the issue? Be specific about the first funnel cuts, cohort breakdowns, denominators, and time windows you would inspect, and how the findings would change your next shipping decision.
- Perplexity
- Metrics
- Hard
Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.
Start a mock interview on this question · Mock interview from a job description
What this question tests
Metrics diagnosis under a real tension: can you cut the data the right way to explain a segment-specific retention drop hiding behind an overall positive headline number.
How to approach it
- First cut: retention by the specific high-value segment flagged versus the rest of the base, confirming the drop is isolated and not an artifact of overall mix shift.
- Second cut: device and surface the campaign ran on for that segment, since a cross-device prompt can create friction, like repeated prompts or inconsistent state, more for some device combinations than others.
- Third cut: cohort by campaign exposure timing, comparing users converted right after the campaign to those converted through normal channels in the same window, to isolate the campaign's effect from a seasonal trend.
- Check denominators carefully: is retention measured as percentage of the high-value segment's original size, or as percentage of segment members who converted, since the campaign may have pulled in lower-intent members of that segment who convert but do not stick.
- Use the findings to decide the next shipping move: if the campaign is pulling in lower-intent members of a high-value segment, narrow targeting criteria rather than pulling the campaign entirely, since overall conversion gains may still be worth keeping for other segments.
What a strong answer includes
- Checks whether the campaign changed the composition of who converted in that segment, a specific and likely explanation the question is testing for.
- Names denominator ambiguity explicitly, since percentage of original segment size versus percentage of new converts tell very different stories.
- Proposes a targeted fix, narrowing criteria, rather than an all-or-nothing kill of a campaign that is working for other segments.
Common mistakes
- Looks only at the aggregate retention number without segmenting to find where the drop concentrates.
- Jumps to killing the whole campaign without checking whether the loss is isolated to a specific device, surface, or converted-cohort composition.
Likely follow-up questions
- How would you distinguish a targeting problem from a product friction problem in the cross-device flow.
- What time window would you use to decide the retention drop is real and not noise.
More metrics questions
- What metrics would tell you whether Perplexity is winning against Google Search?Perplexity · Metrics · Hard
- Queries per user dropped 20% after a UI change. How do you investigate?Perplexity · Metrics · Medium
- You have one team and can materially advance only one of these areas in the next 6-12 months: Skills, collaboration, artifact generation, or Projects. How would you choose where to invest, and what leading indicators and longer-term outcome metrics would you use to defend that roadmap to leadership?Perplexity · Metrics · Hard
- Users can create Skills in Computer, but many run a Skill once and never use it again. How would you determine whether the main issue is setup friction, weak discoverability, inconsistent results, narrow applicability, or lack of trust, and what product changes would you test first to improve repeat use and sharing?Perplexity · Metrics · Hard
- Suppose Perplexity launches a new 'resume previous work' feature for research and productivity workflows. What metrics would you use to determine whether it improves retention, and how would you separate durable user value from short-term engagement spikes caused by novelty or accidental usage?Perplexity · Metrics · Hard
- Pick one Perplexity use case, research, investing, or shopping, and design a first prototype in Computer for a domain expert. What exact job-to-be-done would it solve, what would the user see and control step by step, and which early metrics would tell you the prototype is genuinely improving productivity?Perplexity · Metrics · Medium
More questions from Perplexity
Learn the skill behind it
Chapters of the AI PM course that teach what this question tests.
- Chapter 9: Prove it paid off: outcomes, economics, and pricing
- Chapter 2: Data fluency: SQL, logs, and reading the truth yourself
- Chapter 14: Get the job: the AI PM interview loop