Metrics question

You are PM of Medium and you notice some fluctuations in % of active users. After 3 weeks of monitoring, you notice one morning that the % of active users drop by 35%. What would you do to figure out the drop and how will you restore it?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Root-cause diagnosis of a sudden metric drop, distinguishing data or measurement issues from real user behavior issues.

First rule out a measurement or logging error: a 35% overnight cliff is more often instrumentation than real behavior.
If confirmed real, segment the drop by platform, region, and cohort, new versus returning, to localize where it's happening.
Check for an external cause: a recent release, a third-party login or API outage, a redefinition of 'active,' or a competitor or news event.
Correlate timing precisely with recent deploys, campaign changes, or infrastructure incidents.
Based on root cause, propose the fix: a hotfix rollback for a bug, or a targeted re-engagement push for genuine churn.
Define recovery as the active-user rate returning to baseline within a defined window.

Treats instrumentation error as the first, most likely hypothesis for a sudden cliff, not a gradual decline.
Segments systematically, platform, geography, cohort, before jumping to a fix.
Ties the investigation to a concrete recent change rather than speculating broadly.

Jumping straight to product fixes before confirming the drop is real and diagnosing its cause.
Not segmenting the data, missing that the drop might be isolated to one platform or region.

Chapters of the AI PM course that teach what this question tests.