Metrics question

Netflix's completion rates are dropping. How would you diagnose and solve this problem?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Metric diagnosis for a content engagement problem, structuring hypotheses across content, product and user segments.

Confirm the metric definition and measurement window, ruling out a tracking or reporting change first.
Segment completion rate by content type, genre, and release date to see if the drop concentrates in specific titles.
Segment by device and region to rule out a technical issue, like buffering or a broken player update.
Check for recent product changes, like autoplay settings, recommendation algorithm updates, or UI changes to the player.
Prioritize the strongest hypothesis and propose a fix, such as reverting a recent change or improving recommendations for the affected segment.

Rules out measurement error first, since completion rate can be skewed by a tracking bug before assuming real behavior changed.
Segments by content type and finds, for example, the drop concentrates in a specific genre or newly released titles, narrowing the cause.
Considers a product change hypothesis, like a recommendation algorithm update surfacing less relevant content, rather than only blaming content quality.

Jumping to a content quality conclusion without segmenting by device, region, or content type first.
Ignoring recent product or algorithm changes that commonly cause sudden completion rate shifts.

Chapters of the AI PM course that teach what this question tests.