Metrics question

You noticed a drop in user engagement on recommended videos in Youtube, what metrics would you track to guide your next steps?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Building a metric tree to diagnose a recommendation system engagement drop.

Define the goal metric: engagement on the recommended video shelf, such as click through rate and watch time from that shelf.
Build the metric tree: impressions of recommendations, click through rate, watch completion rate, and downstream subscribe or like actions.
Check guardrails: overall session time and total watch time, to see if users shifted to search or other surfaces instead.
Segment by content category, device type, and new versus returning viewers to localize the drop.
Check for a recent change: an algorithm update, UI redesign of the shelf, or a content supply shift.
Propose the strongest hypothesis and the data needed to confirm it before proposing a fix.

Builds a clear metric tree from impressions down to watch completion rather than naming metrics in a random list.
Distinguishes a real engagement drop from users simply migrating to another surface like search, using overall watch time as a guardrail.
Proposes segmenting by content category, since a drop concentrated in one category points to a supply or ranking issue.
Names a recent-change hypothesis, like a ranking algorithm update, as a concrete first thing to check.

Naming a long list of metrics without organizing them into a tree from top to bottom.
Ignoring the possibility that users moved to a different surface rather than disengaging entirely.

Chapters of the AI PM course that teach what this question tests.