Metrics question
You are the PM responsible for the "Recommended for you" content strip in the Disney Plus app, and you have noticed a sudden 10% drop in total average minutes viewed. What do you do?
- Disney
- Metrics
- Hard
Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.
Start a mock interview on this question · Mock interview from a job description
What this question tests
Structured root-cause investigation for a recommendation surface's engagement drop, distinguishing content, algorithm, and technical causes.
How to approach it
- Quantify the drop: confirm it is specific to the recommendation strip versus overall app-wide viewing, and the exact timeframe.
- Segment by platform, region, and subscriber tenure to localize where the drop concentrates.
- Check for a recent change: an algorithm update, a content catalog change like a major title's licensing expiring, or a UI redesign of the strip.
- Check upstream causes: if fewer new titles were added recently, recommendations may feel repetitive and get skipped more.
- Compare against other content surfaces to see if viewing simply shifted elsewhere rather than declining overall.
- Propose a fix based on the diagnosed cause and track minutes viewed on the strip over the following weeks.
What a strong answer includes
- Considers content catalog changes like licensing expirations as a specific, real cause many candidates would miss.
- Distinguishes a genuine engagement decline from a shift to other surfaces, an important nuance before concluding real harm occurred.
- Segments by subscriber tenure, since recommendation quality perception often differs for new versus long-time users.
- Proposes a concrete recovery-tracking step to validate whatever fix is applied.
Common mistakes
- Jumping to blaming the algorithm without checking for content catalog or licensing changes.
- Not checking whether viewing simply moved to another content surface.
- Ignoring platform or regional segmentation that could localize the cause.
Likely follow-up questions
- How would you distinguish an algorithm regression from a content freshness problem?
- What would you do if the drop was concentrated among long-tenured subscribers only?
- How would you measure recommendation quality more directly going forward?
More metrics questions
- How would you measure the performance and health of a Netflix Original?Microsoft · Metrics · Hard
- Imagine you are the PM in charge of Reactions on Facebook - the new way to interact with posts by using “love”, “haha”, “wow”, “sad”, and “angry” reactions. What would success look like in terms of number of non-like reactions per post at launch and how do you come up with this? Would this number differ by reaction? Why or why not?Meta · Metrics · Hard
- You launched a new signup flow to encourage new users to add more profile information. A/B test results indicate that the % of people that added more information increased by 8%. However, 7 day retention decreased by 2%. What do you do?Google · Metrics · Hard
- Facebook Ads dropped by 20%. Mark Zuckerberg calls you and asks you to fix it. What do you do?Meta · Metrics · Hard
- Imagine you're the product manager for Facebook Marketplace. Since many sellers don't mark items as sold, what existing functionality and metrics could you use to determine whether an item has likely sold?Meta · Metrics · Hard
- Uber has seen a dip in the ride confirmation rate. As a PM at Uber, you are responsible to investigate and fix the issue. How will you go about it?Uber · Metrics · Hard
More questions from Disney
Learn the skill behind it
Chapters of the AI PM course that teach what this question tests.
- Chapter 9: Prove it paid off: outcomes, economics, and pricing
- Chapter 2: Data fluency: SQL, logs, and reading the truth yourself
- Chapter 14: Get the job: the AI PM interview loop