Metrics question
How would you decide whether or not doubling the amount of video content in the home timeline was a good idea?
- Metrics
- Hard
Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.
Start a mock interview on this question · Mock interview from a job description
What this question tests
Tests designing an experiment and metric framework to evaluate a content-mix change rather than guessing at the outcome.
How to approach it
- Clarify the goal: is more video meant to increase total time spent, or is it a bet on video's higher engagement per impression.
- Design an A/B test: a treatment group with doubled video density in the timeline against a control with the current mix.
- Track primary metrics: total time spent and session frequency, since more video should increase watch time if it's genuinely engaging.
- Track guardrails: tweet or post creation rate, and diversity of content seen, since an all-video feed risks crowding out text and photo content.
- Decide based on whether time-spent gains outweigh any drop in content creation or session frequency over a multi-week window.
What a strong answer includes
- Frames this as a testable hypothesis with a clear control group, not a subjective opinion.
- Names the real risk explicitly, crowding out other content types and creator activity.
- Insists on a multi-week window to catch any novelty effect that fades over time.
Common mistakes
- Giving a yes or no opinion without proposing a way to actually test it.
- Ignoring the risk that more video could suppress text or photo content creation.
Likely follow-up questions
- How would you control for a novelty effect in the first week of the test?
- What result would make you kill this change even if time spent went up?
More metrics questions
- How would you measure the success of Medium?Twitter · Metrics · Medium
- How would you measure success of Twitter Lists?Twitter · Metrics · Medium
- How would you design an iron for a blind person?Twitter · Metrics · Medium
- How would you measure the performance and health of a Netflix Original?Microsoft · Metrics · Hard
- Imagine you are the PM in charge of Reactions on Facebook - the new way to interact with posts by using “love”, “haha”, “wow”, “sad”, and “angry” reactions. What would success look like in terms of number of non-like reactions per post at launch and how do you come up with this? Would this number differ by reaction? Why or why not?Meta · Metrics · Hard
- You launched a new signup flow to encourage new users to add more profile information. A/B test results indicate that the % of people that added more information increased by 8%. However, 7 day retention decreased by 2%. What do you do?Google · Metrics · Hard
More questions from Twitter
Learn the skill behind it
Chapters of the AI PM course that teach what this question tests.
- Chapter 9: Prove it paid off: outcomes, economics, and pricing
- Chapter 2: Data fluency: SQL, logs, and reading the truth yourself
- Chapter 14: Get the job: the AI PM interview loop