Metrics question
Instagram is launching video calling. How would you test and measure its success?
Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.
Start a mock interview on this question · Mock interview from a job description
What this question tests
Launch testing strategy and metric design for a new communication feature entering a space with established competitors.
How to approach it
- Define the rollout plan, a phased test, for example a limited beta with a subset of close friends users before a full rollout.
- Define adoption metrics, percent of eligible users who try video calling at least once, and repeat usage rate within 30 days.
- Define quality metrics, call completion rate, call drop rate, and average call duration, since technical reliability is core to whether users keep using it.
- Define a retention angle, whether video calling adoption correlates with increased overall app session time or messaging frequency.
- Propose the test method, an A B test comparing users with access to video calling against a control group on 30 day retention and messaging engagement.
What a strong answer includes
- Proposes a specific phased test plan, close friends beta before full rollout, appropriate for a feature this dependent on network effects between the two calling parties.
- Names concrete reliability metrics, call completion and drop rate, recognizing that a video feature's success depends heavily on technical quality, not just adoption.
- Connects the feature to the core product, testing whether video calling adoption lifts overall messaging engagement and retention, not treating it as an isolated success metric.
Common mistakes
- Measuring only adoption with no attention to technical reliability metrics like call drop rate.
- No phased rollout plan for a feature that depends on both parties having access.
- Not connecting the new feature's success back to core app engagement and retention.
Likely follow-up questions
- How would you handle low adoption if only one side of a friendship has been given access during the beta?
- What would make you delay full rollout even if adoption metrics looked good?
More metrics questions
- How would you measure the success of Facebook Likes?Meta · Metrics · Medium
- How would you measure improvements made to Facebook messenger?Meta · Metrics · Easy
- Imagine you are the PM in charge of Reactions on Facebook - the new way to interact with posts by using “love”, “haha”, “wow”, “sad”, and “angry” reactions. What would success look like in terms of number of non-like reactions per post at launch and how do you come up with this? Would this number differ by reaction? Why or why not?Meta · Metrics · Hard
- You launched a new signup flow to encourage new users to add more profile information. A/B test results indicate that the % of people that added more information increased by 8%. However, 7 day retention decreased by 2%. What do you do?Google · Metrics · Hard
- Define the metrics for YouTube search.Google · Metrics · Medium
- You are the PM of Instagram app. The MAU has been constant but DAU has declined. What will you do?Meta · Metrics · Medium
More questions from these companies
Learn the skill behind it
Chapters of the AI PM course that teach what this question tests.
- Chapter 9: Prove it paid off: outcomes, economics, and pricing
- Chapter 2: Data fluency: SQL, logs, and reading the truth yourself
- Chapter 14: Get the job: the AI PM interview loop