Metrics question
What A/B tests would you run to increase the number of messages sent and received on WhatsApp?
Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.
Start a mock interview on this question · Mock interview from a job description
What this question tests
Experimentation design skill: forming testable hypotheses tied to a specific engagement goal.
How to approach it
- Clarify the goal: increase both messages sent and received, meaning tests should drive genuine two way conversation, not just one sided sends.
- Hypothesis 1: smarter reply suggestions reduce typing friction and increase reply rate, tested via an A/B on suggestion visibility.
- Hypothesis 2: read receipts increase pressure to reply quickly, tested by comparing reply latency with receipts on versus off.
- Hypothesis 3: group chat prompts (like polls) increase message volume in group threads specifically, tested on group chats only.
- For each test, define the primary metric (messages sent and received per user) and a guardrail (uninstall or opt out rate).
- Prioritize tests by expected impact and implementation cost, running the cheapest high confidence test first.
What a strong answer includes
- Frames each test as a specific hypothesis with a mechanism, not just 'test a new feature and see what happens'.
- Distinguishes sent from received explicitly, since a good test should drive genuine two way exchange, not one sided messaging.
- Gives an illustrative number, e.g. assumes reply suggestions could lift reply rate by 8 to 10 percent based on similar features elsewhere.
- Names a guardrail metric, opt out or uninstall rate, to catch a test that boosts messaging at the cost of annoyance.
Common mistakes
- Proposing tests with no clear mechanism connecting the change to the metric.
- Not distinguishing messages sent from messages received, missing half the stated goal.
- No guardrail metric to catch a test that increases messaging but annoys users.
Likely follow-up questions
- How would you prioritize these tests?
- What guardrail would make you stop a test early?
- How would you measure impact on group chats versus 1:1 chats?
More metrics questions
- How would you measure the success of Facebook Likes?Meta · Metrics · Medium
- How would you measure improvements made to Facebook messenger?Meta · Metrics · Easy
- Imagine you are the PM in charge of Reactions on Facebook - the new way to interact with posts by using “love”, “haha”, “wow”, “sad”, and “angry” reactions. What would success look like in terms of number of non-like reactions per post at launch and how do you come up with this? Would this number differ by reaction? Why or why not?Meta · Metrics · Hard
- You launched a new signup flow to encourage new users to add more profile information. A/B test results indicate that the % of people that added more information increased by 8%. However, 7 day retention decreased by 2%. What do you do?Google · Metrics · Hard
- Define the metrics for YouTube search.Google · Metrics · Medium
- You are the PM of Instagram app. The MAU has been constant but DAU has declined. What will you do?Meta · Metrics · Medium
More questions from these companies
Learn the skill behind it
Chapters of the AI PM course that teach what this question tests.
- Chapter 9: Prove it paid off: outcomes, economics, and pricing
- Chapter 2: Data fluency: SQL, logs, and reading the truth yourself
- Chapter 14: Get the job: the AI PM interview loop