Metrics question
You are the PM at DoorDash, and you are about to release a new feature. A/B testing of that feature shows that the number of orders from users goes up, but the number of restaurants that were ordered from goes down (say from 100 to 90). Will you launch this feature or not?
- Meta
- Metrics
- Hard
Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.
Start a mock interview on this question · Mock interview from a job description
What this question tests
Tests metrics trade-off judgment on a real marketplace tension: can you reason about supply-side (restaurant) health versus demand-side (order volume) impact before deciding whether to launch.
How to approach it
- Restate the result: orders per user rose, but restaurant diversity dropped from 100 to 90, a 10 percent decline that needs investigation before a launch call.
- Form a hypothesis: the feature likely concentrates orders on fewer restaurants, for example a recommendation algorithm favoring popular or high-margin ones over the long tail.
- Assess the real impact: more orders helps near-term revenue, but concentration can hurt long-term health if long-tail restaurants churn from reduced order flow.
- Check for a guardrail breach: whether the restaurants that dropped off are systematically small or independent versus randomly distributed.
- Make a conditional call: if the drop concentrates on small partners with churn risk, run a segmented test first rather than launching broadly; if it is minor and non-systematic, launch with monitoring.
- Define the metric to monitor post-launch: restaurant partner retention and revenue per restaurant, not just order volume.
What a strong answer includes
- Avoids a flat yes or no, instead investigating the mechanism behind the drop first, the actual skill being tested.
- Distinguishes a random, minor concentration effect from one that systematically hurts small restaurants, since the implications differ.
- Names restaurant partner churn as a concrete guardrail to monitor post-launch rather than treating the A/B result as final.
- Weighs the short-term revenue gain against the platform's long-term dependency on restaurant supply diversity.
Common mistakes
- Giving a flat yes/no answer without investigating why restaurant count dropped.
- Ignoring that a marketplace needs supply-side health (restaurant diversity) as much as demand-side growth (orders).
- No plan to monitor the trade-off after launch if the decision is to proceed.
Likely follow-up questions
- How would you segment the data to check whether small restaurants are disproportionately affected?
- What test would you run to isolate the cause of the restaurant-count drop?
- How would you decide the point at which restaurant concentration becomes an unacceptable risk?
More metrics questions
- How would you measure the success of Facebook Likes?Meta · Metrics · Medium
- How would you measure improvements made to Facebook messenger?Meta · Metrics · Easy
- Imagine you are the PM in charge of Reactions on Facebook - the new way to interact with posts by using “love”, “haha”, “wow”, “sad”, and “angry” reactions. What would success look like in terms of number of non-like reactions per post at launch and how do you come up with this? Would this number differ by reaction? Why or why not?Meta · Metrics · Hard
- You launched a new signup flow to encourage new users to add more profile information. A/B test results indicate that the % of people that added more information increased by 8%. However, 7 day retention decreased by 2%. What do you do?Google · Metrics · Hard
- Define the metrics for YouTube search.Google · Metrics · Medium
- You are the PM of Instagram app. The MAU has been constant but DAU has declined. What will you do?Meta · Metrics · Medium
More questions from Meta
Learn the skill behind it
Chapters of the AI PM course that teach what this question tests.
- Chapter 9: Prove it paid off: outcomes, economics, and pricing
- Chapter 2: Data fluency: SQL, logs, and reading the truth yourself
- Chapter 14: Get the job: the AI PM interview loop