Metrics question
Clickthrough rate for notifications are down by 10%. Why?
- Meta
- Metrics
- Medium
Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.
Start a mock interview on this question · Mock interview from a job description
What this question tests
Root cause diagnosis for a metrics regression: can you generate and prioritize hypotheses systematically rather than guessing the first plausible cause.
How to approach it
- Confirm the drop is real and not measurement noise, checking if it is consistent across platforms, regions, and notification types, or isolated to one segment.
- Check for a recent change first, since a sudden drop often correlates with a release, like a new notification grouping feature, permission prompt change, or an OS level update affecting delivery.
- Hypothesize content related causes: notification relevance dropped due to a ranking model change, or notification volume increased, causing fatigue and lower per notification attention.
- Hypothesize delivery related causes: a spike in notifications marked as spam or muted, or a platform level delivery delay reducing timeliness and relevance.
- Prioritize investigation by segment size, checking the largest affected segment first, since it likely explains most of the drop.
- Define the fix validation plan: once a cause is isolated and addressed, confirm CTR returns toward baseline in the affected segment specifically before declaring it resolved.
What a strong answer includes
- Checks for a recent change first, the highest probability cause for a sudden metric shift, before broader hypothesizing.
- Separates content causes (relevance, volume) from delivery causes (spam, delay), covering the two main failure categories.
- Prioritizes investigation by segment size rather than chasing every hypothesis equally.
Common mistakes
- Jumping to one guessed cause without checking for a recent product or platform change first.
- Investigating the entire user base as one group instead of segmenting to find where the drop actually concentrates.
Likely follow-up questions
- How would you distinguish a ranking model issue from a delivery issue?
- What would you do if the drop is isolated to one platform, like iOS?
More metrics questions
- How would you measure the success of Facebook Likes?Meta · Metrics · Medium
- How would you measure improvements made to Facebook messenger?Meta · Metrics · Easy
- Imagine you are the PM in charge of Reactions on Facebook - the new way to interact with posts by using “love”, “haha”, “wow”, “sad”, and “angry” reactions. What would success look like in terms of number of non-like reactions per post at launch and how do you come up with this? Would this number differ by reaction? Why or why not?Meta · Metrics · Hard
- You launched a new signup flow to encourage new users to add more profile information. A/B test results indicate that the % of people that added more information increased by 8%. However, 7 day retention decreased by 2%. What do you do?Google · Metrics · Hard
- Define the metrics for YouTube search.Google · Metrics · Medium
- You are the PM of Instagram app. The MAU has been constant but DAU has declined. What will you do?Meta · Metrics · Medium
More questions from Meta
Learn the skill behind it
Chapters of the AI PM course that teach what this question tests.
- Chapter 9: Prove it paid off: outcomes, economics, and pricing
- Chapter 2: Data fluency: SQL, logs, and reading the truth yourself
- Chapter 14: Get the job: the AI PM interview loop