Metrics question
Utilization went down by 45% on app XYZ in Italy for the month of August. Give a reason why and draft a plan to fix it.
Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.
Start a mock interview on this question · Mock interview from a job description
What this question tests
Tests root-cause diagnosis under a large, specific, localized metric drop and turning findings into an actionable fix plan, not just a list of hypotheses.
How to approach it
- Clarify 'utilization': confirm the exact definition (DAU/MAU, sessions per user, or feature usage rate) and reconfirm the 45% drop is measured correctly, not a tracking or data pipeline bug.
- Rule out an instrumentation error first: check if the analytics SDK, a recent app update, or a regional server issue could be under-reporting rather than reflecting real behavior.
- Check for external, Italy-specific causes: a competitor launch or promotion, a regulatory change (e.g., app store policy), a seasonal effect (August is peak vacation month in Italy, which could genuinely reduce app usage for many product types), or a local outage/payment processor issue.
- Check for product-specific causes: a recent release that specifically affected the Italian localization (translation bug, broken payment method, App Store listing issue in that region).
- Prioritize hypotheses by likelihood and ease of check: August seasonality is a strong, cheap-to-verify hypothesis (compare year-over-year August dips), so check that before assuming a product bug.
- Draft a fix plan conditional on root cause: if seasonal, plan a re-engagement campaign for September; if a bug, hotfix and add regional QA; if competitive, propose a localized win-back promotion.
What a strong answer includes
- Checks for instrumentation/measurement error first, a step many candidates skip but that senior interviewers specifically look for.
- Names August seasonality as a strong, Italy-specific, cheap-to-verify hypothesis (Italians commonly vacation heavily in August), showing real-world grounding.
- Separates hypotheses into external (seasonality, competitor, regulation) versus internal (bug, localization, release) causes for a structured investigation.
- Ties each root cause to a distinct, specific fix rather than one generic 'we'll investigate' answer.
Common mistakes
- Jumping straight to 'there must be a bug' without checking simpler explanations like seasonality or an instrumentation error.
- Not being specific to Italy or August, giving an answer that would apply to any country or month.
- Providing hypotheses with no prioritization or verification plan.
Likely follow-up questions
- How would you distinguish a real usage drop from a tracking bug quickly?
- What year-over-year data would you pull first?
- If it turns out to be seasonal, would you still take any action?
More metrics questions
- Spotify installs increased 25%. What are reasons this might have happened? Give at least four.PayPal · Metrics · Easy
- Set goals for Facebook notifications.Spotify · Metrics · Medium
- How would you measure the success of YouTube?Spotify · Metrics · Medium
- There are about 1 million inactive Netflix users. What would you do about them?Spotify · Metrics · Hard
- What are 10 important metrics for Spotify under engagement category? Which one would you improve? Why?Spotify · Metrics · Medium
- How do you measure the success of Airbnb online experiences?Spotify · Metrics · Medium
More questions from these companies
Learn the skill behind it
Chapters of the AI PM course that teach what this question tests.
- Chapter 9: Prove it paid off: outcomes, economics, and pricing
- Chapter 2: Data fluency: SQL, logs, and reading the truth yourself
- Chapter 14: Get the job: the AI PM interview loop