Metrics question
5% users met the same error message in the shopping cart page. What will you do to find the root cause? Provide a solution.
- Walmart
- Metrics
- Medium
Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.
Start a mock interview on this question · Mock interview from a job description
What this question tests
Tests structured debugging of a user-facing error at scale: distinguishing a widespread technical issue from segment-specific noise before proposing a fix.
How to approach it
- Quantify scope: 5 percent hitting the same error message suggests a specific, reproducible bug rather than random noise.
- Segment by platform, browser, payment method, and cart contents to see if the error concentrates in one path, such as a specific payment provider or promo code.
- Check recent changes: correlate the error's onset with a recent deploy, a third-party payment or inventory API change, or a promo campaign.
- Reproduce the error directly using the same segment conditions to confirm the root cause rather than relying on logs alone.
- Ship a fix or rollback for the identified cause, and add monitoring or alerting for this specific error code going forward.
- Communicate proactively to affected users if the error blocked checkout, since it directly affects revenue.
What a strong answer includes
- Uses the 5 percent figure as a clue: consistent enough to point to a specific cause, not so widespread that it is a full outage.
- Segments by concrete dimensions (payment method, promo code, platform) that are known common failure points in checkout flows.
- Correlates the error's start time with recent deploys or third-party API changes, which is the fastest way to find root cause in production.
- Treats this as revenue-critical and proposes both an immediate fix and longer-term monitoring, not just a one-time patch.
Common mistakes
- Investigating without first segmenting to find where the 5 percent concentrates.
- Fixing the symptom (suppressing the error message) instead of the underlying cause.
Likely follow-up questions
- What would you do differently if the error only appeared for one payment method?
- How would you prevent a regression like this from reaching 5 percent of users again?
More metrics questions
- Walmart's order return rate is increasing. As a product manager, what things would you look into to isolate the problem?Amazon · Metrics · Medium
- How do you define success for Yelp reviews?Google · Metrics · Medium
- Late deliveries lead to customer churn. What data we should look at to prove this hypothesis for a food delivery app?PayPal · Metrics · Medium
- One of the brands in the grocery category is not selling and has stocks left that are soon to expire. Specify all possible reasons for it and also suggest solutions for each of the reasons.Shopify · Metrics · Medium
- What is the most important metric for Uber Eats? Why?Uber · Metrics · Medium
- How do you convert offline customers to omnichannel customers who transact online for retail stores?Shopify · Metrics · Hard
More questions from Walmart
Learn the skill behind it
Chapters of the AI PM course that teach what this question tests.
- Chapter 9: Prove it paid off: outcomes, economics, and pricing
- Chapter 2: Data fluency: SQL, logs, and reading the truth yourself
- Chapter 14: Get the job: the AI PM interview loop