Metrics question

You’re the PM for an e-commerce marketplace and you’re responsible for the recommendations feature. In the past quarter, you ran an A/B test for your new feature and you’re supposed to present the results to the execs. What metrics would you present to them?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Whether you can select and present metrics that tell executives the true business impact of a test, not just headline lift.

Clarify the test's goal, likely increasing purchase conversion or basket size through better recommendations.
Lead with the primary metric tied to that goal, such as conversion rate or revenue per session in test versus control.
Add supporting metrics that explain why it moved, like click-through rate on recommended items and average items per order.
Include guardrail metrics execs will ask about, such as return rate and customer complaints, to show the win is not hollow.
Report statistical confidence and sample size so execs know the result is reliable, not noise.
Translate the lift into a business number, like projected incremental revenue if rolled out to all traffic.

Leads with one clear primary metric instead of a long list, so the headline result is unambiguous.
Includes a guardrail metric like return rate, showing the lift did not come from over-recommending unsuitable items.
Translates statistical lift into a dollar estimate execs actually care about, not just a percentage.
States the confidence level and sample size plainly, avoiding overclaiming from a short or small test.

Presenting a wall of metrics with no clear primary result, leaving execs unsure what to conclude.
Reporting the lift without checking statistical significance or guardrail metrics like returns.

Chapters of the AI PM course that teach what this question tests.