Metrics question

Imagine that in your daily routine of checking conversion funnel, you realize a 20% drop in checkout funnel. What would you do?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Real-time diagnostic reasoning for a sudden funnel drop, prioritizing speed and hypothesis testing under pressure.

Confirm the drop is real, check for a tracking or reporting bug before assuming a genuine conversion problem.
Check for an immediate technical cause first, since a 20 percent drop is large enough to suggest a broken checkout, payment processor outage, or a recent deploy.
Segment the drop, by device, browser, payment method, and geography, to localize where the problem is concentrated.
Check recent changes, any deploy, pricing change, or third-party payment provider update in the last 24 hours.
If a bug is found, escalate for an immediate fix or rollback, given checkout is revenue-critical and time-sensitive.
Define a follow up, a postmortem to identify why the issue was not caught before release and add monitoring to catch it faster next time.

Treats a 20 percent drop as likely a technical incident first, given the scale and suddenness, rather than assuming a subtle UX issue.
Prioritizes checking recent deploys and payment provider status immediately, since checkout funnels are highly sensitive to small technical breaks.
Segments quickly by device and payment method to localize the fault before it spreads further.
Proposes a concrete incident response step, escalate and roll back, appropriate to the severity and revenue impact.

Treating a sudden 20 percent drop as a slow UX investigation rather than a likely incident needing urgent triage.
Not checking recent deploys or payment provider status first.
Proposing a long-term analysis process without addressing the urgency of the immediate drop.

Chapters of the AI PM course that teach what this question tests.