Metrics question

Venmo’s sign-up rate dropped to 2% after rolling out a new UX. What would you do?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Root cause diagnosis under pressure: separating measurement issues from real UX regressions, then proposing a fix path.

First verify the number: check if 2 percent is a tracking issue introduced by the new UX before assuming it is a real behavior change.
If confirmed real, break the funnel into stages, landing page view, form start, form completion, verification, to find exactly where the drop happens.
Compare the new flow to the old flow step by step for added friction, like extra fields or a broken step on certain devices.
Segment by device and channel, since a UX regression often hits one platform disproportionately.
Propose an immediate mitigation, rolling back or feature flagging off the new UX for the affected segment while root cause is confirmed.

Checks instrumentation first, a step many candidates skip, before concluding the drop is a real user behavior problem.
Uses funnel stage breakdown to localize exactly where the regression happens instead of treating the number as opaque.
Proposes an immediate mitigation given the severity, not just investigate and report back later.

Jumping straight to hypotheses about UX design without checking if tracking broke first.
No urgency or mitigation plan despite a severe, sudden metric drop.

What would you check first if the drop was isolated to one platform, like mobile web?
How would you prevent this kind of regression from shipping in the future?