Metrics question

Weekly active users of Codex dropped 15% after a pricing change. How do you investigate?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Structured metrics investigation skill: isolating cause from a single correlated event.

Confirm the drop is real: check for measurement issues, like a tracking change coinciding with the pricing change, before assuming a true behavioral drop.
Segment the drop: break WAU decline down by user tier, usage volume, and geography to see if it is concentrated or uniform.
Form hypotheses: price-sensitive light users churned entirely, or heavy users reduced frequency but did not fully leave, which look different in the data.
Check the funnel: compare new signups, reactivations, and churned users separately, since a 15 percent WAU drop could be almost entirely reduced signups, not churn.
Cross-reference with support tickets or survey data mentioning price to confirm the pricing change is the actual driver, not a coincidental separate issue.

Insists on segmenting the drop by user cohort before concluding pricing caused it, since a confound like a competing product launch could coincide.
Separates churn from reduced frequency, since these require very different fixes, one needs win-back offers and the other needs value reinforcement.
Proposes a candidate check, like comparing the drop size against the assumed price-sensitive segment size, to see if the magnitude is plausible.
Recommends a fast qualitative check, like exit surveys for canceled accounts, to validate the pricing hypothesis with direct evidence.

Assuming correlation equals causation without segmenting or checking for confounds like seasonality or a concurrent product change.
Treating WAU as one number instead of decomposing it into new, retained, and churned users.

Which user segment would you expect to be most price sensitive, and how would you confirm that?
What would you recommend if the drop is concentrated in your most profitable segment?