Metrics question

Facebook recently launched a new feed algorithm. As a result, the average session duration dropped by 20%. What would you do?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Diagnosing an intentional launch's unintended metric impact, and questioning whether the drop is actually bad.

Confirm the drop is attributable to the algorithm launch by checking timing and any A/B test data.
Question whether shorter sessions are actually bad, since more relevant content could mean faster satisfaction.
Segment the drop by user cohort to see who's most affected.
Check secondary metrics: session frequency, content quality ratings, and ads served per session.
Name the real business risk: fewer ad impressions per session hurts revenue.
Propose a next step: a holdback comparison or a partial rollback for the affected segment.

Questions the assumption that shorter sessions are automatically bad, tying it to satisfaction and quality metrics.
Uses the fact that it's a known launch to check existing A/B test data directly rather than guessing.
Names the real business risk: shorter sessions likely mean fewer ad impressions, the actual guardrail.
Proposes a clear next step, a holdback comparison or partial rollback, instead of ending on 'investigate more'.

Assuming session duration drop is automatically bad without checking satisfaction or quality signals.
Not using the fact that this was a tested launch to check existing experiment data first.

Chapters of the AI PM course that teach what this question tests.