Metrics question

Define goals for a handheld gaming device for facebook. Follow up question: after 6 months, you see WAU increasing by 10% but session length decreasing by 12%, then would you still ship it?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Tests defining launch goals for a hardware bet plus judging a mixed post-launch signal, a classic engagement quality versus quantity trade-off.

Set initial goals: WAU growth, since hardware needs a critical mass of users, alongside session length as a proxy for genuine engagement.
Add a guardrail: device return rate and app-attach rate, since hardware sales alone don't prove real usage.
For the follow-up, don't treat WAU up and session length down as automatically good or bad; investigate why session length dropped.
Check if shorter sessions reflect more frequent, snackable play, which could be healthy, or a drop in per-session satisfaction, which is a warning sign.
Decide based on retention curves and satisfaction data: ship if new users are returning and rating well despite shorter sessions, hold if retention is also declining.

Refuses to answer the follow-up with a gut reaction, insisting on investigating the cause behind the session-length drop first.
Distinguishes healthy behavior change, more frequent short sessions, from a real quality problem, dropping satisfaction.
Ties the final ship decision to retention data, the metric that actually predicts long-term device success.

Automatically treating rising WAU as good news without investigating the session-length drop.
Making a ship or hold decision without checking retention or satisfaction data first.

What retention benchmark would make you comfortable shipping despite shorter sessions?
How would you separate casual new users from core gamers in this WAU growth?