Metrics question

Rooms on Facebook was launched 3 months ago. Would you also launch it on Instagram? How would you run a A/B test to see the success of this feature on instagram?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Cross platform launch judgment paired with rigorous A/B test design, covering hypothesis, metrics, and statistical validity.

State the decision criteria: launch on Instagram only if Rooms shows strong retention and usage depth on Facebook first.
Review the Facebook data available after three months, such as weekly active room creators and session length.
If signals are positive, recommend an Instagram launch, since its audience skews toward casual video hangouts already.
Design the A/B test: randomly assign users to see the Rooms entry point or not, measuring adoption and downstream engagement.
Define the primary metric, such as rooms created per active user, plus a guardrail like time spent on core Instagram features.
Determine sample size and test duration needed for statistical significance given Instagram's daily active user base.

Uses the Facebook launch data as real evidence to inform the Instagram decision, rather than treating it in isolation.
Designs a proper randomized test with a clear primary metric and a guardrail against cannibalizing core engagement.
Names a specific illustrative sample size or duration, such as two weeks reaching a target confidence level.
Distinguishes short term adoption from a longer term retention signal before declaring the feature a success.

Proposes launching without referencing the existing Facebook data as a decision input.
Describes an A/B test with no primary metric or guardrail defined.

How would you handle a result that is positive but not statistically significant?
What guardrail metric would concern you most in this test?
How would you decide the test duration given Instagram's usage patterns?