Behavioral question

Can you give us a few examples where you have planned and conducted A/B tests?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Practical experimentation experience, checking you understand test design fundamentals beyond just 'running A/B tests'.

Pick two distinct examples, ideally testing different types of changes, such as a UI change and a pricing or messaging change.
For each, state the hypothesis clearly, what you expected to happen and why, before describing the test itself.
Describe the test design: how you defined the control and variant, what metric you used as the primary success measure, and the sample size or duration.
State the result for each, including whether it was statistically significant, and be honest if a test was inconclusive or contradicted your hypothesis.
Describe the decision made based on the result, showing the test actually informed action, not just observation.
Mention any learning about test design itself, like a flawed metric choice or unexpected external factor, from either example.

Gives two genuinely different test types, showing range beyond a single repeated pattern, like one UI test and one pricing or messaging test.
States the hypothesis explicitly before the result, showing structured thinking rather than a post-hoc narrative.
Includes an honest inconclusive or negative result in at least one example, which is more credible than claiming every test succeeded.
Shows the test result led to a concrete decision or follow-up action, closing the loop rather than just reporting a number.

Describing tests with no clear hypothesis stated upfront.
Claiming every test was a clean, positive success, which reads as unlikely and rehearsed.
Not describing what decision was actually made based on the results.

Chapters of the AI PM course that teach what this question tests.