Metrics question

Late deliveries lead to customer churn. What data we should look at to prove this hypothesis for a food delivery app?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Tests hypothesis-driven analysis: identifying the specific data needed to establish, not assume, a causal link between late deliveries and churn.

State the hypothesis precisely: late deliveries cause churn, versus both being driven by something else, like a bad restaurant partner.
Identify the datasets: order-level delivery time (actual vs promised ETA) joined to each customer's retention status afterward.
Compare churn rate for customers with a late delivery (e.g., 15+ minutes past ETA) versus those without, controlling for order frequency.
Check a dose-response pattern: does churn rise with lateness severity, which strengthens the causal case if monotonic.
Control for confounds: region, restaurant partner, and order value, since some regions may have other service issues too.
If correlational data is inconclusive, look for a natural experiment, like a region where delivery times improved, and check if churn dropped after.

Distinguishes correlation from causation explicitly and proposes a stronger causal check via a natural experiment.
Proposes a dose-response check, a specific and effective way to strengthen or weaken the hypothesis.
Names real confounds (region, restaurant partner, order value) instead of assuming a clean two-variable relationship.
Grounds the analysis in concrete, joinable datasets rather than a vague 'look at delivery and churn data.'

Treating a raw correlation as proof of causation without controlling for confounds.
Not defining 'late' with a threshold, making the analysis impossible to run.
No proposal for getting closer to causal evidence beyond one correlation check.

Chapters of the AI PM course that teach what this question tests.