Metrics question

What metrics would you use to measure success in designing the estimated time of arrival?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Whether the candidate can define both accuracy and trust metrics for a prediction feature that directly affects user behavior.

Clarify the goal: ETA accuracy drives trust in the platform and reduces cancellations, so both prediction quality and behavioral impact matter.
Define the core accuracy metric: mean absolute error between predicted and actual arrival time, in minutes.
Add a directional bias check: whether errors skew toward overestimating or underestimating time, since underestimating erodes trust more.
Tie accuracy to behavior metrics: cancellation rate before pickup, and rider or customer satisfaction ratings tied to on-time arrival.
Set a guardrail for edge cases, such as ETA error during peak traffic or bad weather, where accuracy typically degrades most.

Separates raw accuracy (mean absolute error) from perceived trust (cancellations, ratings), since both matter differently.
Flags that underestimating ETA is worse than overestimating, since it breaks trust more visibly.
Names a specific edge case, peak traffic or weather, where accuracy needs separate monitoring.

Measuring only average accuracy without checking for a systematic overestimate or underestimate bias.
Ignoring how ETA errors affect cancellation rate and trust, not just the number itself.

How would you handle ETA accuracy differently for first time versus repeat users?
What would you do if accuracy is good on average but bad in dense urban cores?